Many government agencies rely on critical applications running on legacy systems, which is highlighted in a 2023 report by the Government Accountability Office (GAO). These applications are typically monolithic, built on outdated technologies that necessitate workarounds, lack scalability, and incur escalating operational costs. The GAO report identified numerous federal IT legacy systems in urgent need of modernization. To address these challenges, Booz Allen Hamilton, an Amazon Web Services (AWS) Partner, offers innovative solutions using multi-tenant architectures. These solutions aim to reduce costs and save time across multiple projects. This post delves into one such mission-critical application developed for a large-scale federal agency, showcasing how modern cloud technologies can transform legacy government systems. By one estimate, the agency is saving $1.25 million annually by moving the application off of their mainframe, based on the reduction in the number of MIPS (millions of instructions per second) that the application was using.

Problem statement

The core challenge faced by the agency was an aging Java-based application hosted in an on-premises data center, tethered to mainframe databases running on decades-old infrastructure. This legacy system had become increasingly difficult to maintain and integrate with other applications, needing numerous workarounds to preserve its functionality. The outdated technologies not only severely limited future modernization possibilities, but also restricted the application’s ability to scale beyond its initial design parameters. Furthermore, the agency found itself allocating an ever-growing portion of its IT budget to maintaining this legacy system, leaving little room for innovation. As licensing and operational costs continued to escalate, it became apparent this approach was unsustainable, hindering the agency’s ability to adopt new technologies and improve its service delivery.

Approach and agency buy-in

To address the agency’s challenges, Booz Allen Hamilton proposed a strategic migration of the application to AWS, with the objective of removing mainframe dependencies and gradually transitioning to a cloud-native, scalable architecture. Recognizing the complexity and mission-critical nature of the application, Booz Allen Hamilton devised a multi-year, incremental approach to gain agency buy-in and support operational continuity during the transition process. The migration strategy encompassed three principal phases. The first, Replatforming, involves moving the Java application to the cloud with minimal modifications, migrating the mainframe database to Amazon Aurora MySQL, and establishing a continuous data replication pipeline between on-premises and the cloud. The objective of phase two is to Refactor the application to support horizontal scaling and high availability in the cloud. The third phase involves Re-Architecting it by decomposing the monolithic application into independent microservices.

This phased approach was crucial in garnering support from both IT services and business users, helping to immediately reduce costs while creating a solid foundation for future enhancements by using modern cloud technologies. As part of our strategy, we also developed systems and interfaces capable of operating in both on-premises and cloud environments at the same time during the transition period, making sure of uninterrupted service delivery until the modernization process was complete.

Challenges and solution

Many applications that are running on legacy infrastructure aren’t immediately ready to be migrated to the cloud following a Rehosting (lift and shift) approach. This application was built using Java and the agency wanted to containerize it when moving to the cloud. One of the advantages of containerization is that the original hosting environment can be faithfully replicated, which allows the application to continue to run in the cloud with few modifications. However, the original codebase wasn’t designed to work with multiple instances of the application running at the same time, an essential requirement for achieving resiliency in the cloud by deploying multiple pods. Furthermore, the application is integrated into a larger group of applications, many of which must remain on-premises for an extended period until they can be migrated to the cloud. In turn, the application needed to be integrated with several on-premises services and maintain continuous data synchronization with on-premises data.

Using containerization

A key objective of the application’s migration to the cloud was to use the existing containerization features built on Amazon Elastic Kubernetes Service (Amazon EKS) by Booz Allen Hamilton in partnership with the agency. These features provide a managed service of common cloud infrastructure and capabilities to development teams such as security, compute, databases, and release management. This allowed the application team to focus on migration concerns while leaving infrastructure, operations, security, and deployment of the environments to the infrastructure team (highlighted in the following figure).

Figure 1. Migration acceleration using a fully-managed containerization platform

The infrastructure team is responsible for deploying and managing EKS clusters for multiple tenants along with tools that provide common container functionality. This includes cross-cutting concerns such as secrets management, monitoring, logging, network configuration, cluster software and security updates, and operations support. Moreover, the application team was responsible for containerizing the application and making modifications to the codebase to make it work in the cloud.

The infrastructure team also provides essential capabilities for CI/CD release pipelines and common security controls. Preconfigured pipelines are integrated end-to-end from application code repositories to multiple cloud environments that automate the release process and help accelerate application migrations. The application team transformed existing repositories and build processes to conform to the enterprise standards of the new environment. Another advantage of using existing infrastructure functionality was to accelerate the security and ATO process by inheriting security controls. The application team only had to document application-specific configurations and customizations over and above what the multitenant infrastructure services provide to all tenants.

Replatforming the application

Migrating the application necessitated more than taking a Rehosting (lift-and-shift) approach, because some aspects wouldn’t function if moved as-is to the cloud. Booz Allen Hamilton Replatformed the application by containerizing the codebase and operating environment, moving to modern build and code repository tools, transforming the data access layer, and Refactoring external connections to other systems both on-premises and in the cloud.

Another key objective was to Replatform the application with as few changes as possible in order to accelerate migrating it off of legacy infrastructure. We containerized by using a multi-stage build process incorporating base images that were designed to match the on-premises hosting, operating system, and runtime environments. Mimicking the on-premises environment allowed Booz Allen Hamilton to have greater confidence in build reliability, limit testing, and remediation. Avoiding rework down the road is important because future Refactoring of the application is planned for later phases of the program. Encapsulating the legacy application in a single container allowed Booz Allen Hamilton to focus on higher value objectives in the current phase.

As with a lot of legacy applications, the deployment of code was linked to legacy delivery processes and tools. In this case, the codebase was stored in an SVN repository, which had to be migrated to GitHub to use the agency’s managed cloud services and integrated CI/CD pipelines. Furthermore, ongoing maintenance and enhancements of the application necessitated Booz Allen Hamilton to manage production releases to the legacy environment while coordinating updates with the new codebase and pipelines in the cloud. Booz Allen Hamilton established a branching strategy in SVN to continue ongoing updates on-premises while pushing periodic and cumulative updates to GitHub while the application was being migrated. The legacy application was also linked to an ANT build process, which had to be migrated to Maven to support the new environment. Both of these efforts helped accelerate the migration without either committing to future rework or adding unnecessarily to the timeline.

A large part of the migration effort went into transforming the databases from a legacy mainframe system to Aurora MySQL in AWS. The application’s data layer in particular needed significant Refactoring to work with the new MySQL database in the cloud while making as few changes to the application code as possible. Most of the application’s Java code was migrated unchanged. However, the data access object layer (DAO) had dependencies on mainframe constructs throughout the codebase that had to be Refactored to work with MySQL.

The application also maintains multiple connections with external systems that needed to be Refactored as part of the replatforming process. Some of these systems are legacy on-premises systems and others are modern cloud-based ones. Booz Allen Hamilton also integrated the application with a new cloud-based software as a service (SaaS) authentication service: a standardized service for cloud-based applications, a connection to an existing on-premises mail server, and a legacy document management system.

Although a multi-pod deployment was planned, the legacy code unfortunately wasn’t structured using a provider model, and the changes needed to support it would have rippled through the codebase and exceeded the schedule. The changes needed would have been too extensive to Replatform the application. Moreover, future optimization phases were envisioned, thus it was decided to deploy the application as a single node. Functionality such as caching, session management, and Cron jobs in particular were written with code that is dependent on specific features of the underlying hosting service. Booz Allen Hamilton proposed a single-node deployment that gave the agency the flexibility to defer further updates until a later Refactoring phase, keeping the current project on schedule.

Replatforming the data

The data presented the largest technical challenge, because it had to be migrated from mainframe databases shared with other applications, and this meant that not all of the data could be moved directly to the cloud. Booz Allen Hamilton analyzed the data to determine which data the application owned but needed to provide to on-premises applications as read-only, data on-premises applications owned that this application needed a copy of in the cloud, which data the migrated application needed to update on-premises, and which data on-premises systems needed to update to the cloud.

The data migration involved a combination of a one-time bulk data load using multiple data migration tools. The bulk data load was accomplished using a two-hop solution where the data was first extracted from mainframe databases on-premises into an intermediary relational database. This was necessary because the AWS Database Migration Service (AWS DMS) didn’t have a way to connect directly to the mainframe. The extraction had to be performed by a mainframe data extraction tool and the data loaded into Microsoft SQL Server. From there, AWS DMS made a connection and loaded the data into an Amazon Aurora MySQL database.

Figure 2. ‘Replatforming the data’ high-level architecture

The reliance on shared data with other on-premises applications necessitated the synchronization of data between cloud and on-premises databases. However, a significant challenge arose due to the limitations of proprietary mainframe tools for achieving bi-directional data synchronization. To address this issue, Booz Allen Hamilton developed a separate solution for each direction of data flow. A considerable volume of data is synchronized from the mainframe databases to the Amazon Aurora MySQL database, which is accomplished at the data layer through the use of a mainframe data extraction tool and AWS DMS. Conversely, the requirements for synchronizing data from Amazon Aurora MySQL back to the mainframe are minimal, and this is managed through a write-back API.

Conclusion

The migration of the application marked the initial step in transitioning a series of interconnected applications to the cloud. It has also paved the way for future migrations through a gradual, incremental strategy. Taking a phased approach, as opposed to overhauling the application all at once, allowed Booz Allen Hamilton to give the agency the flexibility to migrate the application with minimal modifications and make sure of its delivery on time and on budget.

There are many benefits to the agency. Reducing the workload on costly mainframe systems has resulted in significant cost savings. In the past, the legacy database and its drivers caused frequent connectivity issues that needed nightly restarts and even some daytime restarts each month. After migrating the application, it hasn’t encountered any major outages, reducing application downtime by 70% by lowering the time needed to take the application offline while performing maintenance from 8-10 hours a month on-premises to 1-2 hours/month in the cloud. There has also been a noticeable increase in response times of around 50% as seen in a recently sampling of 12 test screens and some core backend jobs. Migrating the application to an agency-managed set of cloud services has also led to much faster deployment cycles by around 85%, because it now takes less than an hour to deploy the application as compared to the 6-8 hours it used to take on-premises. The migration also decreased reliance on manual disaster recovery backups, and a more dependable and resilient infrastructure, ultimately enhancing customer satisfaction. The agency regards this migration as a signature success and a blueprint for a series of future application migrations.

About the AWS + Booz Allen Hamilton partnership

Booz Allen Hamilton and Amazon Web Services are teaming up to build next-gen, end-to-end solutions for defense, healthcare, intelligence, and beyond. The solutions combine Booz Allen Hamilton’s mission expertise and proprietary tech with the power and scale of AWS to turn raw data into real-time decisions that fast track results for America—at the edge, at speed, and at scale. Together, we’re not only ready for the future—we’re building it.

About Booz Allen Hamilton

Booz Allen Hamilton is the advanced technology company delivering outcomes with speed for America’s most critical defense, civil, and national security priorities. We build technology solutions using AI, cyber, and other cutting-edge technologies to advance and protect the nation and its citizens. By focusing on outcomes, we enable our people, clients, and their missions to succeed, accelerating the nation to realize our purpose: Empower People to Change the World®.