AWS Partner Network (APN) Blog
Modernize COBOL Workloads with Amazon EKS powered by Generative AI
By: Todd Berson, CTO – BSC Analytics
By: Nestor Gandara, Principal Partner SA – AWS
 |
| BSC Analytics |
 |
COBOL has powered mission-critical banking, insurance, and government systems for over sixty years. Despite its continued reliability, around 85% of COBOL applications run on outdated mainframe infrastructure that’s expensive, difficult to scale, and isolated from modern cloud services. Organizations today face a pressing question: how do we modernize these essential applications without rewriting millions of lines of code?
Amazon Elastic Kubernetes Service (Amazon EKS) offers an answer. By containerizing COBOL applications and running them on EKS, businesses can preserve their existing logic while gaining access to modern cloud infrastructure’s flexibility, scalability, and efficiency. When combined with the capabilities of large language models (LLMs), enterprises can further extend the value of their COBOL data by introducing automation, real-time analytics, and actionable insights.
Driving Business Value with AI-Powered COBOL
COBOL systems excel at producing structured, rules-based data, but they weren’t built to extract trends or generate intelligence from that data. This is where LLMs shine. Imagine a COBOL-based financial system generating thousands of transaction logs daily. Traditionally, analysts would comb through these reports manually. LLM integration can summarize those logs automatically, highlighting anomalies and even flagging potential fraud. In the insurance world, COBOL-driven systems produce raw claim data that agents translate into formal reports. With LLMs, that transformation is automated, drastically reducing time and improving accuracy. Even in customer service, where COBOL systems support backend operations, LLMs can bridge the gap by powering real-time, chatbot-driven interactions that respond using legacy data.
Re-platforming COBOL with Amazon EKS
Amazon EKS provides a fully managed Kubernetes environment ideal for running containerized COBOL workloads. Unlike traditional mainframes, which require significant upfront investment and rigid scaling, EKS offers an elastic infrastructure that adjusts dynamically based on workload demands. For batch processes standard in COBOL systems, Kubernetes CronJobs makes scheduling straightforward, enabling enterprise-scale automation. Adding services like Amazon EFS for persistent file storage ensures COBOL workloads can read and write data without code changes. Integration with Amazon RDS enables COBOL data to interact with relational cloud services.
Security and compliance are paramount for COBOL workloads, which often handle sensitive financial and government data. EKS meets this challenge by offering fine-grained access controls through AWS IAM, encryption at rest via AWS Key Management Service, and private networking using AWS PrivateLink. With multi-AZ deployment, workloads are distributed across availability zones for high availability, reducing the risk of downtime.
Observability is also improved. Amazon CloudWatch, AWS X-Ray, and Amazon Managed Services for Prometheus provide detailed metrics and traceability, giving teams visibility into how legacy workloads perform in the cloud.
Augmenting COBOL Workflows with Generative AI
Once COBOL workloads are containerized and running on EKS, they become eligible for a new class of enhancements. In a typical deployment, COBOL batch jobs generate structured output—like CSV files or fixed-width reports—that is written to a persistent file system such as Amazon EFS. From there, a containerized microservice can pick up this output, format it into JSON or prompt-based formats, and pass it along to an LLM for analysis.
These LLMs don’t replace COBOL logic—they work alongside it. The processed insights can then be stored in Amazon S3 or a relational database, served to dashboards, or delivered via APIs to downstream business applications. This architecture enables legacy systems to drive modern business value without the overhead of complete rewrites.
Modernization in Action: A Condensed Customer Success Story
A major automotive supply company relied on legacy COBOL applications to run its critical business operations. These applications were hosted on aging, on-premises infrastructure that was becoming increasingly expensive and unreliable. Due to a single point of failure, the company faced performance bottlenecks multiple times a day, rising maintenance costs for legacy hardware, and unacceptable risks.
The company modernized with Amazon EKS in partnership with BSC Analytics. Reliability improved with high availability across multiple availability zones. After decommissioning the hardware and moving to elastic cloud infrastructure, costs reduced by 18%. By introducing LLMs to analyze COBOL-generated reports, the company gained faster insights into operations, streamlining inventory management and speeding up order processing.
Figure 1 comparison table showing the transformation from Legacy COBOL to Modernized COBOL on EKS. That illustrates the evolution from an aging mainframe system to a containerized cloud solution, highlighting multiple benefits including a 30% cost reduction, increased availability, automated insights, and elastic scaling capabilities
Figure 1 – Cobol modernization comparison
Implementation at a Glance
The Amazon EKS cluster was deployed with two node groups to enable COBOL workloads and LLM services. Standard compute instances for COBOL processing and one GPU-enabled node group for running self-hosted LLM models. Both node groups used encrypted Amazon EBS volumes for storage, and IAM roles ensured secure AWS service access.
Amazon EFS provided persistent storage for COBOL data, which was deployed with Terraform and configured with mount targets across availability zones. The COBOL code was not transformed—we used GnuCOBOL, an open-source COBOL compiler, which allowed us to run the source code directly in a Linux-based container. The environment was containerized using Docker, and no emulation was required. This approach reduced migration complexity while enabling us to deploy COBOL applications on Amazon EKS alongside modern LLM services. Each application was deployed via Helm charts and managed with ArgoCD, ensuring consistent and scalable delivery across environments.
Supporting services included ExternalDNS for managing DNS records, the AWS Load Balancer Controller for traffic ingress, and the Nvidia device plugin to enable GPU access within pods.
Best Practices for Long-Term Success
Preprocessing is essential when integrating AI with COBOL workflows. Legacy output must be translated into formats suitable for LLMs, typically JSON or structured prompts. A modular design keeps COBOL processing and AI enhancement in separate services, enabling independent scaling and debugging.
Security should remain top of mind. Use IAM roles to restrict access, encrypt data at rest and in transit, and avoid exposing internal workloads to the public internet. Monitoring is equally essential. Cloud-native observability tools offer visibility into processing pipelines, allowing teams to track COBOL execution and AI processing performance in real-time.
Looking Ahead
Modernizing COBOL with Amazon EKS is more than a cost-saving initiative—it’s a way to bring decades-old systems into the future. By combining COBOL’s reliability with Kubernetes’s scalability and LLMs’ intelligence, enterprises can keep the strengths of their legacy code while introducing entirely new capabilities.
This approach opens doors to innovation. Predictive analytics can identify system bottlenecks before they happen. Natural language interfaces can simplify access to legacy data. Even automated code translation, using tools like Amazon Q Developer, can accelerate further modernization in the future.
Conclusion
The modernization of COBOL workloads is no longer a question of “if”—it’s a question of “how soon.” Amazon EKS offers the cloud-native foundation to run these critical applications flexibly and confidently. LLM integration adds the intelligence layer to make legacy systems functional and transformative.
By embracing this strategy, organizations gain cost efficiency, real-time analytics, improved security, and the agility to evolve. More importantly, they extend the lifespan—and increase the business value—of systems that still lie at the heart of their operations.
Modernization doesn’t have to mean replacement. With Amazon EKS and generative AI, COBOL’s next chapter is just beginning.
Ready to modernize your legacy systems without starting over? Contact BSC Analytics to explore how we can help you containerize, deploy, and enhance your COBOL workloads with AI—fast, securely, and at scale.
.
BSC Analytics – AWS Partner Spotlight
BSC Analytics is an AWS Premier Partner with Generative AI Competency, empowers enterprises with end-to-end cloud, data, and AI solutions—leveraging services like Amazon EKS, SageMaker, and Redshift to deliver scalable architectures, predictive insights, and real-time intelligence across modern infrastructures.