AWS Database Blog
Why Regeneron chose Amazon RDS Custom for Oracle to deploy COTS and GxP applications on AWS
This is a guest post by Prerak Raghuvanshi, Principal IT Engineer at Regeneron Pharmaceuticals, in partnership with AWS.
Regeneron, a leading biotechnology company, effectively harnesses traditional on-premises solutions with a sophisticated database architecture to bolster essential commercial-off-the-shelf (COTS) and GxP business applications. The company strategically aligns with vendor recommendations tailored to approved use cases, thereby ensuring optimal performance and compliance for their lab management, clinical, and manufacturing systems. In choosing a database engine for these business applications and SAS-based solutions, Regeneron navigates a focused selection process. Despite facing some vendor-imposed constraints, their dedicated database engineering team excels in maintaining high standards and implementing innovative solutions within these parameters. This approach underscores Regeneron’s commitment to leveraging cutting-edge technology while adhering to industry standards and best practices.
As part of Regeneron’s cloud-first strategy, the database engineering team has adopted Amazon Relational Database Service (Amazon RDS) as their preferred option for migrating or deploying various COTS and GxP applications on AWS. This approach helps them lower operational overhead, optimize costs, and enhance availability. For legacy COTS and GxP applications that need customization of the database, underlying server, and operating system configurations, which are not supported on RDS for Oracle, they utilize Amazon RDS Custom as the standard solution.
The database engineering team at Regeneron implemented a leading manufacturing COTS application on Amazon RDS Custom for Oracle to streamline operational overheads in the manufacturing process. This allowed Regeneron to automate essential database administration tasks, providing database administrators the flexibility to customize both the database environment and operating system to precisely fit the unique needs of their applications. The automation of backups and other operational tasks not only reinforce data security and improved resilience against potential failures but also greatly simplified deployment complexities traditionally associated with on-premises systems. Furthermore, the scalability offered by RDS Custom ensures that Regeneron’s database infrastructure can optimally adapt to their ongoing business growth, showcasing their commitment to innovation and operational excellence.
In this post, we highlight why Regeneron chose to use Amazon RDS Custom for Oracle to deploy COTS and GxP applications on AWS. This decision underscores their commitment to advancing from a legacy architecture to a robust, scalable, and resilient managed service. By doing so, Regeneron not only enhances their backend database infrastructure but also ensures adherence to GxP procedures, demonstrating their dedication to operational excellence and regulatory compliance.
Key features of RDS Custom for Oracle
Regeneron needed to deploy a manufacturing planning and scheduling application to support complex bio-pharma manufacturing processes. This application come with vendor provided custom database scripts to create logical structure and varies by application. Many of these scripts need to be run with elevated privileges such as ‘SYSDBA’ in the database and create various application specific object structures.
RDS Custom for Oracle provides added flexibility to migrate legacy on-premises COTS applications to the cloud by enabling specialized customizations to both the underlying database and operating system. This managed service is particularly valuable for Oracle industry specialized applications in sectors like healthcare and life sciences, combining the benefits of database automation with the customization capabilities needed for complex enterprise workload
Some key benefits of RDS Custom include the ability to automate many of the same administrative tasks as Amazon RDS, such as lifecycle management of databases, automated backups and point-in-time recovery (PITR), fully managed read replicas, monitoring the health of RDS Custom DB instances, and observing changes to the infrastructure, OS, and database processes. Also, additional benefits that are specific to RDS Custom are the capability to configure third-party applications, install custom patches, gain elevated privileges to install legacy applications, as well as the ability to create your own custom automation.
Solution overview
The following diagram shows the key components of the RDS Custom architecture. The RDS Custom DB instance resides in a virtual private cloud (VPC) and consists of the EC2 instance, instance endpoint, OS installed on the EC2 instance, and Amazon Elastic Block Store (Amazon EBS) for any additional file systems.
With RDS Custom, you have to use your own database media. This service model is known as Bring Your Own Media (BYOM). To get started, you need to create a custom engine version (CEV), which is a binary volume snapshot of a database engine and a specific Amazon Machine Image (AMI). For Oracle databases, the process involves storing the necessary installation files and patches in an Amazon Simple Storage Service (Amazon S3) bucket. RDS Custom then uses these files along with the AMI to create the CEV. When the CEV is ready, you can create an RDS Custom for Oracle DB instance, which will be based on the CEV you provided. To get started with CEV creation, refer to Working with custom engine versions for Amazon RDS Custom for Oracle.
RDS Custom has external automation and monitoring software that communicates with agents on the DB instance and other components within the RDS Custom environment. This software is responsible for collecting metrics, performing backups, sending notifications, performing automatic instance recovery, and resolving issues with the underlying EC2 instance. You can pause automation mode when performing customizations to prevent unintended interference with RDS Custom automation. RDS Custom provides full access to the host for you to make changes. As long as the changes don’t interfere with RDS Custom automation, the host will be within the support perimeter; otherwise, it will be outside the support perimeter. You will be notified when a host is outside the support perimeter.
Application architecture
Regeneron used Amazon RDS Custom for Oracle to deploy a manufacturing planning and scheduling application that needed admin rights on the database server. The primary database instance is deployed in one Availability Zone (AZ), and another standby instance is configured in a different AZ. Oracle Data Guard synchronizes the primary and standby instances in real-time. Users access the application using Amazon AppStream 2.0. AWS Direct Connect is being used to provide a private network connection between Regeneron’s corporate network and AWS.
Challenges and lessons learned
This was Regeneron’s first RDS Custom deployment. During configuration and testing, they learned quite a few things that they’re excited to share with the community. RDS Custom for Oracle provides elevated privileges (such as SYSDBA) to access the database and perform custom installations on the RDS Custom host, which allowed them to meet their specific application requirements. During the process, they had to learn a new way to apply modifications and customizations on the host by turning off the automation, making changes, and then turning it back on. For more information about common maintenance best practices you need to follow, refer to Make configuration changes to an Amazon RDS Custom for Oracle instance: Part 1. In this section, we discuss lessons learnt throughout the implementation.
Initial Setup
Before you create an RDS Custom for Oracle instance, make sure all the prerequisites, including the required AWS Identity and Access Management (IAM) policies, are implemented. Also, modify any service control policies (SCPs) implemented in your account as needed to allow these IAM policies.
RDS Custom instances create their own AWS CloudTrail log group. If you want to use another existing or new CloudTrail group, we recommend reaching out to AWS Support and opening a support case to allow this.
Patching
Amazon RDS Custom for Oracle provides flexible database patching capabilities through the OPatch utility, offering two distinct approaches to maintain their database environments.
Our recommended approach leverages Custom Engine Versions (CEVs), which streamlines patch management across multiple database instances. In this approach we create a new CEV containing the desired Oracle Database Release Update, then modify the DB instances to adopt this version. This method proves particularly valuable for managing multiple instances, as a single CEV can be applied consistently across the database fleet.
For time-sensitive situations requiring immediate patch deployment, RDS Custom also supports manual “one-off” patching. This approach involves temporarily pausing RDS Custom automation, applying patches directly to the database binaries on the underlying EC2 instance using OPatch, and then resuming automation. While effective for urgent scenarios where CEV creation timelines don’t align with business needs, this method is recommended only when standard CEV workflows cannot meet critical patching deadlines.
Operations
You should not modify the default shell at the system level because that might interfere with the RDS automation and your instance may be put outside the support perimeter. If you need to run any script using a different shell (such as using a Korn shell instead of the default bash shell), we recommend doing so at the session level to run any specific commands or scripts rather than changing it at the host level. Regeneron enhanced its database security by implementing Amazon RDS Custom for Oracle’s comprehensive encryption capabilities, utilizing AWS Key Management Service (AWS KMS) for data-at-rest encryption and TLS for data-in-transit protection, creating a robust security framework for its sensitive database workloads.
Backup
Regeneron uses their own enterprise data protection tool,Druva to seamlessly integrate with RDS custom for Oracle. In this way we are able to store backups for long-term to meet our compliance requirement as well as using the same data protection tool for on-prem, self-managed and RDS databases for a unifying data protection experience. We are taking RMAN backup of the RDS Custom EC2 using Druva Direct to Cloud (DTC) and also snapshot backups of the RDS instance
Monitoring
We have added the RDS custom for Oracle instance as a target for the Oracle Enterprise Manager (OEM) monitoring tool. This let us incrementally modernize our database environment while keeping the same monitoring tool used for on-premise and cloud environment. We could also leverage existing database performance tool, SolarWinds DPA, to perform our query tuning and optimization workflows. The flexibility of RDS custom for Oracle to integrate with existing observability and data protection tools accelerated our launch time and reduced the need of re-skilling our workforce.
Overall, RDS Custom for Oracle provided us a lot of flexibility to run legacy applications that need elevated database privileges and customization, but any customization you’re performing on the instance must be tested in a lower environment to validate that they are fully supported by the RDS automation. For more information about common customizations you might want to implement, refer to the series Make configuration changes to an Amazon RDS Custom for Oracle instance. You can also use AWS CloudFormation based automation to implement a Multi-AZ configuration for achieving high availability for deploying mission-critical applications on RDS Custom.
Summary
With the successful deployment of the first GXP COTS application on Amazon RDS Custom for Oracle, Regeneron is now able to concentrate more on strategic initiatives like cloud and database architecture design, rather than routine tasks such as managing on-premises backups. The distinctive feature of Amazon RDS Custom, which allows for customized deployment with elevated access, has expedited Regeneron’s cloud migration by leveraging managed services. As Regeneron plans to migrate additional applications to AWS, they intend to utilize RDS Custom (including RDS Custom for SQL Server) for applications that require elevated privileges or host-level customizations, where traditional Amazon RDS may not suffice. By simplifying production deployment and maintenance, Amazon RDS Custom aids in reducing lead times, thereby enabling organizations to swiftly respond to customer needs and rapid market changes.
Try out RDS Custom to enhance your own legacy on-premises solutions, and let us know your feedback and questions in the comments section.