Streamlining EC2 Updates by Automating AMI Swaps

Introduction

Maintaining Amazon Machine Images (AMIs) current within Auto Scaling Groups (ASGs) is crucial for ensuring operational efficiency and security. This blog post outlines an automated solution for updating ASGs with the latest golden AMI after code deployments or system patching. By leveraging AWS Step Functions and AWS Backup events, we propose a streamlined process for AMI replacement, thus mitigating the risks associated with outdated AMIs.

The following example list a series of tasks that need automation:

Initial creation of Microsoft Windows-based AMI
Code deployment
Update AMI whenever there are new versions of application code and configuration
Creating a new golden, hardened AMI
Updating ASGs with the latest golden, hardened AMI

AWS Step Functions offers a reliable, serverless approach to automating infrastructure updates. This orchestration service simplifies the update process, leading to enhanced operational efficiency. Implementing Step Functions reduces the potential for human error and ensures consistently up-to-date infrastructure with the latest application versions.

Pre-requisites

To access the AWS Lambda function code and Step Function definition, retrieve the Git repository at https://github.com/aws-samples/example-code-for-automation-with-amazon-ec2-auto-scaling. Using Step Functions enhances operational efficiency, ensuring consistent deployment and maintenance of infrastructure with the latest application versions.

Automation Workflow

The process follows these key steps:

An AWS Backup job triggers the initial process
AMI creation triggers an EventBridge rule, which launches a Step Function
The Step Function orchestrates multiple Lambda functions:

– GetASGAndLaunchTemplate: Retrieves configuration details

– Sysprep: Prepares instances for AMI creation

– UpdateASG: Updates launch template versions

– Status Check Functions: Monitor instance and AMI states

– Cleanup: Removes unused resources

Figure 1 shows the architecture of the automation in our solution.

Figure 1: Flow Dig. Of Automation workflow

Implementation Details

Lambda function 1: GetASGAndLaunchTemplate

This Lambda function handles the initial setup by:

Extracting instance IDs from ARNs
Retrieving instance details
Reading ASG configurations
Managing launch template information
Creating backup AMIs

{"resourceArn": "arn:aws:ec2:region:account-id:instance/i-0123456789abcdef0"  }  # Get the resource ARN from the event      
 resource_arn = event['resourceArn']     # Extract the instance ID from the resource ARN      
 Instance_id = resource_arn.split('/')[-1]

To create this function, use the following steps:

Open the AWS Lambda console
Select Create function
Choose Author from scratch
Name the function (e.g., GetASGAndLaunchTemplate),
Choose the Python runtime
In the code editor, add the function code. Use the get-asg-and-launch-template_v3.py script from the GitHub repository you cloned earlier
Define Lambda function permissions with an execution role. Add the following permissions through the AWS Identity and Access Management (IAM) console or attach an inline policy.
Ensure the Lambda function execution role has permissions to use AWS Systems Manager (SSM). The role must have the following policies:
1. AmazonEC2ReadOnlyAccess
2. AmazonAutoScalingReadOnlyAccess

Lambda function 2: Sysprep:

This Lambda function manages the operating system preparation by:

Initializing Amazon Elastic Compute Cloud (EC2) and AWS Systems Manager client
Executing sysprep commands using Run Command, a tool in AWS Systems Manager
Managing timeout configurations

Repeat steps 1 to 7 as shown earlier to create a new AWS Lambda function and then, in the code editor, add the function code. Use the sysprep_v1.py script from the GitHub repository you cloned earlier.

Ensure the Lambda execution role has permissions to use SSM. The role must have the following policies:

AmazonEC2RoleforSSM
AmazonSSMFullAccess

Lambda function 3: UpdateASG:

This Lambda function handles ASG updates through:

Updating the auto scaling group with a new AMI
Parameter extraction from input events and validation
Launch template version management
AMI comparison and updates
ASG configuration updates

Repeat steps 1 to 7 as shown earlier to create a new AWS Lambda function and then, in the code editor, add the function code. Use the updateASG_v1.py script from the GitHub repository you cloned earlier.

Add the following permissions through the IAM console or attach an inline policy:

ec2:DescribeLaunchTemplateVersions
ec2:CreateLaunchTemplateVersion
ec2:ModifyLaunchTemplate
autoscaling:UpdateAutoScalingGroup.

Lambda functions 4 and 5: Status Check Functions

Two functions to ensure smooth operations:

check-instance-state: Monitors instance availability
check-ami-status-function: Tracks AMI creation status

Repeat steps 1 to 7 as shown earlier to create a new AWS Lambda function and then, in the code editor,add the function code. Use the check-instance-state_v1.py script from the GitHub repository you cloned earlier.

Follow the previous steps to create another function (e.g., check-ami-status). In the code editor, add the function code. Use the check-ami-status-function_v1.py script from the GitHub repository you cloned earlier.

Ensure the Lambda execution role from these two functions includes the AmazonEC2ReadOnlyAccess policy.

Lambda function 6: Cleanup

Manages resource cleanup by removing:

Temporary instances
Unused AMIs
Other residual resources

Repeat steps 1 to 7 as shown earlier to create AWS Lambda function and then, in the code editor, add the function code. Use the Cleanup_v1.py from the GitHub repository you cloned earlier.

Ensure the Lambda execution role has permissions to call terminate_instances and deregister_image by including the following permissions:

ec2:TerminateInstances
ec2:DeregisterImage

Setting Up the Infrastructure

Step Function Configuration

Create a state machine that orchestrates the Lambda functions using the provided ASL (Amazon State Language) JSON template.

Open the AWS Step Functions Console and navigate to State Machines using the left navigation menu. Select Create state machine. In the code editor, add the function code. Use the ASL (Amazon State language) JSON file “StepFunction_v4” from the GitHub repository you cloned earlier. This script will create a state machine like the one in Figure 2.

Figure 2: Visual Definition of State Function

Amazon EventBridge Rule Setup:

Configure EventBridge to:

Monitor AWS Backup events
Trigger the Step Function workflow
Transform and pass relevant input data

After you create the Step Function, you will create an Amazon EventBridge Rule to invoke it. Open the Amazon EventBridge Console and select Rules from the left navigation menu.

Select Create rule. Enter the rule name, description and event bus (default) as shown in Figure 3, then select Next.

Figure 3: create Rule Page

Under Build event pattern, use the following code (in JSON format) to configure the source pattern.

{

“source”: [“aws.backup”],

“detail-type”: [“Backup Job State Change”]

}

In the Target section, select AWS Service for Target Type and Step Functions state machine in the target. (as shown in the Figure 4)

Figure 4: configuring Target pattern in EventBridge Rule

Choose the name of State Function you created. Under Additional settings, select Input transformer for Configure target input. (as shown in Figure 5)

EventBridge Rule Transformer. Figure 5: EventBridge Rule Input Transformer

Here, we are using the following input transformer. (as shown in Figure 6)

Figure 6: EventBridge Rule Input Transformer

Skip the Configure tags step, go to Review and create and select Create rule to create the Event Bridge Rule.

AWS Backup

Schedule regular backups or create on-demand backups of application servers. Successful backup completion triggers the automation workflow, as shown in following figure.

Figure 7: Creating on-demand backup

Once the AWS Backup job runs for your EC2 Instance and the job state changes to Completed, EventBridge will execute the sysprep process. Verify the AMI ID in the ASG launch template under Latest version. This should match with backup AMI created by the Step Function.

AWS Step Functions empowers applications with robust automation capabilities, guaranteeing reliability, scalability, and ease of management. This facilitates the creation of complex workflows that integrate diverse AWS services, enabling businesses to adapt rapidly to evolving operational requirements and bolstering their agility within the cloud ecosystem.

Cleanup

This Automation process use an AWS Step Function and six Lambda Functions – the dynamic duo that orchestrates the entire process. One of the most attractive features? You only pay for what you use. These serverless components sit quietly in your AWS account, not burning through your budget while idle. They charge only when the automation runs.

But what about all those resources created during the automation process? Here’s where it gets interesting. The solution implements a cleanup mechanism using a dedicated Lambda function which deletes resources once they’ve served their purpose. This approach ensures your AWS environment stays tidy and cost-efficient.

And when you’re completely done with the automation? No worries. You can easily remove the Step Functions and Lambda Functions through the AWS Console with just a few clicks. It’s as simple as selecting the components and hitting delete.

Conclusion

AWS Step Functions offer a robust solution for automating the creation of Amazon Machine Images (AMIs) and the subsequent update of Auto Scaling Groups (ASGs). This orchestration service minimizes manual intervention, leading to increased operational efficiency and improved consistency within the infrastructure

Automated workflows facilitate the standardized creation and management of Amazon Machine Images (AMIs), thereby mitigating the potential for human error. Seamless integration with diverse Amazon Web Services (AWS) offerings enables streamlined updates to launch configurations and templates, ensuring Auto Scaling Groups (ASGs) consistently utilize the most current images.

AWS Step Functions enhance process robustness through integrated error handling and monitoring features, facilitating swift remediation of emergent issues. Consequently, adoption of this service promotes a more resilient and agile cloud infrastructure, freeing development teams to concentrate on innovative endeavors instead of manual intervention. Ultimately, leveraging this automation contributes to enhanced application uptime and performance.

The presented automation strategy effectively implements AWS Step Functions to manage intricate workflows and facilitate integration with diverse AWS services. By leveraging serverless automation capabilities, organizations can optimize cloud operations and enhance overall efficiency.

—————————————————————————————————————————————————————

Migration & Modernization