Skip to main content

AWS Solutions Library

Guidance for Data Transfer from Amazon S3 Glacier Vaults to Amazon S3

Open guide
Go to sample code

Overview

This Guidance demonstrates how to automate data transfers to simplify management and enhance both accessibility and cost-effectiveness of archived data. It shows how to automatically restore, copy, and transfer Amazon Simple Storage Service (Amazon S3) Glacier vault archives to S3 buckets and desired storage classes, including S3 Glacier storage classes. This automation saves time and minimizes the likelihood of human error during data transfer, helping ensure a more reliable and consistent operation for managing archived data.

How it works

These technical details feature an architecture diagram to illustrate how to effectively use this solution. The architecture diagram shows the key components and their interactions, providing an overview of the architecture's structure and functionality step-by-step.

Download the architecture diagram

Deploy with confidence

Everything you need to launch this Guidance in your account is right here

We'll walk you through it

Dive deep into the implementation guide for additional customization options and service configurations to tailor to your specific needs.

Open guide

Let's make it happen

Ready to deploy? Review the sample code on GitHub for detailed deployment instructions to deploy as-is or customize to fit your needs. 

Go to sample code

Well-Architected Pillars

The architecture diagram above is an example of a Solution created with Well-Architected best practices in mind. To be fully Well-Architected, you should follow as many Well-Architected best practices as possible.

This Guidance automates the process of copying archives from Amazon S3 Glacier vaults to S3 buckets, reducing manual effort and the risk of errors to improve operational efficiency. Moving data to different Amazon S3 storage classes enables storage cost optimization based on access patterns and retention requirements. The pre-built CloudWatch dashboard visualizes copy operation progress, providing better visibility into the data transfer process and enabling effective monitoring and troubleshooting.

Read the Operational Excellence whitepaper 

Lambda is a serverless compute service, which helps reduce the attack surface and responsibilities associated with managing underlying infrastructure. This minimizes user involvement in managing and securing compute resources, improving the overall security posture.

Read the Security whitepaper 

The pre-built CloudWatch dashboard provides visibility into the data transfer process, allowing you to monitor progress and identify potential issues or bottlenecks. This enhanced visibility enables you to quickly detect and address reliability-related problems, helping ensure successful completion of data transfers. By using a serverless compute service that automatically scales and manages the underlying infrastructure, you can reduce the risk of infrastructure-related failures or performance degradation.

Read the Reliability whitepaper 

Lambda functions are triggered based on events, such as the initiation of the data transfer process. The event-driven nature of Lambda functions optimizes performance by only executing necessary compute resources when required. This helps reduce overall resource utilization and improving efficiency. The automatic scaling and management of underlying infrastructure helps ensure that necessary compute resources are allocated on-demand.

Read the Performance Efficiency whitepaper 

By allowing users to move data to different Amazon S3 storage classes, this Guidance enables storage cost optimization based on access patterns and retention requirements. This helps reduce overall storage costs by placing frequently accessed data in performance-optimized storage classes while moving less frequently accessed data to more cost-effective storage classes. Lambda helps optimize costs by only charging for compute time used, rather than requiring users to manage and pay for underlying infrastructure.

Read the Cost Optimization whitepaper 

Lambda reduces energy consumption and carbon footprint associated with managing and maintaining underlying infrastructure. Serverless computing leads to more efficient resource utilization and potentially lower energy usage compared to traditional server-based architectures. 

Read the Sustainability whitepaper 

Disclaimer

The sample code; software libraries; command line tools; proofs of concept; templates; or other related technology (including any of the foregoing that are provided by our personnel) is provided to you as AWS Content under the AWS Customer Agreement, or the relevant written agreement between you and AWS (whichever applies). You should not use this AWS Content in your production accounts, or on production or other critical data. You are responsible for testing, securing, and optimizing the AWS Content, such as sample code, as appropriate for production grade use based on your specific quality control practices and standards. Deploying AWS Content may incur AWS charges for creating or using AWS chargeable resources, such as running Amazon EC2 instances or using Amazon S3 storage.