Skip to main content

AWS Solutions Library

Guidance for Writing High Transaction Workloads on Amazon S3 Express One Zone

Overview

This Guidance demonstrates how the unique features of the Amazon S3 Express One Zone storage class can decrease network latency and make the writing of high throughput transactions both instantaneously scalable and fast. S3 Express One Zone is ideal for any application where it's important to minimize the latency required to access an object. This can be human-interactive workflows, like video editing, where creative professionals need responsive access to content from their user interfaces. S3 Express One Zone offers additional advantages to analytics and machine learning workloads that share similar responsiveness requirements from their data, particularly those workloads that require numerous smaller accesses or large numbers of random accesses. It’s compatible with other AWS services that support data analysis as well as artificial intelligence and machine learning (AI/ML) workloads, such as Amazon EMR, Amazon SageMaker, and Amazon Athena.

How it works

This architecture diagram demonstrates how the unique features of Amazon S3 Express One Zone can decrease network latency and make writing higher throughput transactions scalable and fast.

Download the architecture diagram

Well-Architected Pillars

The architecture diagram above is an example of a Solution created with Well-Architected best practices in mind. To be fully Well-Architected, you should follow as many Well-Architected best practices as possible.

S3 Express One Zone is a high-performance, single-Availability Zone S3 storage class that is purpose-built to deliver consistent single-digit millisecond data access for your most latency-sensitive applications. S3 Express One Zone can improve data access speeds by 10x and reduce request costs by 50% compared to S3 Standard and scales to process millions of requests per minute for your most frequently accessed datasets. Applications can benefit immediately from requests completing up to an order of magnitude faster. S3 Express One Zone provides similar performance elasticity to other S3 storage classes. As with S3, you don’t need to plan or provision capacity or throughput requirements in advance. You can scale your storage up or down, based on need, and access your data through the Amazon S3 APIs. 

Read the Operational Excellence whitepaper

S3 Express One Zone supports both IAM authorization and session-based authorization. With S3 Express One Zone, you authenticate and authorize requests through a new session-based mechanism, S3 CreateSession, which is optimized to provide the lowest latency. You can use CreateSession to request temporary credentials that provide low latency access to your bucket. These temporary credentials are scoped to a specific S3 directory bucket. Once authenticated with CreateSession, data in the S3 Express One Zone storage class is strongly consistent and always stored in S3 directory buckets, with S3 Block Public Access always enabled on directory buckets. All data stored in S3 Express One Zone is automatically encrypted using Server-Side Encryption (SSE-S3).

Read the Security whitepaper

S3 Express One Zone is one of several S3 storage classes that you can choose from based on the data access, data redundancy, and cost requirements of your workloads. S3 Express One Zone is designed for 99.95% availability within a single Availability Zone backed by the Amazon S3 Service Level Agreement. S3 Express One Zone objects are stored in a single AWS Availability Zone that you choose. In the unlikely case of the loss or damage to all or part of an Availability Zone, data in an S3 Express One Zone storage class may be lost. For example, events like fire and water damage could result in data loss. Apart from these types of events, our S3 Express One Zone storage classes use similar engineering designs as our Regional storage classes to protect objects from independent disk, host, and rack-level failures, and each are designed to deliver 99.999999999% data durability.

Read the Reliability whitepaper

S3 Express One Zone uses a unique architecture to optimize for performance and deliver consistently low request latency. S3 Express One Zone stores data on high-performance hardware, and its object protocol has been enhanced to streamline authentication and metadata overhead. While you have always been able to choose a specific AWS Region to store your S3 data, with S3 Express One Zone you can select the specific AWS Availability Zone within an AWS Region to store your data. You can choose to co-locate your storage with your compute resources in the same Availability Zone to further optimize performance, which helps lower compute costs and run workloads faster. Additionally, to further increase access speed and support hundreds of thousands of requests per second, data is stored in a new bucket type: an Amazon S3 directory bucket. Each directory bucket can support hundreds of thousands of transactions per second (TPS), regardless of the key names or access patterns.

Read the Performance Efficiency whitepaper

S3 offers a range of storage classes that you can choose from based on the data access, data redundancy, and the cost requirements of your workloads. S3 Express One Zone does not require pre-provisioning, so you only pay for what you use. It will optimize storage for high-performance workloads, reduce timeout (503 responses), expedite queries, and reduce your compute costs for performance demanding applications. S3 Express One Zone is built for high transaction rates and typically has lower API request costs than other S3 storage classes. The volume of storage billed in a month is accrued based on total storage used per hour, measured in gigabyte per month (GB-Month). Request costs are based on the size of the request. You are charged a per-request fee for access based on the request type—such as PUTs and GETs. An additional per-GB fee will be charged for the portion of the request size exceeding 512 KB. 

Read the Cost Optimization whitepaper

S3 Express One Zone helps you to scale your storage up or down based on your requirements. You no longer need to manage multiple storage systems for low latency workloads because your applications access data through the same S3 APIs you use today. S3 Express One Zone can grow to petabyte scale, supporting any size workload while keeping a minimal resource footprint. S3 directory buckets can scale up to hundreds of thousands of transactions per second to meet your most demanding workloads. It also offers faster, more scalable storage performance that can reduce CPU and GPU idle time from CPU or GPU idle workloads, optimizing your compute costs.

Read the Sustainability whitepaper

Disclaimer

The sample code; software libraries; command line tools; proofs of concept; templates; or other related technology (including any of the foregoing that are provided by our personnel) is provided to you as AWS Content under the AWS Customer Agreement, or the relevant written agreement between you and AWS (whichever applies). You should not use this AWS Content in your production accounts, or on production or other critical data. You are responsible for testing, securing, and optimizing the AWS Content, such as sample code, as appropriate for production grade use based on your specific quality control practices and standards. Deploying AWS Content may incur AWS charges for creating or using AWS chargeable resources, such as running Amazon EC2 instances or using Amazon S3 storage.