Amazon S3 FAQs

A vector bucket is purpose-built for storing and querying vectors. Within a vector bucket, you do not use the S3 object APIs, but rather dedicated vector APIs to write vector data and query it based on semantic meaning and similarity. You can control access to your vector data with the existing access control mechanisms in Amazon S3, including bucket and IAM policies. All writes to a vector bucket are strongly consistent, which means that you can immediately access the most recently added vectors. As you write, update, and delete vectors over time, S3 vector buckets automatically optimize the vector data stored in them to achieve the optimal price-performance, even as the data sets scale and evolve.

A bucket is a container for objects and tables stored in Amazon S3, and you can store any number of objects in a bucket. General purpose buckets are the original S3 bucket type, and a single general purpose bucket can contain objects stored across all storage classes except S3 Express One Zone. They are recommended for most use cases and access patterns. S3 directory buckets only allow objects stored in the S3 Express One Zone storage class, which provides faster data processing within a single Availability Zone. They are recommended for low-latency use cases. Each S3 directory bucket can support up to 2 million transactions per second (TPS), independent of the number of directories within the bucket. S3 table buckets are purpose-built for storing tabular data in S3 such as daily purchase transactions, streaming sensor data, or ad impressions. When using a table bucket, your data is stored as an Iceberg table in S3, and then you can interact with that data using analytics capabilities such as row-level transactions, queryable table snapshots, and more, all managed by S3. Additionally, table buckets perform continual table maintenance to automatically optimize query efficiency over time, even as the data lake scales and evolves. S3 vector buckets are purpose-built for storing and querying vectors. Within a vector bucket, you use dedicated vector APIs to write vector data and query it based on semantic meaning and similarity. You can control access to your vector data using the existing access control mechanisms in Amazon S3, including bucket and IAM policies. As you write, update, and delete vectors over time, S3 vector buckets automatically optimize the vector data stored in them to achieve the optimal price-performance, even as the data sets scale and evolve.

There are no set up charges or commitments to begin using Amazon S3. At the end of the month, you will automatically be charged for that month’s usage. You can view your charges for the current billing period at any time by logging into your Amazon Web Services account, and selecting the 'Billing Dashboard' associated with your console profile. With the AWS Free Usage Tier*, you can get started with Amazon S3 for free in all Regions except the AWS GovCloud Regions. Upon sign up, new AWS customers receive 5 GB of Amazon S3 Standard storage, 20,000 Get Requests, 2,000 Put Requests, and 100 GB of data transfer out (to internet, other AWS Regions, or Amazon CloudFront) each month for one year. Unused monthly usage will not roll over to the next month. Amazon S3 charges you for the following types of usage. Note that the calculations below assume there is no AWS Free Tier in place.

Starting July 15, 2025, new AWS customers will receive up to $200 in AWS Free Tier credits, which can be applied towards eligible AWS services, including Amazon S3. At account sign-up, you can choose between a free plan and a paid plan. The free plan will be available for 6 months after account creation. If you upgrade to a paid plan, any remaining Free Tier credit balance will automatically apply to your AWS bills. All Free Tier credits must be used within 12 months of your account creation date. To learn more about the AWS Free Tier program, refer to AWS Free Tier website and AWS Free Tier documentation.

You can get started with S3 Vectors in four simple steps, without having to set up any infrastructure outside of Amazon S3. First, create a vector bucket in a specific AWS Region through the CreateVectorBucket API or in the S3 Console. Second, to organize your vector data in a vector bucket, you create a vector index with the CreateIndex API or in the S3 Console. When you create a vector index, you specify the distance metric (Cosine or Euclidean) and the number of dimensions a vector should have (up to 4092). For the most accurate results, select the distance metric recommended by your embedding model. Third, add vector data to a vector index with the PutVectors API. You can optionally attach metadata as key value pairs to each vector to filter queries. Fourth, perform a similarity query using the QueryVectors API, specifying the vector to search for and the number of the most similar results to return.

You can create a vector index using the S3 Console or the CreateIndex API. During index creation, you specify the vector bucket, index, distance metric, dimensions, and optionally a list of metadata fields that you want to exclude from filtering during similarity queries. For example, if you want to store data associated with vectors purely for reference, you can specify these as non-filterable metadata fields. Upon creation, each index is assigned a unique Amazon Resource Name (ARN). Subsequently when you make a write or query request, you direct it to a vector index within a vector bucket.

You can add vectors to a vector index using the PutVectors API. Each vector consists of a key, which uniquely identifies each vector in a vector index (e.g. you can programmatically generate a UUID). To maximize write throughput, it is recommended that you insert vectors in large batches, up to the maximum request size. Additionally, you can attach metadata (for example, year, author, genre, and location) as key value pairs to each vector. When you include metadata, by default all fields can be used as filters in a similarity query unless specified as non-filterable metadata at the time of vector index creation. To generate new vector embeddings of your unstructured data, you can use Amazon Bedrock’s InvokeModel API, specifying the model ID of the embedding model you want to use.

You can use the GetVectors API to look up and return vectors and associated metadata by the vector key.

You can run a similarity query with the QueryVectors API, specifying the query vector, the number of relevant results to return (the top k nearest neighbors), and the index ARN. When generating the query vector, you should use the same embedding model that was used to generate the initial vectors stored in the vector index. For example, if you use Amazon Titan Text Embeddings v2 in Amazon Bedrock to generate embeddings of your documents, it is recommended that you use the same model to convert a question to a vector. Additionally, you can use metadata filters in a query, to search vectors that match the filter. When you run the similarity query, by default the vector keys are returned. You can optionally include the distance and metadata in the response.

S3 Vectors offers highly durable and available vector storage. Data written to S3 Vectors is stored on S3, which is designed for 11 9s of data durability. S3 Vectors is designed to deliver 99.99% availability with an availability SLA of 99.9%.

S3 Vectors delivers sub-second query latency times. It uses the elastic throughput of Amazon S3 to handle searches across millions of vectors and is ideal for infrequent query workloads.

For performing similarity queries for your vector embeddings, several factors can affect average recall, including the embedding model, size of the vector dataset (number of vectors and dimensions), and the distribution of queries. S3 Vectors delivers over 90% average recall for most datasets. Average recall measures the quality of query results—90% means the response contains 90% of the ground truth closest vectors, that are stored in the index, to the query vector. However, because actual performance may vary depending on your specific use case, we recommend conducting your own tests with representative data and queries to validate that S3 vector indexes meet your recall requirements.

You can see a list of vectors in a vector index with the ListVectors API, which returns up to 1,000 vectors at a time with an indicator if the response is truncated. The response includes the last modified date, vector key, vector data, and metadata. You can also use the ListVectors API to easily export vector data from a specified vector index. The ListVectors operation is strongly consistent. So, after a write you can immediately list vectors with any changes reflected.

With S3 Vectors, you pay for storage and any applicable write and read requests (e.g., inserting vectors and performing query operations on vectors in a vector index). To see pricing details, see the S3 pricing page.

Yes. While creating a Bedrock Knowledge Base through the Bedrock Console or API, you can configure an existing S3 vector index as your vector store to save on vector storage costs for RAG use cases. If you prefer to let Bedrock create and manage the vector index for you, use the Quick Create workflow in the Bedrock console. Additionally, you can configure a new S3 vector index as your vector store for RAG workflows in Amazon SageMaker Unified Studio.

Yes. There are two ways you can use S3 Vectors with Amazon OpenSearch Service. First, S3 customers can export all vectors from an S3 vector index to OpenSearch Serverless as a new serverless collection using either the S3 or OpenSearch console. If you build natively on S3 Vectors, you will benefit from being able to use OpenSearch Serverless selectively for workloads with real-time query needs. Second, if you are a managed OpenSearch customer, you can now choose S3 Vectors as your engine for vector data that can be queried with sub-second latency. OpenSearch will then automatically use S3 Vectors as the underlying engine for vectors and you can update and search your vector data using the OpenSearch APIs. You gain the cost benefits of S3 Vectors, with no changes to your applications.

Amazon S3 Access Points are endpoints that simplify managing data access for any application or AWS service that works with S3. S3 Access Points work with S3 buckets and Amazon FSx for OpenZFS file systems. You can control and simplify how different applications or users can access data by creating access points with names and permissions tailored to each application or user.

Using S3 Access Points with S3 buckets, you no longer have to manage a single, complex bucket policy with hundreds of different permission rules that need to be written, read, tracked, and audited. Instead, you can create hundreds of access points per bucket that each provide a customized path into a bucket, with a unique hostname and access policy that enforces the specific permissions and network controls for any request made through the access point.

Using S3 Access Points with FSx for OpenZFS, you can access your FSx data using the S3 API as if the data were in S3. With this capability, your file data in FSx for OpenZFS is accessible for use with the broad range of artificial intelligence, machine learning, and analytics services and applications that work with S3 while your file data continues to reside on the FSx for OpenZFS file system.

With S3 Access Points, you can access file data in Amazon FSx for OpenZFS using S3 APIs and without moving data to S3. S3 Access Points attached to FSx for OpenZFS file systems work similarly to how S3 Access Points attached to S3 buckets work, providing data access via S3 with access controlled by access policies, while data continues to be stored in either FSx for OpenZFS file systems or S3 buckets. For example, once an S3 Access Point is attached to an FSx for OpenZFS file system, customers can use the access point with generative AI, machine learning, and analytics services and applications that work with S3 to access their FSx for OpenZFS data.

You can import data from within the same AWS Region into the S3 Express One Zone storage class via the S3 console by using the Import option after you create a directory bucket. Import simplifies copying data into S3 directory buckets by letting you choose a prefix or bucket to import data from without having to specify all of the objects to copy individually. S3 Batch Operations copies the objects in the selected prefix or general purpose bucket and you can monitor the progress of the import copy job through the S3 Batch Operations job details page.

S3 directory buckets that have no request activity for a period of at least 3 months will transition to an inactive state. While in an inactive state, a directory bucket is temporarily inaccessible for reads and writes. Inactive buckets retain all storage, object metadata, and bucket metadata. Existing storage charges will apply to inactive buckets. On an access request to an inactive bucket, the bucket will transition to an active state, typically within a few minutes. During this transition period, reads and writes will return a 503 SlowDown error code.

You should use Amazon S3 Metadata if you want to use SQL to query the information about your S3 objects to quickly identify specific datasets for your generative AI, analytics, and other use cases. S3 Metadata keeps metadata up to date in near real time, so you can use any Iceberg-compatible client to run SQL queries to find objects by the object metadata. For example, you can use a SQL query to return a list of objects that match certain filters such as objects added in the last 30 days across any bucket.

S3 Metadata is designed to automatically generate metadata that provides additional information about objects that are uploaded into a bucket and to make that metadata queryable in a read-only table. These metadata tables are stored in Amazon S3 Tables, which are built on Apache Iceberg and provide a managed way to store and query tabular data within S3. S3 Metadata creates and maintains system-level metadata such as object size, custom metadata such as tags and user-defined metadata during object upload, and event metadata such as the IP address that sent the request. As data in your bucket changes, S3 Metadata updates in near real time to reflect the latest changes. You can then query your metadata tables using various AWS analytics services and open source tools that are Iceberg-compatible, including Amazon Athena, Amazon QuickSight, and Apache Spark.

You can get started with S3 Metadata with just a few clicks in the S3 console. Just select the general purpose S3 bucket on which you would like to enable S3 Metadata, and S3 will analyze the data in your bucket and build a fully managed Apache Iceberg table that contains metadata for all of your objects. Within minutes, you can begin to query your metadata using any query engine or tooling that supports Apache Iceberg.

Your S3 Metadata tables are stored in an AWS managed table bucket in your AWS Account called aws-s3. Your tables will be read-only, and only S3 will have permission to write, update, or delete metadata. 

S3 Metadata stores metadata in two managed tables in your account: journal tables and live inventory tables. 

The S3 Metadata journal table provides a view of changes made within your bucket. As objects are added to, updated, and removed from your general purpose S3 buckets, the corresponding changes are reflected in the journal tables in near real time. Journal tables are useful for understanding the behavior of your applications, and for identifying any change made to your datasets. For example, you can write SQL queries for journal tables to find S3 objects that match a filter such as objects added in the last 30 days, objects that were added by active requesters, or objects that have metadata changes across the last week.

The S3 Metadata live inventory table contains a complete list of all the objects in your bucket. Live inventory tables are updated hourly and contain all the information that S3 knows about your objects. Live inventory tables are useful for discovering or identifying datasets in your bucket, based on the characteristics generated in object metadata. For example, you can use live inventory tables to identify training datasets for machine learning, to use in storage cost optimization exercises, or to help enforce governance controls.

When you add new objects to your bucket, you will see entries in the journal table within minutes, and you will see entries in the live inventory table on the next hourly refresh. When you enable S3 Metadata on an existing bucket, S3 will automatically start a backfill operation to generate metadata for all your existing objects. This backfill typically finishes in minutes but can take several hours if your existing datasets contain millions or billions of S3 objects.

The S3 Inventory report provides a scheduled alternative to Amazon S3’s synchronous List API. You can configure S3 Inventory to provide a CSV, ORC, or Parquet file output of your objects and their corresponding metadata on a daily or weekly basis for an S3 bucket or prefix. You can simplify and speed up business workflows and big data jobs with S3 Inventory. You can also use S3 inventory to verify encryption and replication status of your objects to meet business, compliance, and regulatory needs. Learn more at the Amazon S3 Inventory user guide.

S3 Tables provide purpose-built S3 storage for storing structured data in the Apache Parquet, Avro, and ORC formats. Within a table bucket, you can create tables as first-class resources directly in S3. These tables can be secured with table-level permissions defined in either identity- or resource-based policies, and are accessible by applications or tooling that supports the Apache Iceberg standard. When you create a table in your table bucket, the underlying data in S3 is stored as Parquet, Avro, or ORC files. Then, S3 uses the Apache Iceberg standard to store the metadata necessary to make that data queryable by your applications. S3 Tables include a client library that is used by query engines to navigate and update the Iceberg metadata of tables in your table bucket. This library, in conjunction with updated S3 APIs for table operations, allows multiple clients to safely read and write data to your tables. Over time, S3 automatically optimizes the underlying Parquet, Avro, or ORC data by rewriting, or "compacting” your objects. Compaction optimizes your data on S3 to improve query performance.

You can expect up to 3x faster query performance and up to 10x higher transactions per second (TPS) compared to storing Iceberg tables in general purpose Amazon S3 buckets. This is because table buckets automatically compact the underlying Parquet, Avro, or ORC data for your tables to optimize query performance, and the purpose-built storage supports up to 10x the TPS by default.

Table buckets give you the ability to apply resource policies to the entire bucket, or to individual tables. Table bucket policies can be applied using the PutTablePolicy and PutTableBucketPolicy APIs. Table-level policies allow you to manage permissions to tables in your table buckets based on the logical table that it is associated with, without having to understand the physical location of individual Parquet, Avro, or ORC files. Additionally, S3 Block Public Access is always applied to your table buckets.

Table buckets support the Apache Iceberg table format with Parquet, Avro, or ORC data.