AWS for Industries
Unlock real-time advertising insights with Amazon Marketing Stream and AWS
Real-time campaign performance visibility empowers Amazon Ads advertisers to seize optimization opportunities the moment they emerge. Amazon Marketing Stream integrated with Amazon Web Services (AWS) unlocks powerful business capabilities that transform advertising investments into precision-driven growth engines.
We will present options for how Amazon Ads advertisers can deploy infrastructure to support Amazon Marketing Stream on AWS. This direct integration with AWS creates a powerful pipeline for advertising data that enables more agile campaign management and deeper analytical insights. We’ll also demonstrate how to utilize Amazon QuickSight and Amazon Q for QuickSight. They transform data into actionable business intelligence—leveraging generative business intelligence capabilities, with natural language interactions, with your advertising data.
Amazon Marketing Stream
Amazon Marketing Stream is a push-based messaging system that delivers hourly Amazon Ads campaign metrics and information on campaign changes in near real time through the Amazon Ads API.
Advertisers receive advertising campaign metrics and information on campaign changes in near real-time. Amazon Marketing Stream reduces reporting delays from days to minutes so advertisers can monitor campaign performance as it happens. This immediate visibility allows for daily monitoring of promotional events for quick identification of underperforming campaigns before budgets are depleted.
Amazon Marketing Stream receives near real-time data feeds from multiple Amazon Ads products including:
- Sponsored Products
- Sponsored Brands
- Sponsored Display
- Amazon DSP
It delivers comprehensive datasets on:
- Traffic metrics
- Conversion data
- Budget usage
- Recommendations
- Campaign logs
The granularity and timeliness of Amazon Marketing Stream data enable sophisticated optimization algorithms for dynamic bid adjustments and automated budget reallocation. For example, search-based advertising, Amazon Marketing Stream provides immediate feedback on keyword performance. This helps advertisers quickly identify converting search terms and optimize their targeting strategy.
Amazon Marketing Stream also creates valuable opportunities to connect advertising data with other business systems. Advertisers can synchronize inventory management to prevent promoting out-of-stock items, conduct margin analysis by combining ad spend with product cost data, and map customer journeys across advertising and owned channels.
Data delivery
Unlike traditional reporting methods that deliver data in batches, Amazon Marketing Stream is a push-based messaging system—providing data to your AWS environment within minutes of events occurring. This push-based approach allows for significantly higher data delivery and volume without API throttling concerns.
The data arrives in JSON format through either Amazon Simple Queue Service (Amazon SQS) queues or Amazon Data Firehose (Firehose) delivery streams, streamlining the ability to parse and integrate with existing systems. Each message contains detailed metadata including timestamps, event types, and identifiers that allow for precise tracking and analysis. This direct integration with AWS services eliminates the need for custom data transfer mechanisms and provides reliable, secure delivery of advertising insights.
Connecting to AWS
The technical integration between Amazon Marketing Stream and AWS is designed for security, reliability, and ease of implementation. Amazon Marketing Stream leverages AWS Identity and Access Management (IAM) roles for secure cross-account access. Amazon Ads can then deliver data to the advertiser’s AWS resources without exchanging credentials.
The Amazon Marketing Stream data pipeline begins when events occur within Amazon Ads. Amazon Marketing Stream prepares this data before delivering it to the advertiser’s AWS account through either Amazon SQS or Firehose. From there, advertiser systems process the data for storage, analysis, or action.
Advertisers control their data streams through the Amazon Ads API, where they can create subscriptions to specific datasets and configure delivery destinations. Amazon Marketing Stream delivers data as JSON messages with consistent schemas, featuring well-documented field definitions and comprehensive metadata for context and processing. This standardized approach streamlines integration with downstream analytics systems and confirms data consistency across applications.
Architecture overview
Advertisers have flexibility in how they integrate with Amazon Marketing Stream by using Amazon SQS or Firehose. Depicted in Figure 1 is Amazon Marketing Stream and the common elements of an advertiser architecture.
Figure 1: Architecture diagram depicting multi-paths for Amazon Marketing Stream on AWS
Deployment Options
Amazon Marketing Stream Integration
As depicted in the architecture in Figure 1, you can integrate Amazon Marketing Stream with your AWS environment using either Amazon SQS or Firehose (Detailed steps for integration can be found in the Developers Guide). Amazon Ads intentionally provides customers with flexibility and a choice in selecting Amazon SQS or Firehose. We will discuss the trade-offs between using each approach.
First, let’s discuss the similarities between using Amazon SQS or Firehose as a destination for Amazon Marketing Stream. Both Amazon SQS and Firehose support key Amazon Marketing Stream features including:
- Hourly traffic and conversion data
- Near real-time messaging
- The ability to avoid API throttling
Each of these services also support Amazon Marketing Stream reporting data, which includes an advertiser’s processed (non-raw, summarized) performance data. This includes traffic and conversions sent hourly, along with data dimensions necessary for analytics. Messaging data is also supported, sending notifications on entity state changes and other events.
This messaging includes:
- Campaign
- Ad group
- Ad
- Targeting entity
- Consumption of budget changes whenever budget allotment changes by 5% or more
Amazon Simple Queue Service
The primary advantage of adopting Amazon SQS as your Amazon Marketing Stream destination is for ease of integration with advertiser-owned applications. You can have your own application poll data directly from the Amazon SQS queue, while still benefiting from the scalability and durability of Amazon SQS.
You can have an application running on Amazon Elastic Compute Cloud (Amazon EC2) or through managed container services (such as Amazon Elastic Kubernetes Service (Amazon EKS) or Amazon Elastic Container Service (Amazon ECS)) for processing queue messages and store them in a database. This provides you the flexibility to consume your Amazon Marketing Stream data in a variety of ways.
Amazon Data Firehose
Amazon Data Firehose, a fully managed service for delivering real-time streaming data, is also a supported Amazon Marketing Stream destination. For use cases where data should be delivered to Amazon Simple Storage Service (Amazon S3), Firehose support streamlines architectures. Advertisers can have Firehose consume their Amazon Marketing Stream data and deliver directly to Amazon S3. No Amazon SQS queue or AWS Lambda functions are required with this approach.
Firehose also provides additional delivery endpoints. No additional development is required.
Firehose supports destinations such as:
- Amazon S3
- Amazon Redshift
- Amazon OpenSearch Service
- Amazon OpenSearch Serverless
- Splunk
- Apache Iceberg Tables
- Any custom HTTP endpoint or HTTP endpoints owned by supported third-party service providers
With Firehose, you don’t need to write applications or manage resources. Configure your Marketing Stream to send data to Firehose and it automatically delivers the data to the destination you specified.
You can also configure Firehose to transform your data before delivering it. While both Firehose and Amazon SQS receive JSON from Amazon Marketing Stream, Firehose supports:
- Built-in parquet and orc conversion
- Extracting data fields from records
- JQ parsing language for transformations
- Custom transformation through AWS Lambda integration
Considerations
Amazon SQS and Firehose both provide different capabilities as Amazon Marketing Stream destinations. Consider how you will utilize your Amazon Marketing Stream data.
If consuming your data from a supported Firehose destination (including Amazon S3), then Firehose will streamline your architecture. By integrating directly with Firehose, managing Amazon SQS and the associated costs can be avoided.
However, Amazon SQS remains a good option for advertisers with custom applications or a requirement to send data to a location not supported by Firehose (such as a relational database).
An additional consideration is multi-region requirements. An individual Amazon Marketing Stream sends data to its local region. If you have multiple marketing streams and would like to consolidate the data, Firehose can optionally send the data to additional regions. Sending to multiple regions may incur cross-region data transfer costs.
S3 bucket replication is also an option for consolidating data from multiple regions to a single location. Similarly, there may be a need to distribute your Amazon Marketing Stream data from its single region to multiple regions. Again, Firehose and S3 bucket replication can facilitate this use case.
Amazon Ads supports both Amazon SQS and Firehose destinations to provide advertisers with flexibility. To estimate the cost of either integration option, please use the AWS Pricing Calculator.
Visualizing data using Q for QuickSight
Advertisers can optionally build onto their Amazon Marketing Stream integration with Amazon QuickSight dashboards for analysis.
Figure 2: Amazon Marketing Stream Pipeline with Amazon QuickSight
Figure 2 shows Amazon Marketing Stream delivering data directly into Amazon Data Firehose, which is stored in an S3 bucket. The data can then be queried using Amazon Athena and business intelligent dashboards built within Amazon QuickSight.
Figure 3: Business insight dashboard generation using Amazon QuickSight
Business Intelligence is a primary way advertisers derive value from Amazon Marketing Stream data. QuickSight offers seamless integration with your marketing data hosted on AWS and provides generative AI-powered dashboarding and insight creation capabilities through Amazon Q in QuickSight. The animation in Figure 3 demonstrates how users can create rich and interactive ad placement analysis dashboards using natural language.
Please note that to enable natural language querying on Amazon Marketing Stream data, you will need to create a Q Topic and populate it with metadata. You can consult our documentation on Sponsored Products performance datasets to understand how best to achieve this. The following screenshot (Figure 4) shows an example of a topic field definition.
Figure 4: Q Topic metadata configuration for Marketing Stream data
Figure 5: Business scenario generation using Amazon QuickSight
This animation in Figure 5 showcases how analysts and decision makers can use the QuickSight scenario feature. You can ask business-driven questions about ad placement return on investment (ROI) for a specific placement type and time frame. By asking a question, QuickSight autonomously retrieves the necessary data, performs calculations, and creates visuals to present the answer. This can greatly shorten time to insight for scenarios such as hourly ROI analysis, and conversion rates for every ad type and time of day.
Conclusion
Being able to know in real time how your campaign is performing is imperative for optimizing advertising spend. Using Amazon Marketing Streams with AWS can help speed up information and insights about your Amazon Ads data as campaign events change.
We demonstrated how to use Amazon Marketing Streams to ingest Amazon Ads data in near real-time into a data lake. We discussed trade-offs between each integration option, Amazon Simple Queue Service and Amazon Data Firehose, noting that each approach has benefits depending on downstream consumers.
We also demonstrated the benefit of leveraging an Amazon QuickSight dashboard, using generative AI powered by Amazon Q in QuickSight, to analyze Amazon Marketing Stream data with straightforward, natural language.
Being able to quickly unlock significant business insights can transform how you manage and optimize advertising investments—keeping you agile and flexible in real time.