AWS for M&E Blog

Evaluate your content with AWS Elemental Inference using the sample clipping portal

Live sports move fast, and audience attention moves faster. A goal scored in the 89th minute is most valuable in the moments right after it happens, when audiences are reacting on social feeds and second screens and a well-timed clip earns the views, shares, and reach that a highlight posted an hour later never will. To capture that attention, broadcasters and rights holders turn a live moment into a polished, shareable clip in seconds, in both 16:9 aspect ratio for traditional players and 9:16 for mobile and social.

Doing this manually is slow and expensive. An operator watches a feed, marks in and out points by hand, reframes for vertical, and pushes files through a transcode pipeline. At the scale of a full match day or season, that work adds up quickly, especially when there are multiple games happening at the same time.

AI video understanding can now operate directly on live content to detect highlights, generate subtitles, and intelligently crop for vertical formats. That transforms what was a manual, per-clip effort into an automated workflow. But adopting a new capability that manipulates your content in real time raises a practical question: how will it perform against your footage, in your environment, before you commit to building around it?

In this post, we introduce a sample application that helps you answer that question for AWS Elemental Inference. The sample is a deployable video clipping portal for live sports. You run it in your own AWS account, point it at your own content, and see how automated highlight detection, smart cropping, and smart subtitles behave end to end. We walk through the customer problem, the architecture, the capabilities it demonstrates, and how to deploy it.

Note: This project is a sample designed to accelerate evaluation and learning. It isn’t intended for production use without adapting it for your circumstances.

The challenge of creating live moments from multiformat clips

Building a highlight clipping workflow from scratch touches a surprising number of moving parts:

  1. Detection – Something recognizes that a goal, save, or dunk just happened, with high confidence.
  2. Reframing – A wide 16:9 broadcast frame is intelligently cropped to 9:16, keeping the action centered, without manual reframing.
  3. Extraction – The relevant seconds are instantly pulled out of a continuous live stream with frame accuracy.
  4. Editing and review – Operators trim, split, and approve clips quickly.
  5. Export and delivery – Finished clips become standardized for downstream publishing.

Each of these is solvable individually but wiring them together to evaluate a single new capability is a lot of undifferentiated work. That setup cost is exactly what slows teams down when they want to test whether Elemental Inference is a fit for their content.

Introducing the sample clipping portal

The sample clipping portal is a complete, working reference that connects Elemental Inference to the rest of an Amazon Web Services (AWS) media workflow so you can focus on evaluation rather than infrastructure. It gives operators a web application to manage live channels, review AI-detected highlights, edit clips in both orientations, and export finished MP4 files.

Because you deploy it into your own account with the AWS Cloud Development Kit (AWS CDK), everything runs under your control. Your content stays in your account, you can inspect every component, and you can adapt the code to mirror your own environment.

What Elemental Inference contributes

Within this workflow, Elemental Inference provides three capabilities that would otherwise require significant custom development:

  1. Smart cropping – Elemental Inference analyzes the live video and returns coordinates for the region of interest. The encoder uses those coordinates to produce a vertical 9:16 rendition automatically, keeping the action in frame.
  2. Smart subtitles – Elemental Inference transcribes speech from the live audio and generates timed subtitle data. The output can be rendered as captions on the video player or passed to packaging for embedded closed-caption tracks, removing the need for manual captioning.
  3. Key moment detection – Elemental Inference identifies highlight moments (such as shots, goals, and saves) and emits timing data so downstream automation can act on it.

The type of sport is configured per event, which determines the moments the system looks for.

How it works

The portal combines Elemental Inference with other AWS media and serverless services:

  1. AWS Elemental MediaLive ingests the incoming video stream and sends it to an Elemental Inference feed.
  2. Elemental Inference analyzes the video for smart cropping and event clipping along with audio for smart subtitles.
  3. The smart cropping feature determines key areas of interest. Elemental Inference returns the desired cropping coordinates back to the MediaLive channel.
  4. MediaLive uses these coordinates to encode a vertical rendition alongside the standard landscape output.
  5. The smart subtitles feature uses advanced speech recognition to transcribe the spoken audio, returning the transcription to MediaLive.
  6. MediaLive uses this subtitle data to feed its caption stream, which it sends to AWS Elemental MediaPackage as Timed Text Markup Language (TTML).
  7. MediaPackage acts as the just-in-time packager and serves as the origin for live content.
  8. When Elemental Inference detects a key moment, it sends timing details through Amazon EventBridge.
  9. The portal displays each detected moment so operators can choose which to publish. Selected highlights are processed by an AWS Step Functions workflow that triggers MediaPackage harvest jobs.
  10. AWS Elemental MediaConvert turns harvested content into frame-accurate MP4 and HLS assets stored in Amazon Simple Storage Service (Amazon S3).
  11. An Amazon CloudFront distribution serves live and on-demand content, with AWS WAF protecting the edge.

The web application is a React single-page app authenticated with Amazon Cognito, backed by Amazon API Gateway and AWS Lambda, with metadata stored in Amazon DynamoDB.

The following architecture diagram shows MediaLive sending content to Elemental Inference. Elemental Inference returns cropping and subtitle metadata to MediaLive and signals key moments through EventBridge.

Diagram showing how AWS Elemental Inference generates metadata from live video. AWS Elemental MediaLive sends video frames to AWS Elemental Inference, which analyzes them and returns subtitle and cropping metadata. The clipping metadata flows to Amazon EventBridge, which publishes events. AWS Step Functions reads those events and uses the timing information to trigger AWS Elemental MediaConvert, which transcodes the video into a shareable clip for social media.

Figure 1: Architecture diagram

Capabilities you can evaluate

The sample is organized around operator workflows, each demonstrating a capability you can test against your own content, including automated highlight detection, dual-orientation output, frame-accurate editing, export and integration, and configurable behavior.

Operators create an event (a time-bounded window such as a single game) and associate it with a channel. While active, Elemental Inference analyzes the stream, and detected events appear in the portal as clips ready for review, as shown in the following screenshot.

Screenshot of the clipping portal showing the list of key moments (clips) with status, tags and action buttons for lock, edit, download, and delete.

Figure 2: Clips table showing key moments with status badges and action icons (lock, edit, and download and delete)

Every channel produces both landscape (16:9) and portrait (9:16) renditions, with the vertical version framed using Elemental Inference smart cropping. Operators can preview both side by side to judge cropping quality on real plays.

A browser-based editor, shown in the following screenshot, lets operators trim, split, and remove segments before export. Edits are applied by MediaConvert, producing a new version while preserving a link to the original clip.

Screenshot showing the clipping portal video editor with a timeline display for top and tail trimming.

Figure 3: Video editor showing trim handles on the timeline

Finished clips are transcoded to MP4 and made available through presigned URLs. Because detection events flow through EventBridge and the application exposes an HTTP API, you can see exactly where to integrate with your own systems, whether that is a media asset manager (MAM), a social publishing tool, or an editorial review process.

System settings let you tune the evaluation: auto-harvest can record both orientations as moments are detected, a harvest buffer adds padding around each moment, and auto-activate can start and stop Elemental Inference based on an event’s scheduled times.

Prerequisites

To deploy the sample, you need to have Node.js, Python, Docker, and the AWS CLI configured for an account with access to MediaLive, MediaPackage, MediaConvert, and Elemental Inference.

Getting started

You deploy the sample with the AWS CDK:

  1. Clone the sample repository on GitHub and install dependencies.
  2. Bootstrap the AWS CDK for your account and Region.
  3. Build and deploy npm run build, then npm run deploy.
  4. Create an Amazon Cognito user, then open the CloudFront URL from the deployment outputs to sign in.

From there, create a channel that points at a video source, create an event, and start reviewing the highlights that Elemental Inference detects. The repository includes a deployment guide, a user guide, and data flow documentation to help you go deeper.

Conclusion

Turning live sports moments into multiformat, shareable clips is high-value work, and it’s exactly the kind of repetitive, time-sensitive task that machine learning is well-suited to accelerate. The hard part is often not the idea but the evaluation: proving that a new capability performs against your content before you build around it.

The sample clipping portal removes that barrier. Deploy it into your own account, point it at your own streams, and evaluate how Elemental Inference handles highlight detection, smart cropping for vertical reframing, and smart subtitles on real content, all working together in a single end-to-end workflow.

Get started: Clone the sample repository on GitHub and deploy your first evaluation today. If you have questions, want to share what you build on top of it, or have feedback on how Elemental Inference performs against your content, join us in the AWS Media Services community on Builder Center.

Further reading

  1. Guidance for configuring Elemental Inference as part of a built-in MediaLive channel configuration
  2. AWS Elemental MediaConvert guide for applying Smart Cropping to file transcode jobs
  3. AWS Elemental Inference guide for standalone service setup
Christer Whitehorn

Christer Whitehorn

Christer is a Principal Solutions Architect on the World-Wide Media and Edge team at AWS. With over two decades of experience in the media technology sector, he has established himself as an expert in multiscreen video delivery, broadcast playout systems, and compression headend solutions. He is passionate about helping customers optimize their media workflows, improve operational efficiency, and enhance viewer experiences through cloud-based technologies.

Kiran Patel

Kiran Patel

Kiran Patel is the Senior Product Manager for AWS Elemental Inference, where he leads product strategy for AI-powered video intelligence. Prior to joining Amazon Web Services (AWS), Kiran worked with the BBC on live streaming and VOD workflows for their News, Sport, and iPlayer products. He has over 20 years of experience working with OTT video.