Capturing AWS Elemental MediaConnect events with Amazon CloudWatch logs

As live video streaming becomes increasingly crucial for media and entertainment businesses, effective observability of your Amazon Web Services (AWS) Elemental MediaConnect (MediaConnect) resources is essential for maintaining reliable video transport. MediaConnect automatically provides useful operational metrics through Amazon CloudWatch, giving you immediate visibility into the performance of the service. However, there’s substantially valuable data available in the form of events, which are not automatically recorded alongside these metrics.

MediaConnect publishes events to Amazon EventBridge in near real-time, capturing details like alerts and health status. These events provide enhanced operational insights and, with a few extra steps, can be stored in Amazon CloudWatch logs. This will create a comprehensive observability solution that combines both metrics and events.

We will walk you through capturing all MediaConnect events and storing them in CloudWatch logs using a quick AWS CloudFormation template.

Solution

This implementation will:

• Automatically capture all MediaConnect events
• Store events in CloudWatch logs for near real-time visibility
• Create a persistent searchable record for root cause analysis
• Deploy through CloudFormation with minimal configuration
• Require no additional infrastructure to manage
• Provide a foundation for custom alerting and metrics

Prerequisites

Before deploying this solution, ensure you have:

• An AWS account with AWS Identity and Access Management (IAM) permissions to create CloudFormation stacks, CloudWatch Log groups, and EventBridge rules.
• Access to either the AWS Management Console or AWS Command Line Interface (AWS CLI). For AWS CLI deployment, confirm the AWS CLI is installed and configured. Alternatively, you can use AWS CloudShell, which provides a browser-based shell with pre-installed and pre-authenticated AWS CLI.

How it works

The CloudFormation template sets up two main components:

1. An EventBridge rule to capture and forward MediaConnect events
2. A CloudWatch log group to store MediaConnect events

Your MediaConnect flows generate operational events. These events automatically flow to EventBridge where our configured rule identifies and captures all events. EventBridge then forwards these matching events to the designated CloudWatch log group where they are stored and available for analysis.

Figure 1: Event flow from MediaConnect to CloudWatch Logs through EventBridge.

Following is the template used to capture events:

---
AWSTemplateFormatVersion: 2010-09-09
Description: Capture all AWS Elemental MediaConnect events and send them to Amazon CloudWatch Logs

Resources:

  MediaConnectLogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: /aws/events/mediaconnect-events
      RetentionInDays: 7

  MediaConnectEventRule:
    Type: AWS::Events::Rule
    Properties:
      Name: MediaConnectAllEventsRule
      Description: Capture all MediaConnect events
      EventPattern:
        source:
          - "aws.mediaconnect"
      State: ENABLED
      Targets:
        - Arn: !GetAtt MediaConnectLogGroup.Arn
          Id: "TargetLogGroup"

Note: This template captures all MediaConnect events and stores them in CloudWatch logs. You can customize the event filtering by modifying the EventPattern in the template.

For example, to capture only API alert events, add a detail-type filter such as:

EventPattern:
  source:
    - "aws.mediaconnect"
  detail-type:
    - "MediaConnect Alert"

Other detail-types include: MediaConnect Flow Status Change, MediaConnect Flow Maintenance, MediaConnect Flow Health, MediaConnect Source Health, MediaConnect Output Health and MediaConnect Flow Content Quality. Adjust the pattern based on your observability requirements.

Deployment

Using the AWS CLI:

1. Save the template as mediaconnect-logging.yaml
2. Deploy by running:

aws cloudformation deploy \
--template-file mediaconnect-logging.yaml \
--stack-name mediaconnect-event-logging

3. Allow a few minutes for the resources to be created and then verify the stack status is CREATE_COMPLETE in the CloudFormation console.

Using the AWS Console:

1. In the AWS CloudFormation console, choose Create stack > With new resources (standard).
2. In the Specify template section, select Upload a template file, then Choose file and upload the previously provided template.
3. Enter a stack name (for example: mediaconnect-event-logging).
4. Review the configuration and choose Create stack. Allow a few minutes and then verify the stack status is CREATE_COMPLETE.

Post deployment

You should start seeing MediaConnect events being captured in CloudWatch logs.

Following is an example of a MediaConnect Flow Health event in CloudWatch logs, that indicates there were continuity count errors in the incoming transport stream.

Figure 2: MediaConnect Flow Health Event in CloudWatch Logs.

Clean up

Delete the CloudFormation stack to remove all created resources. For detailed instructions, read either Deleting a stack on the AWS CloudFormation console or Deleting a stack using AWS CLI.

Customization options

You can enhance this template by:

• Adjusting the log retention period by modifying the RetentionInDays value (currently set to seven days).
• Filtering specific MediaConnect events by adding a detail-type section to the EventPattern.
• Filtering events for a specific MediaConnect Flow by adding a resources section to the EventPattern and specifying the Flow ARN. Pair this by creating a separate log group for each flow.

For advanced observability, you can additionally configure:

• Amazon CloudWatch metric filters to analyze specific patterns and create custom metrics.
• Amazon CloudWatch alarms based on these metrics for automated notifications.

Cost considerations

This solution incurs costs for CloudWatch logs (data ingestion, storage and queries) and EventBridge (event ingestion and delivery). Actual costs will vary based on the volume of events, log retention period and AWS Region.

Review the Amazon CloudWatch pricing and Amazon EventBridge pricing for complete details. As with any AWS solution, you can use AWS Cost Explorer and AWS Budgets to monitor and manage your spending.

Conclusion

AWS Elemental MediaConnect events provide valuable operational insights into your MediaConnect flows in addition to metrics. We showed you a quick, automated solution to capture and store these events for near real-time visibility and historical analysis, enhancing your overall observability capabilities.

Remember to adjust the log retention period and event pattern based on your specific needs. Also consider adding additional features like custom metrics and alarms based on your operational requirements.

Contact an AWS Representative to know how we can help accelerate your business.

Further reading

• Monitoring MediaConnect with Amazon EventBridge events
• Near real-time notifications for AWS Media Services
• Getting started with the workflow monitor for AWS Media Services