AWS Messaging Blog

Track OTP success with AWS End User Messaging SMS feedback

In this post, we show how to implement message feedback for SMS one-time passwords (OTPs) using AWS End User Messaging. OTP verification through SMS is a fundamental component of modern authentication systems. Although sending OTPs follows an established pattern, tracking their delivery and usage presents several challenges. This post shows how to implement the AWS End User Messaging Message Feedback API to monitor OTP delivery and conversion rates effectively. This post highlights the Message Feedback API in an OTP use case; for practical examples and detailed guidance on building a secure OTP architecture, see Build a Secure One-Time Password Architecture with AWS.

Challenges with OTP tracking

Organizations commonly face these key challenges with OTP tracking:

  • Relying solely on Delivery Receipt (DLR) data for confirming message delivery, which is third-party carrier data that can be subject to interpretation by carriers or message providers, whereas conversion tracking through message feedback provides first-party data that can more accurately reflect actual message delivery and usage
  • Measuring accurate user authentication success rates
  • Identifying OTP verification issues across different geographic regions, carriers and delivery paths

To address these challenges, you can use the AWS End User Messaging Message Feedback API to track delivery and conversion rates, providing first-party data for more accurate insights into message delivery and usage patterns. Although OTP use cases are the most common and serve as our example implementation of message feedback, the same tracking logic can also be applied to other types of SMS conversions, such as promotional link clicks, shopping cart additions, account activations, appointment confirmations, and delivery notifications.

Solution overview

The OTP message flow consists of two main phases. Let’s first examine how the system handles the initial OTP request.

Phase 1: OTP request flow

When a customer initiates an OTP request, your system begins a carefully orchestrated process. First, your application receives this request and generates a unique OTP. With the OTP generated, your system prepares to send it through the AWS End User Messaging API, specifically enabling message feedback tracking by setting the MessageFeedbackEnabled parameter to true when calling SendTextMessage.

Upon successful sending, it returns a unique message ID, which your system must store alongside the generated OTP. This message ID serves as a crucial tracking identifier for the entire verification process. The message is then dispatched to the customer’s device, and your system enters a waiting state, ready to process the verification attempt.

The following diagram illustrates the OTP request flow.

OTP Request Flow Diagram

Phase 2: OTP verification flow

The verification process begins when the customer receives the OTP through SMS and submits it back to your system. Upon receiving the submission, your system first validates the OTP against the stored value. This verification step is critical, because its outcome determines how you will update the message feedback status.

If the customer successfully verifies the OTP, your system calls the PutMessageFeedback API with the stored message ID and sets the status to "RECEIVED", indicating successful delivery and usage of the OTP. However, if the verification fails or the customer doesn’t respond within the timeout period, your system sets the status to "FAILED".

If your system doesn’t explicitly update the feedback status within 1 hour, AWS automatically sets it to "FAILED".

The following diagram illustrates the OTP verification flow.

Prerequisites

Before you begin implementing OTP message feedback, make sure you have the following components and permissions in place:

Send SMS with message feedback enabled

You can enable message feedback in two ways. The first method is to use the MessageFeedbackEnabled parameter when sending an SMS, the second is to send a message with a configuration set with message feedback already enabled. Using a configuration set is often more convenient for bulk implementations because you don’t need to specify message feedback settings in each API call.

To send an SMS with message feedback enabled directly, you can use the following function:

import boto3

# Initialize the End User Messaging client
client = boto3.client('pinpoint-sms-voice-v2')

def send_otp_with_feedback():
    # Generate a unique OTP
    otp = generate_otp()  
    
    # Send SMS with feedback enabled
    response = client.send_text_message(
        DestinationPhoneNumber='+15555550123',  # Replace with your destination phone number
        OriginationIdentity='+14255550120',  # Replace with your origination identity
        MessageBody=f'Your verification code is: {otp}',
        MessageFeedbackEnabled=True
    )
    
    # Store OTP details for verification
    store_otp_details(response['MessageId'], otp)
    return response['MessageId']

The function uses the following details:

  • store_otp_details() is a placeholder function where you store the OTP details in a database for later retrieval
  • generate_otp() is a placeholder function where you generate your OTPs to send using SMS

If you prefer to use a configuration set with message feedback enabled, you can use the following alternative function:

def send_otp_with_feedback_using_configuration_set():
    # Initialize the End User Messaging client
    client = boto3.client('pinpoint-sms-voice-v2')
    
    # Generate OTP
    otp = generate_otp()
    
    # Send SMS using configuration set
    response = client.send_text_message(
        DestinationPhoneNumber='+15555550123',  # Replace with your destination phone number
        OriginationIdentity='pool-201d59fffd554bdfbaf9ee8aEXAMPLE',  # Replace with your origination identity
        MessageBody=f'Your verification code is: {otp}',
        ConfigurationSetName='example-us-east-configuration-set'  # Replace with your configuration set name
    )
    
    # Store OTP details for later verification
    store_otp_details(response['MessageId'], otp)
    
    return response['MessageId']

Your configuration set must have message feedback enabled to use this option. You can enable it using the AWS Command Line Interface (AWS CLI) with the following command:

aws pinpoint-sms-voice-v2 set-default-message-feedback-enabled \
--configuration-set-name "YourConfigSetName" \
--message-feedback-enabled

Another option is to use the AWS End User Messaging console, where you can enable message feedback under Set Settings for the desired configuration set.

Update feedback

After you send a message, you can update the message status to indicate whether a user has successfully completed an action, such as entering the OTP on your application or webpage:

def update_message_feedback(message_id: str, status: str) -> dict:
    try:
        # Initialize the End User Messaging client
        client = boto3.client('pinpoint-sms-voice-v2')
        
        # Update the message feedback status
        response = client.put_message_feedback(
            MessageId=message_id,
            MessageFeedbackStatus=status
        )
        
        return response
        
    except Exception as e:
        print(f"Error updating message feedback: {str(e)}")
        raise

# Example usage
message_id = "a1b2c3d4-5678-90ab-cdef-EXAMPLE11111"  # Replace with your message ID
status = "RECEIVED"  # Use "FAILED" for unsuccessful verifications

result = update_message_feedback(message_id, status)
print(f"Feedback status updated: {result}")

Verify feedback metrics

The AWS End User Messaging dashboard provides comprehensive metrics to help you monitor your OTP performance. The following metrics are available for customizable time periods:

  • Number of messages with feedback completion
  • Percentage of messages with feedback completion
  • Number of SMS with feedback completion by country

To review your application’s overall message feedback metrics, choose Dashboard in the AWS End User Messaging console navigation pane, then choose Message Feedback Metrics.

The dashboard presents three key metrics:

  • Number of messages with feedback completion – The count of SMS and MMS messages where the message feedback record is set to RECEIVED
  • Percentage of messages with feedback completion – The percentage of SMS and MMS messages where the message feedback record is set to RECEIVED
  • Number of SMS with feedback completion by country – The count of message feedback received by country

The progression to 100% completion indicates optimal system performance, where all sent OTPs were successfully received and verified by users, and the message feedback record is set to RECEIVED within the expected timeframe. This high completion rate suggests effective message delivery and a smooth user verification experience. Variations in completion rates across countries can help identify potential regional delivery challenges or user behavior patterns.

The 30% conversion starting point shown in this example is used for illustration purposes only, demonstrating messages that were intentionally left unconverted during testing.

Best practices for OTP implementation

For a secure and reliable OTP implementation, follow these best practices to balance security with user experience:

  • Include rate limiting to prevent abuse
  • Implement proper timeout mechanisms for OTPs
  • Make sure error handling provides clear feedback to users
  • Maintain comprehensive logging for security audits

Conclusion

By implementing the Message Feedback API for OTP tracking, you can gain valuable insights into your authentication system’s effectiveness in real time. This approach helps you monitor successful OTP usage and identify potential delivery issues that might affect user authentication, with granular metrics broken down by geographic regions. The data collected through message feedback offers a more accurate picture of actual user interactions compared to carrier-provided delivery receipts, helping you make data-driven decisions about your authentication system.

To build upon this foundation, consider implementing Amazon CloudWatch alerts for your conversion metrics, and optimizing your message templates based on performance data. The combination of real-time feedback, detailed analytics, and proactive monitoring can help make sure your OTP system remains both secure and efficient.

For additional implementation guidance and best practices, refer to the following resources:

About the authors

Rommel Sunga

Rommel Sunga

Rommel is a Senior Solutions Architect at AWS, based in Singapore. He specializes in AWS End User Messaging and Amazon Simple Email Service, helping customers build scalable and reliable communication solutions. With expertise in cloud-based messaging architectures, he focuses on enabling organizations to enhance their customer engagement.