Sold by: PolyAI
Deployed on AWS
Build real-time voice agents with Dialog-RSN-1, PolyAI's audio-native dialog model. It combines speech understanding, turn-taking and function calling, with text responses for a separate text-to-speech system.
Overview
Dialog-RSN-1 is PolyAI's audio-native dialog model for developers building real-time customer service voice agents. It reasons over caller audio alongside the system prompt and conversation history, retaining cues such as tone and hesitation that a transcript can miss.
Use the model as the dialog component of a voice agent. It combines speech recognition, turn-taking decisions, function calling and response generation. A helper voice activity detector triggers model requests; the model decides whether the caller has finished speaking. Speech generation uses a separate text-to-speech system, allowing control over the output voice.
PolyAI reports an 11% relative increase in containment at a national restaurant group and a 37% reduction in response latency at a large insurer after switching to Dialog-RSN-1. Containment is the share of calls resolved without transfer to a human. These are results from individual PolyAI deployments; performance depends on the use case and deployment configuration. See the PolyAI launch article under Learning resources.
Highlights
- Understands caller audio and conversational context to support natural turn-taking.
- Reported deployment results: 11% relative containment improvement at a restaurant group; 37% lower response latency at an insurer.
- Combines speech recognition, turn-taking decisions and response generation in one model, with separate text-to-speech for voice control.
Details
Sold by
Delivery method
Latest version
Deployed on AWS
New
Introducing multi-product solutions
You can now purchase comprehensive solutions tailored to use cases and industries.
Features and programs
Financing for AWS Marketplace purchases
AWS Marketplace now accepts line of credit payments through the PNC Vendor Finance program. This program is available to select AWS customers in the US, excluding NV, NC, ND, TN, & VT.
Pricing
Pricing is based on actual usage, with charges varying according to how much you consume. Subscriptions have no end date and may be canceled any time.
Additional AWS infrastructure costs may apply. Use the AWS Pricing Calculator to estimate your infrastructure costs.
Dimension | Description | Cost/host/hour |
|---|---|---|
ml.g6e.4xlarge Inference (Batch) Recommended | Model inference on the ml.g6e.4xlarge instance type, batch mode | $120.00 |
ml.p4d.24xlarge Inference (Real-Time) Recommended | Model inference on the ml.p4d.24xlarge instance type, real-time mode | $45.00 |
ml.p5.48xlarge Inference (Real-Time) | Model inference on the ml.p5.48xlarge instance type, real-time mode | $110.00 |
Vendor refund policy
n/a - no refund
Legal
Vendor terms and conditions
Upon subscribing to this product, you must acknowledge and agree to the terms and conditions outlined in the vendor's End User License Agreement (EULA) .
Content disclaimer
Vendors are responsible for their product descriptions and other product content. AWS does not warrant that vendors' product descriptions or other product content are accurate, complete, reliable, current, or error-free.
Delivery details
Amazon SageMaker model
An Amazon SageMaker model package is a pre-trained machine learning model ready to use without additional training. Use the model package to create a model on Amazon SageMaker for real-time inference or batch processing. Amazon SageMaker is a fully managed platform for building, training, and deploying machine learning models at scale.
Deploy the model on Amazon SageMaker AI using the following options:
Version release notes
Initial AWS Marketplace release of Dialog-RSN-1, PolyAI's audio-native language model for customer-service voice agents. The model listens to caller audio directly, decides when the caller's turn is complete, and produces a text reply or a tool call in one pass. Speech synthesis is not included; pair it with the text-to-speech engine of your choice. The endpoint speaks the OpenAI Realtime wire protocol over a bidirectional stream, so existing OpenAI Realtime clients work with a base-URL change via the supplied bridge. This release supports English, a 32,000-token context window, and sessions of up to 30 minutes.
Additional details
Inputs
- Summary
Input is a stream of OpenAI Realtime client events over SageMaker's bidirectional-streaming invocation (InvokeEndpointWithBidirectionalStream). The session is configured once with session.update: system instructions, an optional list of function tools with JSON-schema arguments, optional knowledge-base topics via the x_poly_knowledge extension, and the audio input format. Caller audio is then sent as base64 PCM16 mono chunks in input_audio_buffer.append. Results of tool calls are returned to the model with conversation.item.create (function_call_output). The model reasons over the audio itself, so tone, hesitation and non-speech cues reach the model without a separate transcription step. Text-only input via conversation.item.create (input_text) is also accepted.
- Limitations for input type
- Audio: PCM16, mono, 16 kHz or 24 kHz only (audio/pcm); G.711 mu-law/A-law, Opus and stereo are not accepted. Context window: 32,000 tokens across instructions, tools, knowledge topics, history and audio; there is no automatic truncation, so long sessions should be managed by the client. Session: maximum 30 minutes per connection, with an idle timeout; state is per connection and is not carried across reconnects. Language: English. Output modality is text only; a request for audio output is rejected.
- Input MIME type
- application/json
{"type":"session.update","session":{"type":"realtime","model":"dialog-rsn-1","output_modalities":["text"],
"instructions":"You are Sam, a voice assistant for Maison Lumiere, a restaurant. If the caller asks to book, call check_availability before answering. Keep answers to one short sentence.",
"audio":{"input":{"format":{"type":"audio/pcm","rate":16000},"turn_detection":{"type":"server_vad"}}},
"tools":[{"type":"function","name":"check_availability","description":"Check whether a table is free at a given time.",
"parameters":{"type":"object","properties":{"party_size":{"type":"integer"},"date":{"type":"string"},"time":{"type":"string","description":"24-hour HH:MM"}},"required":["party_size","time"]}}]}}
{"type":"input_audio_buffer.append","audio":"<base64 PCM16 chunk>"}
{"type":"input_audio_buffer.append","audio":"<base64 PCM16 chunk>"}
{"type":"conversation.item.create","item":{"type":"function_call_output","call_id":"call_d3200c12","output":"{\"available\":true,\"party_size\":4,\"time\":\"19:30\",\"table\":\"window\"}"}}
Input data descriptions
The following table describes supported input data fields for real-time inference and batch transform.
Field name | Description | Constraints | Required |
|---|---|---|---|
session.instructions | string
System prompt: role, scope, business rules, turn-taking guidance | No | |
session.tools[ ] | function[ ]
JSON-schema function definitions the model may call | - | No |
session.x_poly_knowledge[ ] | object[] Knowledge-base topics {name, content, actions} the model answers from | - | No |
session.audio.input.format | object {"type":"audio/pcm","rate":16000|24000} | - | No |
session.audio.input.turn_detection | object or null server_vad (default) or null for push-to-talk via input_audio_buffer.commit | - | No |
session.output_modalities | ["text"] Fixed; audio output is refused | - | No |
session.max_output_tokens | integer Optional cap on reply length | - | No |
input_audio_buffer.append.audio | base64 string PCM16 mono chunk | - | No |
conversation.item.create.item | object function_call_output or input_text | - | No |
Support
Vendor support
AWS infrastructure support
AWS Support is a one-on-one, fast-response support channel that is staffed 24x7x365 with experienced and technical support engineers. The service helps customers of all sizes and technical abilities to successfully utilize the products and features provided by Amazon Web Services.
Similar products
![DialoGLUE: A Natural Language Understanding Benchmark for Task-Orie[...]](https://d1ewbp317vsrbd.cloudfront.net/99bef72e-e6e0-4e7e-a24b-b068cfbef716.png)
This bucket contains the checkpoints used to reproduce the baseline results reported in the DialoGLUE benchmark hosted
on EvalAI (https://evalai.cloudcv.org/web/challenges/challenge-page/708/overview). The associated scripts for using the checkpoints are located here:
https://github.com/alexa/dialoglue. The associated paper describing the benchmark and checkpoints is here: https://arxiv.org/abs/2009.13570.
The provided checkpoints include the CONVBERT model, a BERT-esque model trained on a large open-domain conversational
dataset. It also includes the CONVBERT-DG and BERT-DG checkpoints described in the linked
paper.
A dataset of over 1000 scraped search keywords, related searches and search suggestions related to the Respiratory Syncytial Virus (RSV).
ILLUIN Dialogue is a platform where you can easily manage the most advanced generative conversational agents on the market
Dialogue Digest extracts key insights from conversations, enabling customer service teams, sales professionals, and teams to streamline follow-ups, improve decision-making, and enhance collaboration.
Customer reviews
No customer reviews yet
Be the first to review this product . We've partnered with PeerSpot to gather customer feedback. You can share your experience by writing or recording a review, or scheduling a call with a PeerSpot analyst.