AWS Partner Network (APN) Blog
Modernizing Healthcare with Generative Voice AI Using Deepgram on AWS
By Ayan Ray, Sr. Partner Solutions Architect – AWS
By Gokhul Srinivasan, Sr. Partner Solutions Architect – AWS
By Eric Yuen, Sr. Partner Solutions Architect – AWS
By Evan Henry, Staff Product Manager – Deepgram
 |
| Deepgram |
 |
Healthcare professionals, particularly doctors and nurses, face an unprecedented burden in today’s medical environment. Instead of focusing on meaningful patient interactions and delivering compassionate care, they often find themselves tethered to screens and paperwork which limits crucial face-to-face connections with their patients. This administrative burden can not only compromise the quality of patient care, but can also intensify the growing fatigue of clinical professionals.
Despite increased enrollment in medical and nursing programs, educational institutions struggle to produce enough graduates to meet the growing demand. The rigorous qualification requirements, extensive training periods, and high-stress nature of healthcare positions make it difficult to quickly expand the qualified workforce.
This mounting pressure on the healthcare system calls for innovative solutions and fundamental changes to address the widening gap between workforce supply and patient demand. Technology like generative voice AI is now the engine driving how clinicians and patients connect, document, and act on medical data.
Generative Voice AI in Healthcare
The integration of voice AI systems into healthcare workflows represents a fundamental shift in how medical professionals’ work. Healthcare documentation is developing beyond basic transcription toward truly ambient clinical intelligence. Modern examination rooms are becoming smart environments with advanced sensing technologies to understand and document clinical encounters.
This integration creates an intelligent system that:
- Captures and analyzes clinical conversations naturally
- Provides real-time clinical documentation without disrupting patient care
- Offers contextual insights during patient encounters
- Maintains accuracy even in challenging clinical environments
The outcome is an organic healthcare setting, where voice AI technology enhances the patient-clinician relationship. Clinicians can redirect their attention from administrative tasks to patient care, spending more time on complex decision-making and building meaningful relationships with patients.
Healthcare organizations can also offer advanced training and skill development programs through AI-powered systems, services and solutions. Medical professionals can access, and master, new procedures and protocols more efficiently to stay current with evolving medical knowledge.
Voice AI Overview
Deepgram, together with Amazon Web Services (AWS), have put medical-grade voice AI at the heart of this solution, transforming how patients, medical professionals and their staff operate together. Deepgram provides a set of powerful APIs and developer tools for healthcare developers to build the clinician and patient-facing experiences.
By integrating Deepgram Nova-3 Medical with Amazon Bedrock (for generative AI orchestration) and Amazon Elastic Kubernetes Service (for scalable deployment), a robust healthcare solution is created to address critical scaling challenges in healthcare operations of various sizes.
Main Services Use
Following are some of the main services used within the generative voice AI solution and what benefits they bring.
Deepgram
Deepgram is a speech intelligence system that provides APIs and models to transcribe, analyze, and process voice data. It’s a toolkit that healthcare developers can use to build custom applications.
Developers can create the experiences and user interfaces that fit their clinicians, workflows, and patients. Whether it is a digital scribe, automating pharmacy refills, or designing entirely new experiences, Deepgram provides the building blocks and real-time voice intelligence to make it possible.
Deepgram Nova-3 Medical
Deepgram Nova-3 Medical is an advanced speech-to-text AI model specifically trained on healthcare terminology. The model delivers highly accurate, real-time transcription even in noisy clinics. The model outperforms competitors with a 63.7 percent improvement in word error rate (WER) and a 40 percent boost in capturing critical medical terms. Built for flexibility, it offers near-instant responses, HIPAA compliance, and streamlines ways to adapt to new drug names, specialties, or workflows, all while scaling smoothly with patients’ needs.
Amazon Bedrock
Amazon Bedrock is a fully managed service that provides streamlined access to high-performing foundation models (FMs) from leading AI companies. It offers a unified API, advanced security and privacy controls, and customization options. Developers can build and scale generative AI applications quickly and securely.
Amazon Elastic Kubernetes Service
Amazon Elastic Kubernetes Service (Amazon EKS) is a managed Kubernetes service that streamlines the deployment, management, and scaling of containerized applications. It provides a highly available and secure environment for running Kubernetes workloads, integrating seamlessly with other AWS services.
Architecture
Deepgram offers two deployment options:
- Deepgram self-hosted deployment on AWS
- Deepgram Dedicated, a Software as a Service (SaaS) deployment
Figure 1: Deepgram self-hosted deployment on AWS
Deepgram Self-Hosted Deployment on AWS
This approach verifies healthcare organizations’ data remains within their own AWS accounts, providing full control and seamless integration with existing healthcare IT infrastructures (Figure 1).
The Deepgram system is deployed on an Amazon EKS cluster, hosting inside the customer’s Amazon Virtual Private Cloud (Amazon VPC). Customers access the system through Amazon API Gateway, which securely routes requests to the Amazon EKS cluster through Secure Web Sockets or HTTPS. Amazon Simple Storage Service (Amazon S3) and Amazon Elastic Container Registry (Amazon ECR) provide essential storage and container management capabilities.
In the customer’s AWS account, Amazon Bedrock orchestrates foundation models, patient registration and appointment booking processes. AWS Lambda functions integrate with a Knowledge Bases and an electronic health records (EHRs) Fast Healthcare Interoperability Resources (FHIR) API endpoint. This serverless architecture provides scalable, secure processing of healthcare data workloads.
Figure 2: Deepgram Dedicated, a SaaS deployment
Deepgram Dedicated, a Software as a Service (SaaS) Deployment
Deepgram Dedicated is a single-tenant SaaS deployment option for customers seeking a Deepgram-managed environment that reduces operational overhead, while maintaining HIPAA compliance and data security (Figure 2). Client applications first connect to the Amazon API Gateway and Amazon EKS cluster within a Deepgram AWS account, then establish a connection to Amazon Bedrock in the customer’s AWS account.
The Knowledge Base, customer application stack, and associated tools are hosted within the customer’s AWS account. Amazon Bedrock handles integrations through AWS Lambda functions, interfacing with both the custom Knowledge Base and an EHRs FHIR API for comprehensive healthcare data processing.
Use Case Examples
Healthcare organizations can adopt this integrated solution across critical workflows, transforming traditional processes into efficient, AI-powered operations.
Following are four use cases that show immediate value in clinical settings:
- AI-Powered medical scribes: Streamline clinical documentation by transcribing doctor-patient conversations in real-time, auto-generating clinical notes with precision. This reduces the administrative burden on healthcare professionals, allowing them to focus more directly on patient care.
- Automated drug prescriptions and refills: Enable automated interpretation of physician orders. This facilitates seamless integration with pharmacy management systems for timely and accurate medication dispatching, reducing the likelihood of human error.
- Enhanced shipping and logistics: Improve medication dispatching and supply chain management through automated interpretation of orders and integration with logistics systems.
- Intelligent registration and scheduling: Automate patient registration and appointment scheduling, cancellations, and follow-ups through conversational AI, improving efficiency and patient experience.
Conclusion
The combination of the specialized medical voice AI capabilities of Deepgram with the robust infrastructure of AWS creates a foundation for sustainable digital transformation in healthcare. The two flexible deployment options cater to diverse organizational needs, delivering HIPAA-aligned operations.
If your organization is ready to experience voice-to-text technology, visit the Deepgram playground and sign up for a trial account at no additional cost to test drive it using your own voice. Then, subscribe to Deepgram Nova-3 Medical through the AWS Marketplace. A trial is available at no additional cost to help you experience firsthand how this solution can revolutionize your clinical operations and patient engagement.
Contact an AWS Representative to know how we can help accelerate your business.
Learn more
- Talk to a Deepgram Expert
- Deepgram recorded demo
- Deepgram deployment guide
- Amazon Bedrock model response
- Amazon Bedrock user guide
- Amazon EKS user guide
.
Deepgram – AWS Partner Spotlight
Deepgram helps companies harness the potential of their voice data with intelligent, tailored speech models built to increase revenue and maximize efficiency. Our speech recognition gives product, operations, and analysis teams high-accuracy, high-performance voice tools that scale as they do.