Innovating New Features Quickly Using AWS Graviton–Based Instances with Read AI

Founded in 2021, Read AI delivers an AI copilot wherever users work that makes meetings, emails, and messages more valuable to teams through AI-generated summaries, transcripts, and highlights. In 2024 alone, Read AI has grown by 730 percent—26 times faster than its industry peers—and taken the top place in the industry in terms of web traffic.

Read AI has been rapidly delivering new features that customers want—for example, custom integrations with business applications including Google Meet, Zoom, Slack, Microsoft Teams, Salesforce, Hubspot, Gmail, Outlook, Jira, Confluence, and others. So, the company has stayed ahead of the curve in its industry, growing organically as users recommend its solution to others. In 2023, after a period of sustained growth, Read AI had two dozen employees who served millions of customers while still a seed-stage startup. Therefore, it needed the ability to experiment with, expand, and release new features quickly and cost-effectively.

Read AI evaluated several proprietary CPU types and, in December 2023, decided to migrate its CPU workloads to Graviton instances. “AWS Graviton–based instances were cost-effective, and the available cores provided the capacity we needed to deliver an AI copilot wherever you work,” says Rob Williams, cofounder and chief technology officer at Read AI. “We got great throughput with a lower total cost.”

Read AI began migrating to Graviton instances in December 2023. As of October 2024, Read experienced over 700 percent growth in active users, and it runs 89 percent of its CPU workloads, which constitute 41 percent of its total workloads, on Graviton instances. The company plans to move all compatible CPU workloads to Graviton instances. “When we migrated the services to Graviton instances, there was no tuning necessary and nothing additional required, and we immediately saw a 20 percent reduction of infrastructure costs,” says Williams.

In addition, Read AI boosted performance using Graviton instances. “Reliably across the board, we achieved performance gains on Graviton instances compared with the x86 counterparts, and we also saw significantly greater flexibility than what we could get with our GPU fleet,” says Williams. “They have a lower sticker price, and we can get more done in each unit of time as well. There was no reason not to migrate to Graviton instances.”

With the flexibility and the performance boost of using Graviton instances, Read AI increases its speed to market for new features. Developers can spin up additional resources to run experiments at low cost and without worrying about capacity. “Graviton instances work the way our developers are used to, and they’re better suited for many development scenarios because they more closely match the hardware that developers use,” says Williams. Additionally, Read AI can move quickly because it runs many workloads, including ML inference, using CPUs on Graviton instances.

This fast experimentation is a defining characteristic of Read AI, one that helped it gain new customers rapidly. Its customers can connect their Read AI environments to other enterprise tools—such as Microsoft Outlook, Gmail, and Microsoft Teams—with a few clicks. “We make it completely seamless for them,” says Williams. “We make efficiency gains from using Graviton instances, and our users benefit from them too.”

Read AI can also support international customers using Graviton instances, which are available globally. Around 30 percent of Read AI’s customers are outside the United States, particularly in Africa and South America. Using Graviton instances, Read AI can help customers comply with data residency regulations. “We can support a diverse set of customers and features using AWS, so almost any information worker, regardless of industry or geography, can get value out of our product,” says Williams.

Read AI is currently adopting AWS Graviton4 instances for additional performance benefits. “We want to be on the preview list for every release of Graviton instances so that we can evaluate them as soon as possible,” says Williams. “Using Graviton instances means that we are comfortable being more diverse in our experiments, so engineers can spin up additional resources to run even small experiments, which makes us very nimble going forward.”

In October 2024, the company introduced a bundle of new features, including an iOS application for real-world meetings and a Chrome extension—Read AI for Gmail—that brings AI to users’ inboxes. The company is also expanding its generative AI–powered experiences. “Using Graviton instances, we can ship diverse features with a quick turnaround, even with a small team,” says Williams.