Cutting Workload Cost by up to 50% by Scaling on Spot Instances and AWS Graviton with SmartNews

Created in 2012, SmartNews is an application that uses ML to scan through more than 10 million articles from 14,000 trusted publishers daily to choose what news to deliver to readers. The app has been downloaded over 50 million times worldwide. Since 2023, the company has had a deal with a major Japanese mobile operator for the SmartNews app to come preinstalled on the operator’s phones, further enlarging the app’s user base.

SmartNews has been using AWS to power its application but wanted to optimize its costs as it grew. On the backend, SmartNews has a containerized architecture on Amazon Elastic Kubernetes Service (Amazon EKS), a managed Kubernetes service, which it has used since 2021. “We use Amazon EKS to create an environment where all the engineering teams can deploy seamlessly while keeping their workloads relatively uniform,” says Ankit Singhal, group head of core infrastructure at SmartNews. In 2024, SmartNews implemented Karpenter, a Kubernetes node lifecycle manager that was developed by AWS and designed to optimize application availability, performance, and cost. Karpenter dynamically scales Kubernetes clusters in near real time, responding to changes in application demand and resource requirements while intelligently selecting cost-effective instance types—including On-Demand, Spot, and Reserved Instances—across multiple Availability Zones.

SmartNews then looked into adopting Spot Instances to further optimize costs, and it worked alongside specialist teams at AWS to learn the best practices and monitoring strategies for using Spot Instances with Amazon EKS. SmartNews also migrated workloads such as its ML inference and news feed to AWS Graviton–based Amazon EC2 instances for further performance and cost optimization. Working alongside the AWS team, the company made decisions on its infrastructure strategy and optimized its code to run successfully on AWS Graviton–based instances. With these two large migrations, SmartNews managed to lower its costs.

By using Spot Instances, SmartNews cut costs for the main compute workload by up to 50 percent. The company has largely predictable workloads—usage is low during the night and consistently scales up at times of the day when people in Japan and US time zones are logging in to check the news. SmartNews scales its capacity up and down using AWS, and it uses an internal monitoring system alongside best practices such as a price-capacity optimized allocation strategy. For some of its backend service workloads, the company uses a mix of 70 percent Spot Instances and 30 percent other payment plans.

Additionally, SmartNews dynamically scales up its cluster when the company sends out a breaking news alert so that the application can respond more quickly to an uptick in traffic. “Using Spot Instances has been a success story for the company,” says Singhal. “We already know what capacity we will need depending on the criticality of the news, and we scale up the systems appropriately.”

SmartNews optimizes costs and performance by using AWS Graviton, lowering the compute costs for its ML workloads by 15 percent. Using AWS Graviton–based instances, the company also reduced latency for these workloads from 190 milliseconds to 60 milliseconds, a 68 percent decrease. Now, SmartNews uses AWS Graviton for 60–70 percent of its ML workloads and is migrating its data pipelines to AWS Graviton as well. Its news ranking algorithms run using Amazon EC2 C7g Instances, which are powered by AWS Graviton3 and provide high price performance in Amazon EC2 for compute-intensive workloads.

“The prime motivation for using AWS Graviton was to save costs, but as a by-product we are getting performance improvements,” says Singhal. “We have seen workloads that now use fewer resources to serve the same number of requests on AWS Graviton.” The company has migrated its Apache Spark workloads to AWS Graviton–based instances as well. Nearly all of its Spark workloads are running on both AWS Graviton and Spot Instances, using Amazon EC2 R6g Instances, powered by AWS Graviton2 processors, and Amazon EC2 R7g Instances, memory-optimized instances powered by AWS Graviton3 processors. This provides a maximum of 8,000 CPU cores and an average of 2,500 CPU cores.

Combining AWS Graviton and Spot Instances, SmartNews has significantly lowered monthly costs and improved performance with no negative impact on customers. Those cost savings contribute directly to future innovation as the company uses them to improve its product and ML models. “All the cost savings that are associated with infrastructure are being reinvested to expand the business and develop new products,” says Singhal.

SmartNews is working to adopt additional AWS services, such as Amazon CloudFront, a content delivery network service. And the company is already using Amazon Bedrock, a fully managed service that offers a choice of high-performing foundation models from leading artificial intelligence companies, to develop new models for classifying and ranking news articles. “We are looking forward to continuing to scale on AWS as we move into the future,” says Singhal.