Audiense Reduces Query Times by 1,400% After Migrating to Amazon OpenSearch Service

Audiense is a social media analytics provider that empowers businesses to understand audiences through social network data analysis. The company’s “Discover Twitter user” feature, which lets customers search and filter X (formerly Twitter) profiles based on keywords, demographics, and other criteria, experienced performance issues as the volume of data grew, making the feature’s analysis less reliable.

Audiense used Apache Solr on AWS to power the “Discover Twitter user” feature for 9 years. This solution was adequate when the company was smaller, but Audiense’s growth created demands that exceeded the existing system’s capabilities. Weekly indexing operations would cause 3–4 days of degraded performance per week; query response times expanded from 10–20 seconds up to 3 minutes due to the growing volume of data. Perhaps most critically, the feature could index data only once weekly. Indexing is the process of organizing data into a searchable structure that facilitates the efficient retrieval of information. With weekly indexing, users often accessed information that was up to 5 days old—a limitation in the fast-moving world of social media.

To improve performance and operational efficiency, Audiense migrated to Amazon OpenSearch Service. The company had previously used Amazon OpenSearch Service and knew it was compatible with its existing query language, which would facilitate a smoother migration. Audiense was also aware of the benefits of using a fully managed service that would reduce operational overhead for its team.

“We had already used Amazon OpenSearch Service to develop another solution at Audiense, so this was our second implementation,” says Jon Colas Gomez, Linux system administrator/DevOps at Audiense. “This meant we could use existing tools, procedures, and documentation that our team had already developed. We believe it’s better to standardize on common technologies rather than introducing entirely new ones.”

Audiense migrated its “Discover Twitter user” functionality from Apache Solr to Amazon OpenSearch Service, carefully planning the transition to avoid disrupting users. The project began in June 2024 and concluded in September, with 2 weeks of A/B testing to compare performance between the old and new systems.

Audiense implemented Amazon OpenSearch Service using different Amazon Elastic Compute Cloud (Amazon EC2) instances, which provide secure and resizable compute capacity for virtually any workload. Audiense used Amazon EC2 I4g Instances—storage-optimized instances powered by AWS Graviton2 processors—for data nodes and Amazon EC2 M7g Instances—general-purpose instances powered by AWS Graviton3 processors—for primary nodes. The team also used open-source Terraform for infrastructure as code, making it simple to create and replicate the infrastructure as needed.

“We haven’t touched Amazon OpenSearch Service since we finalized the development,” says Miguel Echenique, software developer at Audiense. “We haven’t experienced any downtime. Everything has worked.”

The new implementation resolves several critical pain points for Audiense. Most importantly, it facilitates near real-time indexing instead of the weekly batch indexing process that had previously caused downtime and meant users were often viewing stale data. The performance improvements were immediately apparent: Audiense reduced query time by 1,400 percent. “During the A/B testing, we experimented with the time of the queries, and it was exponentially better using Amazon OpenSearch Service,” says Echenique. “We went from 10–20 seconds per query to 20 ms. It was incredible.”

Audiense also enjoys operational benefits by migrating to Amazon OpenSearch Service. The “Discover Twitter user” feature has achieved virtually 100 percent uptime since implementation, whereas the previous system was stable only 4–5 days per week. The switch to a managed service has reduced the operational overhead on Audiense’s site reliability engineering team, who previously spent significant time manually maintaining the Apache Solr infrastructure. Now, they need only approximately 30 minutes every 2–3 months for version upgrades.

The successful migration to Amazon OpenSearch Service has transformed Audiense’s capabilities by providing a high-performing foundation for social media search. The exceptional speed and stability of the new solution has made the feature more reliable, and the managed service aspect has decreased the maintenance burden on Audiense’s engineering team. Now, they can focus on finding new opportunities for innovation.

Audiense’s “Discover Twitter user” feature had been made less prominent on its platform due to the previous performance limitations. This improvement in both performance and reliability has directly enhanced customer satisfaction through more consistent access to fresh data. With robust search functionality now in place, Audiense is planning to highlight this capability in its user interface and develop advanced analytics tools on top of it, transforming what had become an underused feature into a cornerstone offering.

“Now that we have 100 percent uptime using Amazon OpenSearch Service, we will create new functionality for this feature that we couldn’t build previously,” says Echenique. “We don’t know what this will look like yet, but we are definitely looking forward to upgrading our product.”