AWS LLM League sparks AI innovation at AWS Summit Washington, DC 2025

Contributing authors: Mohan CV, Rakesh Raghu, Jaya Padma Mutta, Rajesh Babu Nuvvula, Steven Koufoudakis, Vu Le, and Jai Vumma

Artificial intelligence (AI) has the potential to revolutionize how we work, but harnessing this transformative technology can seem daunting—especially for those new to the field. That’s why, we at Amazon Web Services (AWS) created the AWS Large Language Model (LLM) League, a collaborative, gamified learning program that empowers builders and organizations to build practical generative AI capabilities.

Recently, the AWS LLM League took center stage at the AWS Summit Washington, DC 2025, captivating attendees with its hands-on approach to fine-tuning large language models (LLMs). Over the course of two days, the AWS LLM League attracted close to 100 eager participants each day, reaching maximum capacity just minutes after the doors opened. This enthusiastic response highlights the strong demand for practical, user-friendly AI education that delivers tangible results.

“The overwhelming interest in our AWS LLM League hands-on sessions reflects our public sector community’s drive to innovate with generative AI,” said Alex Martinez, director of solutions architecture, AWS Public Sector Partners. “Seeing standing-room-only crowds building with these technologies pushes the boundaries of what’s possible to deliver even more efficient, scalable AI solutions to our customers.”

Democratizing AI through gamified learning

The AWS LLM League is a gamified learning program that helps organizations and builders of all backgrounds develop practical generative AI capabilities. Think of it as a prestigious generative AI tournament, where participants compete in curated challenges ranging from crafting effective prompts to tuning generic LLMs.

The program begins with a hands-on workshop led by AWS experts, where participants are introduced to the fundamentals of tuning LLMs and gain exposure to tools like Amazon SageMaker Unified Studio. This no-code/low-code environment empowers even those new to AI to start experimenting with fine-tuning techniques.

Armed with these new skills, participants then enter a competitive phase where they put their model-building abilities to the test. Using SageMaker Unified Studio, they fine-tune language models and submit their solutions to a real-time leaderboard. This leaderboard tracks performance based on an AI evaluation criterion, fostering a collaborative yet engaging atmosphere as builders compete to create the most effective AI-powered solutions.

The culmination of this journey is an exhilarating game show style finale, where the top 5 performers showcase their fine-tuned models and compete live, showcasing the true power of their creations.

By providing this immersive, hands-on learning experience, the AWS LLM League empowers organizations to not only upskill their workforce, but also develop and accelerate practical, customized AI solutions that can drive immediate business impact. Whether you’re a seasoned AI practitioner or new to the field, the LLM League offers a unique opportunity to democratize access to these transformative technologies.

Capturing the AWS Summit experience

Over the course of two dynamic days at the AWS Summit Washington, DC 2025, the AWS LLM League showcased its ability to adapt to diverse business challenges, empowering participants to fine-tune generative AI solutions for both the tourism and municipal services sectors.

We focused the first day’s session on AWS Partners—such as InterVision—who have already shown their AI capabilities through the AWS LLM League program. The AWS LLM League provides a unique opportunity for AWS Partners to build practical skills and unlock new possibilities for their customers. During the workshop, participants developed a chatbot designed to help tourists explore Washington, DC’s monuments, historic sites, and cultural attractions. Attendees leveraged the flexibility of Amazon SageMaker Unified Studio to fine-tune language models that could provide accurate answers and capture the characteristics of the DC voice. Throughout the fine-tuning process, participants’ models were evaluated against a set of benchmark questions using an automated LLM-as-a-judge system, where there smaller, cost-effective, custom model’s responses were compared to larger base models. This allowed builders to iteratively refine their solutions, competing against each other on the real-time leaderboard to secure a spot in the final showdown. The top fine-tuned model on the leader board performed better 90 percent of the time than the larger base models.

Figure 1. Participants engage in model customization during the AWS LLM League hands-on workshop

The finale took place in the energetic environment of the Summit’s GenAI Hub, where the top 5 performers showcased their chatbots in front of a panel of expert judges—representing the perspectives of executive stakeholders—as well as the audience, who played the role of end users. This multi-faceted evaluation system—combining technical assessment, domain expertise, and real-world feedback—ensured the winning solution truly addressed the needs of both the business and its customers.

Figure 2. The LLM League finale held in the energetic exposition hall of the Gen AI Hub

Ganbayar Gansukh, who claimed the first-place prize, reflected: “At AWS Summit Washington, DC, I didn’t just attend—I built, learned, and connected with the future of cloud innovation.”

The duo of Daryoush Dadehbeigi & Vijaya Ramya Chillimuntha took second. Daniel McCloskey, who finished in third place, shared his experience: “The final contest was a fun twist and definitely had me thinking carefully about how to phrase my prompts. I had a great time and I’m looking forward to more LLM League events!”

Figure 3. The top finalist and winners of the first day’s LLM League finale pose with AWS staff

The second day’s session was open to all participants who tackled the challenge of enhancing non-emergency contact center services—a critical function for municipalities worldwide. Participants fine-tuned language models that could effectively address a wide range of non-emergency inquiries, from noise complaints to abandoned vehicles, while maintaining a friendly and helpful tone. The top performing model scored higher than the base model 70 percent of the time.

Despite being the last session of the day, the finale for this session continued to captivate participants eager to see how generative AI could transform civil services in their own communities. Throughout the finale, AWS experts discussed the nuances of building a robust chatbot agent beyond LLM fine tuning. They explored techniques like Retrieval Augmented Generation (RAG) to pull contextual information from websites, as well as the importance of responsible AI practices, including the implementation of guardrails to ensure these models operate within ethical boundaries. They also highlighted the potential for agentic AI agents to handle increasingly complex customer interactions.

Figure 4. Ankur Mehrotra, General Manager of Amazon SageMaker sharing industry trends and guidance for practical generative AI application development

Amit Gathani claimed the first-place prize, while Brooke Sexton took second place. “Coming into this event, with no prior knowledge of LLMs or dataset creation, I was not sure what to expect,” said Brooke. “But the thoughtful guidance, supportive environment, and detailed resources transformed my uncertainty into curiosity and eventually confidence. It was more than a competition; it was a learning opportunity.” Joel Jesuraj rounded out the podium, taking third place.

Figure 5. The top finalist and winners of the second day’s LLM League finale pose with AWS staff

By empowering builders of all backgrounds to fine-tune language models for real-world challenges, AWS is paving the way for a future where AI-driven solutions are accessible to organizations and communities alike. The seamless transition between the tourism and municipal service use cases demonstrated the AWS LLM League’s ability to accommodate a wide range of business scenarios. Whether it’s claims processing, educational curriculum planning, or any other industry-specific challenge, the program’s flexible design allows organizations to tailor the experience to their unique needs, while maintaining the same immersive, hands-on learning environment.

Inspiring the next generation of AI leaders

The success of the AWS LLM League at the AWS Summit Washington, DC 2025 underscores the growing demand for accessible, hands-on AI education. By creating a collaborative, gamified environment that challenges participants to fine-tune LLMs for real-world use cases, AWS is leading the charge in democratizing these powerful technologies and inspiring the next generation of AI innovators.

“The enthusiasm and innovative spirit we witnessed at the AWS Summit Washington, DC is a testament to the transformative power of the AWS LLM League,” said Ankur Mehrotra, general manager of Amazon SageMaker. “By empowering our customers and partners to build practical AI skills, we’re not only advancing the state of the art in generative AI, but also laying the foundation for a future where AI-driven solutions are accessible to all.”

Whether you’re a nascent ML practitioner looking to expand your skills or a leader seeking to enable your organization with generative AI capabilities, the AWS LLM League stands ready to guide you on your journey.

Contact our team today to learn how you can participate in the AWS LLM League.

Figure 6. AWS LLM League technical and program delivery team (from left to right): Mohan CV, Rakesh Raghu, Jaya Padma Mutta, Rajesh Babu Nuvvula, Steven Koufoudakis, Vu Le, and Jai Vumma