Democratizing GenAI through a Global Enterprise Portal

Generative Artificial Intelligence (GenAI) is a disruptive technology that can create new and original content, such as text, images, audio, video, and software code. To realize the full potential of this technology, companies may democratize GenAI through a Global Enterprise Portal (GEP) to support use cases across all business units. GenAI use cases span a vast number of areas of domestic and work life, where GenAI is used for content creation and editing, creativity and recreation, research, analysis, and decision making [see HBR Article, 2024]. When adopting and deploying this technology across a global organization, business and IT-decision makers must carefully weigh the trade-offs between centralization and decentralization [AWS Blog Post, 2021]. This trade-off defines which capabilities are made available to users centrally and which allow for use case specific customization. In this blog post, we introduce a novel framework to democratize and scale GenAI through a GEP that allows for centralizing foundations while decentralizing innovation [AWS Blog Post, 2024]. Additionally, we will examine two customer examples demonstrating how the GEP framework can be used to democratize GenAI across a global enterprise.

Figure 1: Key business challenges (top row, blue) companies currently face with respect to data and GenAI and the solutions tenets (bottom row, green) used to overcome them.

GEP Framework

The GEP framework was developed by working backwards [AWS Executive Insights, 2024] from five common business challenges related to data and GenAI (as depicted in Figure 1). First, businesses face significant data quality challenges because available data is often unstructured and poorly curated [HBR Article, 2024]. Such poor data quality makes it difficult for users to build valuable data products, including representative training datasets for GenAI models. Second, assets (such as data, models, software code, etc.) are often distributed across the organization or hidden in organizational silos (i.e., isolated repositories that cannot be accessed by the different departments within an organization). Third, many organizations face significant challenges and difficulties in accessing valuable digital assets. Many organizations lack clear permission structures for data products, artificial intelligence/machine learning (AI/ML) solutions, and generative models, creating barriers when teams attempt to leverage these resources for their specific business needs. Fourth, the lack of clear permission structures often hinders effective collaboration and asset sharing across an organization. Fifth, promoting responsible AI, security, and privacy standards [AWS website, 2025] is very time consuming and complex for customers in the absence of a GEP that democratizes data products [HBR Article, 2022] and GenAI capabilities alike.

To overcome these challenges, Amazon Web Services, Inc. (AWS) developed solution tenets [AWS Blog Post, 2023] to guide the implementation of GEP. In particular, AWS’ GEP solution tenets include:

1. Enterprise asset management capabilities allowing for creating and managing assets at scale;
2. An asset catalog and repository [see Atlan, 2024] for discovering data products, training datasets, prompt templates, application programming interfaces (APIs), AI/ML/GenAI models, and events allowing for the creation of automated workflows;
3. A unified enterprise portal that offers a seamless user-interface/user-experience (UI/UX);
4. Multi-account and multi-region design tailored to the global footprint of the organization; and
5. End-to-end governance capabilities based on AWS’ responsible AI principles [see AWS Website, 2024] that generally assist customers in creating policies, procedures, and guidelines governing its development, deployment, and use of AI/ML models within an organization.

Figure 2: The GenAI Adoption Staircase depicts the path to business value creation over time.

The implementation of a comprehensive GEP typically follows a progressive four-phase approach, illustrated in the GenAI Adoption Staircase depicted in Figure 2. This approach recognizes the foundational relationship between data, traditional AI/ML capabilities, and newer GenAI technologies. While the first phase involves building cloud-native core infrastructure comprised of an AWS Landing Zone [see AWS Prescriptive Guidance, 2024], the second phase involves building out various data capabilities, such as data product creation and cataloging. In this phase, business value is created from enabling use cases that involve sharing and/or monetization of data products [HBR Article, 2022]. The third phase advances existing infrastructure by adding AI/ML capabilities, such as access to AI/ML models, training, inference, and playgrounds to enable the adoption of AI/ML across the customer organization. The last phase creates business value by making a diverse set of GenAI capabilities globally available and providing access to vector databases, embedding models, and state-of-the-art large-language models (LLMs) that can be used to implement different GenAI use cases [McKinsey Digital, 2024].

The AWS GEP framework, shown in Figure 3, is based on the producer-consumer model [AWS Blog Post, 2024]. It enables producers to upload assets—namely data products, models, training datasets for fine-tuning models, and events—and allows consumers to access those assets to implement their use case. Access to GEP is provided through a unified self-service interface that seamlessly integrates with all features of the portal through use case-specific micro-frontends. GEP comprises three types of capabilities, namely:

1. Centralized Capabilities: Reusable capabilities that are made available to all users, such as the asset catalog and cost and security monitoring;
2. Decentralized Capabilities: Use case-specific capabilities, such as a text summarization tool or demand forecasting service, that are not shared across multiple use cases; and
3. Core Capabilities: Capabilities that are required for every cloud environment. This includes an AWS Landing Zone and fully managed and automatically provisioned compute and storage capabilities.

Figure 3: High-level design of the AWS GEP. Producers, consumers, and the portal team can login via a unified enterprise portal with an easy-to-use and seamlessly integrated UI/UX. The portal provides various centralized (solid line, green), decentralized (solid line, blue), and core capabilities (dashed line, white) shown as examples to address enterprise-wide user needs.

GEP Solution Features

When developing an enterprise platform using the GEP framework, organizations should establish comprehensive data governance frameworks during phase two of the GenAI Adoption Staircase (Figure 2) to avoid each use case pursuing its own unsanctioned “shadow GenAI” solution [see MIT Sloan, 2024]. This governance forms the foundation and continues to evolve as GEP matures from phase to phase. For example, while assets in the data phase are limited to structured and unstructured data, they can be expanded to vector databases, fine-tuned foundation models (FMs), and LLM agents in the GenAI phase.

For the remainder of this blog post, we will focus on the GenAI phase and introduce all the solution features required to unlock the GenAI phase, namely: (1) a unified API and self-service for accelerated use case development; (2) a GenAI marketplace to drive enterprise-wide adoption; (3) operational intelligence; and (4) multi-region and multi-tenant design.

Unified API and self-service for accelerated use case development
GEP provides access to approved foundation models, such as LLMs and image generation models, through a standardized API layer that resolves the complexity of accessing different models, including Amazon Bedrock, third-party, and self-hosted solutions. This API layer enables developers to seamlessly switch between models without having to rewrite their code to match the API of a new provider or undergo lengthy approval processes. Instead, developers can focus on work that is specific to their use case.

Moreover, the API layer provides production-ready, higher-level capabilities that typically require substantial infrastructure investment. This may include a chatbot functionality with built-in conversation state management, for example, or end-to-end Retrieval-Augmented-Generation (RAG) pipelines (RAG-as-a-service) handling document ingestion, embedding, and retrieval. Since GEP provides RAG-as-a-service, building a new RAG-powered chatbot only requires API calls, eliminating the need to provision and managing complex vector databases manually. All users can access these APIs through automated self-service approval workflows, enabling them to focus on use case implementation rather than approval processes and manual cloud infrastructure provisioning.

GenAI marketplace to drive enterprise-wide adoption
To accelerate enterprise-wide adoption, GEP provides a GenAI playground UI enabling technical and non-technical users to experiment with various AI capabilities, including RAG. In dedicated workspaces, users can upload and index documents using vector databases, experiment with approved FMs through a user-friendly interface, and build and test custom prompts for their specific use cases. GEP’s asset catalog and repository allows users to publish their chatbot applications directly, making them immediately available to other users in the organization.

This self-service approach removes traditional barriers to AI adoption and enables customers to define and enforce responsible AI principles, security, and compliance standards across their organization. Simple applications can be shared through the marketplace while more complex use cases created in the playground can be developed into full-scale applications using GEP’s unified API capabilities.

Operational intelligence
Effective GenAI governance at enterprise scale requires comprehensive visibility into usage, compliance, security, and costs. GEP provides centralized monitoring capabilities spanning three critical dimensions:

1. Central monitoring: GEP tracks all core GenAI functionalities through a unified dashboard. This includes visibility into API calls and automated guardrail enforcement. The dashboard enables operators to ensure service quality while maintaining compliance with enterprise policies;
2. End-to-end traceability: GEP provides complete lineage of all GenAI interactions. It tracks which data sources are used for retrieval, which models process this data, and how the outputs are utilized. This transparency is crucial for maintaining accountability and supporting audit requirements in regulated environments; and
3. Cost monitoring: GEP enables detailed cost monitoring to help organizations understand and optimize their GenAI investments. It tracks usage costs enabling allocation to relevant business units. This granular view not only provides financial transparency but also helps identify optimization opportunities, such as shifting workloads to more cost-effective models or optimizing prompt lengths to reduce token consumption.

Multi-region and multi-tenant design
GEP is based on a multi-account design allowing for scaling the portal based on increasing resource needs while maintaining strict isolation between different teams and business units within the organization. All asset requests flow through a shared account that acts as a central gateway, then route to dedicated AWS use case accounts that host compute resources for model inference and storage for vector databases as depicted in Figure 4.

Figure 4: High-level access workflow of users to GEP: Users access GEP through the Playground & Marketplace UI (1), and are then redirected to the closest regional API endpoint (2) in the shared account (green, dashed lined) while use case applications can access their preferred regional API directly (3). A serverless cross-region tenant management system (4) identifies the tenant to which the request belongs and redirects it (5) to the respective dedicated use case AWS account (blue, dotted line) to processes the request through a serverless backend. This can be ingestions into the RAG vector database using input data published through GEP, or pure model calls to AWS Bedrock, self-hosted AWS SageMaker, or third-party models (7).

This architecture provides several strategic advantages, including allowing users to:

1. Centralized Security and Governance: Implement unified governance and guardrails across all AI workloads, enable centralized management of model access and usage policies, and allow standardized security controls while maintaining tenant isolation;
2. Resource Isolation: Create natural security boundaries for different AI workloads and environments, increase operational resiliency by limiting the blast radius of potential security incidents or misconfigurations, and provide granular access controls and permissions for different user groups;
3. Compliance Management: Facilitate implementation of distinct security controls per environment, enable users to create of separate data perimeters for sensitive information, and simplify auditing through account-specific CloudTrail logs;
4. Workload Management: Accelerate AI adoption through unified model access and component reusability and allow teams to experiment and innovate independently without affecting production systems;
5. Cost Control: Enable precise tracking and auditing of model usage costs per tenant or line of business, simplify cost attribution and budget management across different department, and provide clear billing boundaries for different AI initiatives;
6. Enterprise Integration: Enable easy access through a unified portal that is seamlessly integrated into enterprise legacy systems; and
7. Scalability: Prevent resource contention as each account gets its own service quotas, support multiple IT operating models and organizational structures, and facilitate easy integration of new teams or departments without disrupting existing workloads.

GEP’s multi-region deployment addresses the following enterprise requirements: It enables customers to manage their data residency configurations according to their own compliance requirements, reduces latency for global operations, and improves overall system resilience through geographic distribution. Organizations can deploy tenants in specific regions while maintaining centralized control and monitoring through the shared gateway account.

Selected Use Cases and Business Outcomes

This GEP framework can be used by large manufacturers to accelerate the secure launch of various GenAI applications, such as a: (1) chatbot for repair and maintenance service engineers to reduce the downtime of automotive production lines; and (2) specification harmonization pipeline that standardizes product specifications across different departments, each of which is described in more detail below.

Chatbot for repair and maintenance services
Manufacturing line errors, for example, caused by calibration drifts in assembly robots, can halt production and potentially result in large financial losses. However, effective troubleshooting often requires technical expertise acquired over decades. Unskilled engineers struggle to find solutions in extensive documentation while many experts are nearing retirement.

Manufacturers can use GEP to build a conversational search engine based on RAG. This tool can help engineers quickly identify error resolution actions and locate relevant documentation from over 500,000 pages. The team could transfer their documentation to an Amazon S3 bucket governed by GEP and perform pre-processing to standardize it for RAG. Moreover, teams can leverage GEP’s GenAI RAG service and pre-built chatbot features and focus their attention on use-case specific activities, such as prompt engineering.

The raw and preprocessed documentation governed through the portal can be a valuable asset for other use cases, which could use it as consumers. The architecture blueprints are reusable for similar chatbots in the manufacturing and other industrial domains, enabling further acceleration of future use cases.

Specification harmonization pipeline
In certain use cases, manufacturers might desire to standardize their product specifications across all departments to reduce inefficiencies in product development and delays in time-to-market. However, manually harmonizing specifications may take years of expert effort.

In such cases, manufacturers can leverage the GenAI capabilities of GEP to implement an intelligent specification-based standardization workflow. This solution allows users to submit existing specification documents which are then processed by an agentic workflow (i.e., a workflow that is executed automatically by GenAI). This workflow automatically extracts relevant requirements, aggregates them, and reformats them according to the corporate standard, which reduces inefficiencies and accelerates product development.

Conclusion

The path to democratizing GenAI across a global organization demands a balanced approach between centralized control and democratized innovation. The AWS GEP framework delivers precisely this balance, as demonstrated by the two use cases implemented by a large automobile manufacturer. Organizations that move decisively to implement such portals may gain lasting competitive advantages through accelerated innovation and enhanced collaboration. The GEP framework is here! Assess your position on the GenAI Adoption Staircase (Figure 2) and take the first step toward enterprise-wide GenAI democratization today!