DeepSeek R1 Distill Qwen 1.5B

Follow the steps below to launch, configure, and run the DeepSeek LLM-Inference stack in your own AWS account.

1. Prerequisites

• AWS account with permissions to create VPC, EKS, EC2 (g5/ p4/p5), ACM, IAM, SSM, and SNS resources.
• Sufficient g5/p4/p5 GPU quota in two AZs of your chosen region. You need a quota of atleast 8 vCPUs to launch the stack on the cheapest available A10 GPU instance (g5.xlarge). You can request the quota increase at https://us-east-1.console.aws.amazon.com/servicequotas/home/services/ec2/quotas/L-DB2E81BA 
• (Optional) Existing VPC with >= 2 private subnets that have NAT-gateway egress if you plan to reuse your own network. Otherwise, the stack will automatically create a new VPC and subnets in your AWS account.

2. Subscribe & launch

• Click Continue to Subscribe on the AWS Marketplace page and accept the terms.
• Choose Continue to Configuration -> Continue to Launch.
• Select the CloudFormation delivery option and choose the region in which you want to deploy.
• Press Launch to open the stack in CloudFormation, then Next to the parameter wizard.

3. (Optional) Configure stack parameters

• ClusterName - Friendly name for the EKS cluster (default llm-inference).
• CreateVPC - true (build a new /16 VPC) or false (use an existing one automatically discovered) (default set to true). Leave the parameters at their defaults unless you have a specific reason to change them. Click Next, add optional tags, then Create stack.

4. Monitor deployment (~15 - 20 min)

• In the CloudFormation console watch the Events tab; status will progress through CREATE_IN_PROGRESS -> CREATE_COMPLETE.
• First, CloudFormation creates an EKS cluster.
• Next, CloudFormation starts an EC2 instance to setup the EKS cluster. The EC2 instance installs Kubectl, Helm, Git, and the NVIDIA device plugin and sets up the helm chart on the EKS cluster. During this Phase, a GPU node joins the EKS cluster. Once the EKS cluster setup is finished, the EC2 instance self-terminates and signals success.
• Finally, a validated ACM certificate is issued, allowing the load balancer on the EKS cluster to be accessed via a friendly URL.

5. Locate your endpoint when the stack deployment is complete:

• Open the Outputs tab; copy the value of CustomerDomain (e.g., https://<your-unique-identifier>.aws.aumlabs.ai/docs).
• The same URL is stored in SSM Parameter Store at /eks/<ClusterName>/customer-domain

6. Test the model Open https://<your-unique-identifier>.aws.aumlabs.ai/docs to access the auto-generated Swagger / OpenAPI UI, or invoke directly:

The API endpoint also supports batch calls. You can send a comma separated list of prompts.

Expect a JSON response containing the model's reply in the following format:

7. Operate & scale After launch you can manage the cluster just like any EKS environment: scale nodes, or extend with additional micro-services if needed. The service is already production ready so you don't need to make changes unless absolutely required.

8. Cleanup Delete the CloudFormation stack to remove all resources: EKS, GPU instances, load balancer, VPC (if created), ACM certificate, IAM roles, and SSM parameters. There is no leftover cost once the stack is gone.

This product does not collect or export customer data to external systems outside of your AWS account.

Need help or have feature requests? Use the Support tab on the AWS Marketplace listing or email support@aumlabs.ai .