Amazon EC2 P6e-GB200 UltraServers and P6-B200 instances

With P6e-GB200 UltraServers, customers can access up to 72 Blackwell GPUs within one NVLink domain to use 360 petaflops of FP8 compute (without sparsity) and 13.4 TB of total high- bandwidth memory (HBM3e). P6e-GB200 UltraServers provide up to 130 terabytes per second of low-latency NVLink connectivity between GPUs and up to 28.8 terabits per second of total Elastic Fabric Adapter networking (EFAv4) for AI training and inference. This UltraServer architecture on P6e-GB200 enables customers to leverage a step change improvement in compute and memory, with up to 20x GPU TFLOPS, 11x GPU memory, and 15x aggregate GPU memory bandwidth under NVLink compared to P5en.

P6-B200 instances provide 8x NVIDIA Blackwell GPUs with 1440 GB of high bandwidth GPU memory, 5th Generation Intel Xeon Scalable processors (Emerald Rapids), 2 TiB of system memory, up to 14.4 TB/s of total bidirectional NVLink bandwidth, and 30 TB of local NVMe storage. These instances feature up to 2.25x GPU TFLOPs, 1.27x GPU memory size, and 1.6x GPU memory bandwidth compared to P5en instances.

P6e-GB200 UltraServers and P6-B200 instances are powered by the AWS Nitro System with specialized hardware and firmware designed to enforce restrictions so that no one, including anyone at AWS, can access your sensitive AI workloads and data. The Nitro System, which handles networking, storage, and other I/O functions, can deploy firmware updates, bug fixes, and optimizations while it remains operational. This increases stability and reduces downtime, which is critical to meeting training timelines and running AI applications in production.

To enable efficient distributed training, P6e-GB200 UltraServers and P6-B200 instances use fourth-generation Elastic Fabric Adapter networking (EFAv4). EFAv4 uses Scalable Reliable Datagram protocol to intelligently route traffic across multiple network paths to maintain smooth operation even during congestion or failures.

P6e-GB200 UltraServers and P6-B200 instances are deployed in Amazon EC2 UltraClusters that enable scaling up to tens of thousands of GPUs within a petabit-scale nonblocking network.

Each NVIDIA Blackwell GPU features a second-generation Transformer Engine and supports new precision formats such as FP4. It supports fifth-generation NVLink, a faster, wider interconnect delivering up to 1.8 TB/s of bandwidth per GPU.

The Grace Blackwell Superchip, a key component of P6e-GB200, connects two high- performance NVIDIA Blackwell GPUs and an NVIDIA Grace CPU using the NVIDIA NVLink-C2C interconnect. Each Superchip delivers 10 petaflops of FP8 compute (without sparsity) and up to 372 GB of HBM3e. With the superchip architecture, 2 GPUs and 1 CPU are co-located within one compute module, increasing bandwidth between GPU and CPU by an order of magnitude compared to current generation P5en instances.

P6e-GB200 UltraServers and P6-B200 instances provide 400 GB per second per GPU of EFAv4 networking for a total of 28.8 Tbps per P6e-GB200 UltraServer and 3.2 Tbps per P6-B200 instance.

P6e-GB200 UltraServers and P6-B200 instances support Amazon FSx for Lustre file systems so you can access data at hundreds of GB/s of throughput and millions of IOPS required for large- scale AI training and inference. P6e-GB200 UltraServers support up to 405 TB of local NVMe SSD storage while P6-B200 instances support up to 30 TB of local NVMe SSD storage for fast access to large datasets. You can also use virtually unlimited cost-effective storage with Amazon Simple Storage Service (Amazon S3).

Amazon SageMaker is a fully managed service for building, training, and deploying ML models. With Amazon SageMaker HyperPod (P6-B200 support coming soon), you can more easily scale to tens, hundreds, or thousands of GPUs to train a model quickly at any scale without worrying about setting up and managing resilient training clusters.

AWS Deep Learning AMIs (DLAMI) provides ML practitioners and researchers with the infrastructure and tools to accelerate DL in the cloud, at any scale. AWS Deep Learning Containers are Docker images preinstalled with DL frameworks to streamline the deployment of custom ML environments by letting you skip the complicated process of building and optimizing your environments from scratch.

If you prefer to manage your own containerized workloads through container orchestration services, you can deploy P6-B200 instances with Amazon Elastic Kubernetes Service (Amazon EKS) or Amazon Elastic Container Service (Amazon ECS).

P6e-GB200 UltraServers will also be available through NVIDA DGX Cloud, a fully managed environment with NVIDIA’s complete AI software stack. You get NVIDIA’s latest optimizations, benchmarking recipes, and technical expertise.

Learn more