What is a GPU?

A graphics processing unit (GPU) is an electronic circuit that can perform mathematical calculations at high speed. Computing tasks like graphics rendering, machine learning (ML), and video editing require the application of similar mathematical operations on a large dataset. A GPU’s design allows it to perform the same operation on multiple data values in parallel. This increases its processing efficiency for many compute-intensive tasks.

A GPU is excellent at performing general-purpose parallel processing, but historically, this wasn’t always the case. As the name suggests, GPUs were initially designed for one specific task: controlling image display.

What is a GPU: Origin of the graphics processing unit

Before the GPU, we had dot matrix screens, which were released in the early 1900s. Vector and raster displays came next, and then later the first video game consoles and PCs were released. At the time, a non-programmable device called a graphics controller coordinated the display with the screen. Graphics controllers traditionally relied on the CPU (central processing unit) for processing, although some included on-chip processors.

Around the same time, there was a 3D imaging project concerned with generating a single pixel on a screen with a single processor. The goal was to produce an image that combines many pixels in a short amount of time. This project was the origin of the GPU as we know it.

It wasn’t until the late 1990s that the first GPUs were released. These were aimed at the gaming and computer-aided design (CAD) markets. The GPU integrated a previously software-based rendering engine and transformation and lighting engine with the graphics controller—all on a programmable chip.

Evolution of GPU technology

Nvidia was the first to market the single-chip GeForce 256 GPUs in 1999. The 2000s and 2010s marked a growth era where GPUs gained functions like ray tracing, mesh shading, and hardware tessellation. These led to increasingly advanced image generation and graphics performance.

It wasn’t until 2007 that Nvidia officially released CUDA, a software layer making parallel processing available on the GPU. Around this time, it became clear that GPUs were very effective at performing highly specific tasks. GPUs excel at tasks that require a large amount of processing power to achieve a particular outcome.

When Nvidia released CUDA, it opened up GPU programming to a wider audience. Developers could then program GPU technology for all sorts of different compute-intensive practical applications. GPU computing started to become far more mainstream.

GPUs are an in-demand chip for simulation and other emerging applications. They’re a key hardware technology in artificial intelligence and machine learning (AI/ML) applications.

GPUs can be used across a wide range of compute-intensive applications, including large-scale finance, defense applications, and research activities. Here are some of the most prevalent uses of GPUs today.

Gaming

The GPU’s first applications that extended beyond large business and government visualization applications were in personal gaming. GPUs were used in the gaming consoles of the 1990s and still are in PCs and current gaming consoles. GPUs are essential for complex graphical rendering.

Professional visualization

GPUs are used in professional applications such as CAD drawing, video editing, product walkthroughs and interactivity, medical imagery, and seismic imaging. They are also applied to other complex image and video editing and visualization applications. Browser-based applications can even use the GPU through libraries such as WebGL and WebGPU.

AI workloads and AI models

Modern graphics processing units are central to artificial intelligence workloads. AI relies on training and running large AI models, having to repeat the same complex calculations to process massive datasets. Unlike a central processing unit, a GPU is designed for parallel graphics processing, making it ideal to handle thousands of operations at once.

Any system with deep learning, image recognition, or other large-scale AI applications is impractical to run on a computer’s CPU. Instead, you can train and deploy AI models on GPUs, for handling the most computationally demanding tasks.

Training a machine learning (ML) model requires a large amount of compute power. Training workloads run on GPUs for accelerated results, and you can achieve results quickly by using a cloud GPU.

Blockchain

Cryptocurrencies are built on blockchains. A particular type of blockchain, proof of work, typically heavily relies on GPUs for operation. Application-specific integrated circuits (ASICs) are a replacement for some types of GPU processing for blockchain.

Proof of stake blockchain algorithmic proofs remove the need for massive amounts of compute power, but proof of work is still pervasive.

Simulation

Advanced simulation applications such as those used in molecular dynamics, weather forecasting, and astrophysics can all be accomplished through GPUs. GPUs are also behind a lot of applications in automotive and large vehicle design, including fluid dynamics.

Scientific computing and simulation workloads

A graphics processing unit can accelerate simulations and data analysis that require complex mathematical calculations. As specialized electronics that can execute calculations simultaneously, GPUs are well-suited for computationally intensive tasks. Compared to traditional CPU cores, GPUs offer raw throughput, making them perfect for intensive workloads like molecular modeling, climate simulation, or large-scale numerical analysis.

Modern GPU technologies combine high memory bandwidth, large video RAM (VRAM), and optimized scheduling to move data efficiently between the GPU chip and system memory. They can maintain optimal performance when working with very large scientific datasets in VRAM, although performance drops significantly when transferring to system memory.

Modern GPUs typically contain a number of multiprocessors. Each has a shared memory block, plus several processors and corresponding registers. The GPU itself has device memory on the board it is housed on, known as VRAM.

Each GPU works slightly differently depending on its purpose, the manufacturer, the specifics of the chip, and the software used for coordinating the GPU. For instance, Nvidia’s CUDA parallel processing software allows developers to specifically program the GPU with almost any general-purpose parallel processing application in mind.

GPUs can be standalone chips, known as discrete GPUs, or integrated with other computing hardware, known as integrated GPUs (iGPUs).

Discrete GPUs

Discrete GPUs exist as a chip that is fully dedicated to the task at hand. While that task has traditionally been graphics, discrete GPUs are now used as dedicated processing for tasks like ML or complex simulation.

When used in graphics, the GPU typically resides on a graphics card that slots into a motherboard. In other tasks, the GPU may reside on a different card or slot directly onto the motherboard itself.

Integrated GPUs

In the early 2010s, we started to see a move away from discrete GPUs. Manufacturers embraced the introduction of the combined CPU and GPU on a chip, known as the iGPU. The first of these iGPUs for PCs were Intel’s Celeron, Pentium, and Core lines. These remain popular across laptops and PCs.

Another type of iGPU is the system on a chip (SoC) that contains components like a CPU, GPU, memory, and networking. These are the types of chips typically found in smartphones.

Virtual

Like other types of computing hardware infrastructure, GPUs can also be virtualized. Virtualized GPUs are a software-based representation of a GPU that shares space alongside other virtual GPUs on cloud server instances. You can use them to run your workloads without having to worry about underlying hardware maintenance.

What is a cloud GPU?

A cloud GPU is a graphics processing unit hosted within the cloud and accessed via the internet. From a hardware standpoint, cloud GPUs are the same class of graphics processors used in on-premises systems, like those on a dedicated graphics card or video cards for enterprise use. However, these GPUs are inside servers and connected through high-speed interconnects, which serve a similar role to PCIe slots in traditional machines. You connect to the GPU through virtual machines or containers, not actually interacting with the physical hardware.

Cloud GPUs are often used for machine learning, deep learning, graphics rendering, or other workloads involving either compute graphics or large-scale data processing. GPUs excel at parallel operations, making them perfect for computationally intensive tasks like training AI models.

While local systems rely on integrated graphics processing units (integrated GPUs), cloud GPUs provide access to discrete GPUs. You can access these GPUs through the internet, offering consistent GPU performance for workloads that need maximum performance.

Being highly scalable, cloud GPUs are frequently used in workloads that are executed remotely and streamed to end users, like cloud gaming. Especially as they become more accessible, more companies are using cloud GPUs to power their parallel processing business use cases like ML applications.

The main difference between a CPU and a GPU is their purpose in a computer system. They have different roles depending on the system. For example, they serve different purposes in a handheld gaming device, a PC, and a supercomputer with several server cabinets.

In general, the CPU handles full system control plus management and general-purpose tasks. Conversely, the GPU handles compute-intensive tasks such as video editing or machine learning.

More specifically, CPUs are optimized for performing tasks such as:

• System management
• Multitasking across different applications
• Input and output operations
• Network functions
• Control of peripheral devices
• Memory and storage system multitasking

‘Graphics processing unit’ and ‘graphics card’ are often used interchangeably but these terms do not describe the same thing. Graphics cards are an add-in board (AIB) that slots into a place on a computer’s motherboard. Graphics cards are not built into the computer itself; they are interchangeable cards. A graphics card comes with a GPU on the board.

The GPU is the main component on graphics cards, alongside other components like Video RAM (VRAM) for video memory, ports (such as HDMI or DisplayPort), and a cooling component. However, a GPU can also be built directly into the motherboard or integrated as an all-in-one chip together with other components, such as an iGPU or SoC unit.

Amazon EC2 is the broadest and deepest compute platform, where you can find dedicated cloud GPU offerings. It offers more than 1,000 instances and your choice of the latest processor, storage, networking, operating system, and purchase model to help you best match the needs of your workload. With EC2, you can easily provision and run graphics processing units in the cloud. Take advantage of GPUs for processing power for video editing, rendering graphics, artificial intelligence, and other parallel processing capabilities.

For example:

• P5 instances, powered by NVIDIA H100 Tensor Core GPUs, and P5e and P5en instances powered by NVIDIA H200 Tensor Core GPUs deliver high performance in Amazon EC2 for deep learning (DL) and high performance computing (HPC) applications.
• P6e UltraServers, accelerated by NVIDIA GB200 NVL72, offer the highest GPU performance for AI training and inference.
• G7e instances, accelerated by NVIDIA RTX PRO™ 6000 Blackwell Server Edition GPUs, deliver cost-effective performance for generative AI inference workloads and the highest performance for spatial computing workloads.

Get started with GPUs on AWS by creating a free account today.