GPU (Graphics Processing Unit)
Also known as: graphics processing unit, GPU computing, AI accelerator
What is a GPU in the Context of AI?
A GPU (Graphics Processing Unit) is a processor originally designed for rendering graphics that has become the primary hardware for training and running AI models. Unlike CPUs, which excel at sequential tasks with a few powerful cores, GPUs contain thousands of smaller cores optimized for parallel computation. Since neural network training involves performing the same mathematical operations (matrix multiplications, vector additions) across massive datasets simultaneously, GPUs provide orders-of-magnitude speedups over CPUs for AI workloads.
GPUs and the AI Revolution
The deep learning revolution of the 2010s was enabled as much by GPU hardware as by algorithmic advances. NVIDIA’s CUDA programming framework, released in 2007, allowed researchers to run general-purpose computations on GPUs, and the company has since dominated the AI accelerator market. The H100 and subsequent Blackwell-architecture GPUs are purpose-built for AI training and inference, featuring specialized tensor cores, high-bandwidth memory, and fast interconnects for multi-GPU clusters. A single frontier model training run may use tens of thousands of GPUs operating in concert for months, costing hundreds of millions of dollars in compute.
Why GPUs Matter for Practitioners
GPU availability is a practical constraint that shapes AI strategy. Organizations must decide between cloud GPU rental (AWS, GCP, Azure), on-premises clusters, or managed inference services. GPU scarcity has driven geopolitical competition, with export controls on advanced chips becoming a tool of national policy. For AI practitioners, understanding GPU capabilities informs decisions about model size, training budgets, inference latency, and whether to fine-tune models or use them via API. The cost and availability of GPUs remains one of the most tangible bottlenecks in AI development.
Related Reading
- TPU - Google’s custom AI accelerator alternative
- AI Infrastructure - The broader compute stack for AI