A GPU (graphics processing unit) is a chip designed to perform thousands of simple calculations at the same time instead of one complex calculation after another. Modern AI models are, underneath, enormous grids of numbers being multiplied and added together — the exact kind of repetitive, parallel math a GPU was built to handle. That overlap, discovered almost by accident, is why a chip designed to draw video-game graphics became the engine of the AI boom.

What a GPU actually is

A GPU packs thousands of small processing cores onto one chip — Nvidia calls its version CUDA cores, AMD calls its stream processors. Each core is simple: it can’t do much on its own. But a modern data-center GPU can run tens of thousands of these cores at once, all working on different pieces of the same problem simultaneously. That’s called parallel processing.

A regular computer’s main chip, the CPU, takes the opposite approach: a handful of powerful cores (typically 4 to 64) built to race through one task after another as fast as possible, with lots of memory dedicated to keeping each core fed. A CPU is a small team of experts who each handle one problem at a time, brilliantly. A GPU is a stadium full of workers who each do one small step of the same giant problem, all at once. For tasks that break into many identical small pieces, the stadium wins by orders of magnitude.

How a graphics chip ended up running AI

GPUs exist because 3D graphics are full of that kind of repetitive math: a game scene might contain millions of pixels, and each one needs roughly the same calculation to figure out its color and position. Nvidia shipped the first chip it marketed as a “GPU,” the GeForce 256, in 1999, built purely to render graphics faster.

The turning point came in 2006, when Nvidia released CUDA, a programming platform that let developers use a GPU’s cores for math that had nothing to do with graphics. Researchers building neural networks noticed that training one — adjusting millions of internal numbers, called parameters, through repeated rounds of matrix multiplication — was exactly the kind of parallel arithmetic CUDA made possible. In 2012, a neural network called AlexNet, trained on two Nvidia gaming GPUs, dramatically outperformed every other image-recognition system at a major competition. That result convinced the AI research community that GPUs, not CPUs, were the hardware deep learning needed. Every major leap since — including the transformer models behind today’s chatbots — has been trained on GPUs.

Why it matters

Training a large AI model means running that matrix math over and over, across enormous datasets, for weeks or months. Doing it on CPUs would take years and cost far more, if it finished at all. GPUs cut that time down to something companies can actually work with — which is also why a single top-tier data-center GPU can cost tens of thousands of dollars, and why AI labs buy them by the tens of thousands.

That demand reshaped the semiconductor industry. It made Nvidia, a company that started out selling graphics cards to gamers, one of the world’s most valuable companies, and it’s the direct cause of the AI compute shortage that has data centers, power grids, and national governments racing to secure more chips. Countries and companies now measure AI capacity in GPUs the way they once measured industrial capacity in factories or steel.

In the news

That race is now playing out at a national scale. Japan’s government and a 44-company industry consortium recently partnered with Nvidia to build a 140-megawatt “AI factory” powered by 27,500 of Nvidia’s next-generation Rubin GPUs, intended to run open AI models for robotics and manufacturing — a sign of how central raw GPU capacity has become to industrial policy, not just tech company budgets.

FAQ

Is a GPU the same as an AI chip? Not exactly. A GPU is the most common type of chip used for AI, but some companies also design chips built only for AI math (often called AI accelerators or inference chips), without the graphics-rendering features a GPU still carries.

Do I need a GPU to use AI tools? No. Using a chatbot or image generator sends your request to the company’s servers, where their GPUs do the work; your own device just displays the result. A GPU matters mainly if you’re training models or running large ones yourself.

Why can’t CPUs just get more cores to compete? They could add more cores, but CPU cores are built to be individually powerful and stay that way — cramming in thousands would require redesigning the chip into something closer to a GPU. The two architectures are optimized for different jobs, not different points on the same scale.

Sources: Wikipedia: Graphics processing unit; Nvidia: history of CUDA and GPU computing; IBM: CPU vs. GPU for machine learning.