What Is a TPU — and How Is It Different From a GPU?

A TPU, or Tensor Processing Unit, is a computer chip Google designed from scratch to run one specific kind of math — the matrix multiplications that make neural networks work — instead of the general-purpose computing a CPU handles or the graphics rendering a GPU was originally built for. Google has used TPUs to power its own AI since 2015, and now rents them out through Google Cloud to outside companies, including rivals such as Anthropic, as an alternative to buying GPUs from Nvidia.

How a TPU Works

Every neural network is, underneath, a long chain of matrix multiplications: grids of numbers multiplied and added together, billions of times per second. A general-purpose chip handles this by constantly shuttling numbers back and forth between memory and its processing cores — a bottleneck that wastes both time and power. A TPU is built around a different layout, called a systolic array: thousands of small arithmetic units arranged in a grid, wired so that data flows through them in one continuous pulse, each unit passing its result straight to the next without pausing to write anything back to memory. Combined with running the math at lower numeric precision than a general-purpose chip would use — an approach neural networks tolerate well, since their calculations don’t need to be exact to be useful — that design is what lets a TPU pack far more calculations into every watt of power than a chip built to do everything.

That single-purpose design is also why a TPU counts as an ASIC: a chip physically wired around one job, with no spare circuitry for anything else. Google has shipped seven TPU generations since 2015. The first ran inference only; the second and third added the ability to train models. The newest, Ironwood (TPU v7), carries 192GB of high-bandwidth memory per chip, with up to 9,216 chips linkable into one pod — the first TPU generation Google built specifically to run trained models rather than train them.

TPU vs. GPU: The Practical Difference

A GPU is programmable and sold by multiple vendors, led by Nvidia — the same chip a gamer buys can be repointed to train a language model overnight, because software tells it what to compute. A TPU has no such flexibility: its circuitry is fixed at the design stage, it’s made only by Google, and — unlike a GPU, which anyone can buy outright — it’s available only by renting time on it through Google Cloud, or indirectly, by using a Google product built on top of it, such as Search or Gemini. What a TPU gives up in flexibility, it’s designed to make up in cost and power efficiency for the one thing it does: running the math behind today’s AI models.

Can You Use a TPU Yourself?

Renting a full TPU pod is enterprise-scale infrastructure, but individual developers can try one for free through Google Colab, a browser-based coding notebook — switching the runtime’s hardware accelerator to “TPU” provisions one for the session, no setup or payment required. For production workloads, Google Cloud rents TPU capacity by the chip-hour: as of July 2026, an on-demand TPU v5e chip lists at roughly $1.20 per hour in Google’s US regions, per Google Cloud’s published pricing, with lower rates for multi-year commitments.

Why It Matters

For Google, TPUs mean it isn’t buying every chip it needs from Nvidia at Nvidia’s prices, in a market where GPU demand has outstripped supply for years. Owning the stack — from chip design through the models that run on it — gives Google a cost advantage few competitors can match. That advantage is now a business of its own: in October 2025, Google agreed to give Anthropic access to up to one million TPU chips and more than a gigawatt of computing capacity, a deal reportedly worth tens of billions of dollars — a sign TPUs have become a product Google sells to rivals, not just infrastructure it keeps for itself. It’s also one piece of a broader shift in the AI chip industry: as hardware costs mount, more AI labs are designing custom silicon instead of relying solely on Nvidia.

In the News

That shift extends well beyond Google: OpenAI recently unveiled its own custom AI inference chip, built with Broadcom and code-named Jalapeño — following the same logic that led Google to build TPUs a decade earlier.