What Is Test-Time Compute — and Why Does It Matter?

Test-time compute is the processing power an AI model uses while it is answering a question, rather than while it was being trained. Instead of producing a reply in one fast pass, a model spends extra computation “thinking” — working through intermediate steps, checking its own logic, or generating several candidate answers and comparing them — before committing to a final response. The idea underlies today’s AI reasoning models, and it has become one of the main levers AI labs pull to make answers more accurate, alongside, and increasingly instead of, simply building bigger models.

From bigger models to longer thinking

For years, the dominant way to improve an AI system was straightforward: train a bigger large language model on more data. Researchers described this relationship as a scaling law — add more parameters and more training compute, and capability improves in a fairly predictable way. That approach still works, but each new gain requires a longer, more expensive training run, and returns have been slowing.

Test-time compute, also called inference-time scaling, offers a second lever. Rather than making the model itself bigger, developers let it use more computation at the moment it answers one specific question. OpenAI popularized the idea when it introduced its o1 model in 2024, reporting that o1’s accuracy on hard problems improved consistently the longer it was allowed to “think” before answering — in addition to improving with more training. That mattered because it gave AI labs a second dial to turn, one that scales with the difficulty of each question rather than with the size of the whole model.

How a model spends the extra time

The core mechanism is chain-of-thought reasoning: instead of jumping straight to an answer, the model generates a hidden sequence of intermediate steps, breaking a hard problem into smaller ones, checking its own work, and sometimes backtracking before it answers. This is the basis of reasoning models such as OpenAI’s o-series, Google’s “Thinking” versions of Gemini, and Anthropic’s Claude, which calls the feature “extended thinking” — developers can set a token budget for how much a model may reason through its API before responding.

A second family of techniques scales compute by generating multiple attempts instead of one longer one. The model produces several independent answers to the same question and either picks the most common one, a method called self-consistency, or uses a separate verifier model to score and select the best candidate. DeepSeek’s R1 model demonstrated the effect starkly: on a set of competition-level math problems, a single answer from the model was correct 71% of the time, but taking a majority vote across many sampled answers pushed accuracy to 86.7%, with no change to the underlying model.

Why it matters

The shift toward test-time compute is reshaping AI economics, not only AI research. If accuracy gains increasingly come from spending more computation per answer rather than from training a bigger, costlier model, the bottleneck moves from training to inference — the repeated work of running a trained model every time a user asks a hard question. That helps explain why chipmakers are racing to build processors optimized specifically for running models rather than training them: fast, cheap inference is now tied directly to how good an answer a user gets, not just to how many users a company can serve at once.

It also changes what “a smarter answer” costs. A reasoning model that thinks for thirty seconds before replying to a hard question consumes far more computation, per answer, than a model that responds instantly, because every reasoning step, seen or hidden, is computation a provider typically bills for. That is why reasoning modes are often priced or rate-limited separately from standard chat, and why deciding whether to let a model “think” has become a real cost-versus-accuracy tradeoff for anyone building on top of AI, not only a technical detail.

In the news

Chipmaker Etched is reportedly in talks for a $20 billion valuation, roughly four times its price last year, as investors bet that demand for hardware built specifically for AI inference — the kind test-time compute depends on — will keep growing as more model providers lean on longer thinking rather than ever-bigger training runs.