ARC-AGI is a benchmark built from puzzles — and, since 2026, small interactive games — that are trivial for humans but stubbornly hard for AI. Created by François Chollet and now maintained by the nonprofit ARC Prize Foundation, it doesn’t test what a model memorized during training. It tests how well a system can work out an unfamiliar problem from just a handful of examples — the same way a curious person would.
What it actually measures
Most AI benchmarks reward “crystallized intelligence”: knowledge a model absorbed from its training data, like facts, code patterns, or common phrasings. ARC-AGI instead targets “fluid intelligence” — the ability to reason about something genuinely new, without relying on prior exposure. Chollet’s 2019 paper defined intelligence itself this way: not what a system knows, but how efficiently it can pick up an unfamiliar skill. That framing is also why the benchmark sits at the center of the industry’s debate over what actually counts as AGI in the first place.
How the puzzles work
The original ARC-AGI-1 (2019) and its harder successor ARC-AGI-2 (2025) are grids of colored squares. Each task shows two to five example pairs — an input grid and the output grid it should turn into — and the solver has to spot the hidden rule and apply it to a new input. No language or cultural knowledge is required, which keeps the test fair across humans everywhere. You can try the puzzles yourself on the ARC Prize site; most people solve at least a few in minutes, while many AI systems still fail outright.
ARC-AGI-3, launched in March 2026, changed the format entirely. Instead of static grids, it drops an AI agent into hundreds of small, hand-built game environments — no instructions, no stated goal, no rulebook. The agent has to explore by trial and error, figure out what “winning” even looks like, and carry what it learned into progressively harder levels, much like a person picking up a new video game with no manual.
Why it’s so hard for AI
Conventional AI benchmarks can be gamed by training on similar examples or by brute-forcing a huge number of guesses. ARC-AGI is deliberately designed to resist that: ARC-AGI-2 removed every task from its evaluation set that earlier systems had solved through brute-force search, and each new version introduces problem types no model has seen before. That’s why jumps in ARC-AGI scores tend to track genuine reasoning progress rather than memorization or dataset leakage — and why the benchmark has repeatedly flagged real inflection points in AI capability, from the arrival of reasoning models to the rise of capable agents.
How the score works
ARC-AGI-3 scores agents with a metric called Relative Human Action Efficiency. Researchers first record how many moves real people need to solve each game the first time they see it, then compare an agent’s move count to that human baseline: a level’s score is roughly (human actions ÷ agent actions)², capped at 1.15. An agent that eventually wins but needs ten times as many moves as a person scores close to zero for that level — finishing isn’t enough, it has to finish efficiently. At launch, human testers averaged 100% while frontier models scored under 1%.
Why it matters
ARC Prize’s founders argue that as long as there’s a wide gap between how efficiently humans and AI systems learn something new, claims of artificial general intelligence are premature — regardless of how fluent a model sounds in conversation. Because each ARC-AGI version is rebuilt to resist shortcuts, a jump in scores is one of the more credible public signals that a lab has made real reasoning progress, not just shipped a bigger or better-tuned model. That’s also why the benchmark now anchors a competition, ARC Prize 2026, offering more than $2 million to teams that narrow the gap.
In the news
That gap narrowed sharply on the public ARC-AGI-3 leaderboard in August 2026, when NVIDIA’s AVO agent reached a perfect score on all 183 levels of the public test set — up from roughly 30% when the underlying Claude Opus 5 model played alone, before being wrapped in AVO’s planning-and-memory architecture. The catch: that score covers only the easier public set. The harder, unseen semi-private and private sets used in the official ARC Prize competition remain untested by AVO, so how close today’s agents really are to human-level efficiency isn’t settled yet.