What Is ARC-AGI — and Why Is It So Hard for AI to Beat?
ARC-AGI is a benchmark of puzzles and games that are easy for humans but hard for AI, built to measure genuine reasoning instead of memorized knowledge.
Read more →Artificial Intelligenceand AI Benchmarks
Every AI News story tagged with both Artificial Intelligence and AI Benchmarks — the two topics side by side, updated as new articles publish.
16 articles
NVIDIA's AVO Agent Aces ARC-AGI-3 With a Perfect Score
NVIDIA's AVO agent architecture, which wraps Claude Opus 5 in persistent memory and self-supervision, cleared every level of the ARC-AGI-3 reasoning benchmark's public set.
Read more →Inherent Labs' Faraday Beats Claude, GPT-5.5 at Research Replication
Inherent Labs, founded by Google DeepMind alumni, says its 27-billion-parameter agent Faraday outperformed Claude Opus 4.8 and GPT-5.5 at replicating published research.
Read more →AI Models Recover Only 3–15% of Research Ideas, Study Finds
A new benchmark testing seven frontier AI models found they could reconstruct a scientific paper's core idea from its pre-publication bibliography alone only 3-15% of the time.
Read more →Anthropic Launches Claude Opus 5, Rivaling Fable 5 at Half Cost
Anthropic released Claude Opus 5 on July 24, pairing near-flagship coding and agent performance with Claude Fable 5's results at roughly half the cost, plus gains in science and finance tasks.
Read more →OpenAI Retracts Endorsement of SWE-Bench Pro Coding Benchmark
An internal audit found that roughly a third of SWE-Bench Pro's coding tasks are flawed, prompting OpenAI to withdraw its recommendation that developers rely on the benchmark.
Read more →Grok 4.5 Launches Cheaper Than Opus, With Mixed Benchmarks
SpaceXAI released Grok 4.5, its first model trained on Cursor data, priced well below Anthropic's Opus 4.8 — though its own self-reported benchmarks show mixed results against it.
Read more →China's GLM-5.2 Rivals US Frontier AI at One-Sixth the Cost
Z.ai has released GLM-5.2, a 744-billion-parameter open-weight model that matches GPT-5.5 on key benchmarks and trails Claude Opus 4.8 narrowly — trained entirely on Huawei silicon with no Nvidia hardware.
Read more →Meta AI Chief Claims In-Training "Watermelon" Model Has Caught Up to GPT-5.5
Meta's superintelligence chief told employees that the company's next model, codenamed Watermelon, has caught up to OpenAI's GPT-5.5 on key benchmarks — though the specific tests and results have not been disclosed.
Read more →Mistral Releases Leanstral 1.5 to Automate Math Proofs and Code Verification
Mistral's new open-source model for Lean 4 fully saturates the miniF2F theorem benchmark, solves 87% of graduate-level math tests, and found five previously unknown bugs in production code — at roughly $4 per proof versus $300 for competing systems.
Read more →What Is an AI Benchmark — and How Do You Know If a Model Is Actually Better?
An AI benchmark is a standardized test used to compare how capable different AI models are. Here's what the major ones measure — and why the scores don't always tell the full story.
Read more →OpenAI Releases GeneBench-Pro to Test AI in Biology Research
OpenAI has released GeneBench-Pro, a 129-problem benchmark that tests whether AI agents can exercise the judgment-intensive analytical reasoning required for real-world computational biology research.
Read more →Z.ai's GLM-5.2 Matches US Frontier AI at One-Sixth the API Cost
GLM-5.2, released on June 16 by Beijing startup Z.ai, sits alongside Claude Opus 4.8 and GPT-5.5 on coding benchmarks while carrying an API price roughly one-sixth that of closed US models.
Read more →ByteDance Doubao Seed 2.1 Claims Parity With GPT-5.5 on Agent Tasks
ByteDance launched Doubao Seed 2.1 Pro and Turbo on June 23, claiming top scores on coding and agent benchmarks and parity with GPT-5.5 and Claude Opus 4.7.
Read more →Zhipu AI's GLM-5.2 Beats GPT-5.5 on Coding at a Sixth of the Price
Chinese lab Zhipu AI has open-sourced GLM-5.2, a 753-billion-parameter model that outperforms GPT-5.5 on software engineering benchmarks while costing roughly one-sixth as much.
Read more →Alibaba Open-Sources AI World Model That Tops GPT-5.4 on Agent Tasks
Alibaba's Qwen team has released Qwen-AgentWorld, a family of open-source language world models that predict how digital environments respond to AI agent actions, with its 397B variant outscoring GPT-5.4 on the new AgentWorldBench benchmark.
Read more →