OpenAI Retracts Endorsement of SWE-Bench Pro Coding Benchmark
An internal audit found that roughly a third of SWE-Bench Pro's coding tasks are flawed, prompting OpenAI to withdraw its recommendation that developers rely on the benchmark.
Read more →AI Benchmarksand AI Research
Every AI News story tagged with both AI Benchmarks and AI Research — the two topics side by side, updated as new articles publish.
10 articles
China's GLM-5.2 Rivals US Frontier AI at One-Sixth the Cost
Z.ai has released GLM-5.2, a 744-billion-parameter open-weight model that matches GPT-5.5 on key benchmarks and trails Claude Opus 4.8 narrowly — trained entirely on Huawei silicon with no Nvidia hardware.
Read more →Meta AI Chief Claims In-Training "Watermelon" Model Has Caught Up to GPT-5.5
Meta's superintelligence chief told employees that the company's next model, codenamed Watermelon, has caught up to OpenAI's GPT-5.5 on key benchmarks — though the specific tests and results have not been disclosed.
Read more →Mistral Releases Leanstral 1.5 to Automate Math Proofs and Code Verification
Mistral's new open-source model for Lean 4 fully saturates the miniF2F theorem benchmark, solves 87% of graduate-level math tests, and found five previously unknown bugs in production code — at roughly $4 per proof versus $300 for competing systems.
Read more →What Is an AI Benchmark — and How Do You Know If a Model Is Actually Better?
An AI benchmark is a standardized test used to compare how capable different AI models are. Here's what the major ones measure — and why the scores don't always tell the full story.
Read more →OpenAI Releases GeneBench-Pro to Test AI in Biology Research
OpenAI has released GeneBench-Pro, a 129-problem benchmark that tests whether AI agents can exercise the judgment-intensive analytical reasoning required for real-world computational biology research.
Read more →Z.ai's GLM-5.2 Matches US Frontier AI at One-Sixth the API Cost
GLM-5.2, released on June 16 by Beijing startup Z.ai, sits alongside Claude Opus 4.8 and GPT-5.5 on coding benchmarks while carrying an API price roughly one-sixth that of closed US models.
Read more →ByteDance Doubao Seed 2.1 Claims Parity With GPT-5.5 on Agent Tasks
ByteDance launched Doubao Seed 2.1 Pro and Turbo on June 23, claiming top scores on coding and agent benchmarks and parity with GPT-5.5 and Claude Opus 4.7.
Read more →Zhipu AI's GLM-5.2 Beats GPT-5.5 on Coding at a Sixth of the Price
Chinese lab Zhipu AI has open-sourced GLM-5.2, a 753-billion-parameter model that outperforms GPT-5.5 on software engineering benchmarks while costing roughly one-sixth as much.
Read more →Alibaba Open-Sources AI World Model That Tops GPT-5.4 on Agent Tasks
Alibaba's Qwen team has released Qwen-AgentWorld, a family of open-source language world models that predict how digital environments respond to AI agent actions, with its 397B variant outscoring GPT-5.4 on the new AgentWorldBench benchmark.
Read more →