LLM Evaluation & Leaderboards
News on LLM evaluation and benchmarks, covering new tests, methodology changes, leaderboard results, and whether findings can be reproduced.
Explore how researchers and engineering teams test language models before and after deployment. Coverage looks at benchmark design, meaningful comparisons, leaderboard changes, and which results teams can reproduce.
Latest coverage
Articles
NewsAllen Institute for AI releases StateBench for long-document state tracking
NewsAi2 releases XYBench to test how assistants handle mistaken premises
NewsApple study finds MMLU weakly tracks LLM judge quality
NewsMBZUAI introduces MedRoundsQA benchmark for diagnostic conversations
NewsVals finds DeepSeek trails GPT-6 Sol Daybreak by 8 points in simulated cyber test
NewsOpenAI introduces LifeVal benchmark for everyday AI tasks
NewsProperty Finder rebuilds AI benchmark, reports 86% lower inference cost
NewsIBM and Kaggle launch maintained ScarfBench Java migration leaderboard
NewsiMerit details Scholars workflows for specialized AI model evaluation
NewsMBZUAI details CT-SpatialVQA; eight tested models score below 50%
NewsMIT benchmark says GPT-5.5 matched 34.6% of recent theory problems
NewsApple study finds tool-discovery gains shift with task complexity