Knowledge benchmarks test breadth of factual understanding across academic domains. MMLU (2020) was the defining benchmark of the 2020-2023 era and is now saturated; frontier models cluster in the low 90s, making it useless for current comparisons. MMLU-Pro addresses saturation with 10-choice questions and mandatory chain-of-thought, requiring more deliberate reasoning.

Humanity's Last Exam (Scale AI, 2025) is the current frontier challenge: roughly 2,500 questions by domain experts (trimmed from the 3,000 in the original paper), designed to remain hard for several years and to leave substantial headroom above current models. MMMU adds multimodal understanding; questions require image interpretation alongside text. For 2026 comparisons, the recommended stack is MMLU-Pro for text knowledge, MMMU for multimodal knowledge, and HLE for frontier limits.

Code benchmarks evolved rapidly through 2021 to 2026. HumanEval (OpenAI, 2021) with 164 Python function-completion tasks was the standard for years. It is now saturated; top models exceed the high 90s on pass@1. MBPP (Google, 2021) with 974 basic Python problems is similarly exhausted. Neither benchmark belongs in a serious 2026 comparison.

LiveCodeBench (2024) addresses contamination by drawing from LeetCode and AtCoder problems released after each model's training cutoff, making contamination structurally unlikely rather than just hopefully absent. For 2026 code comparisons, use LiveCodeBench and SWE-bench Verified. The gap between SWE-bench Verified scores and HumanEval scores reveals how different code completion is from software engineering.

Reasoning is the benchmark frontier in 2026. GPQA (Rein et al., 2023) tests graduate-level, Google-proof scientific reasoning, with questions written by domain PhDs that are specifically designed to be impossible to answer by searching the web. GPQA-Diamond, the 198-question hardest subset, is the canonical reference.

ARC-AGI-2 (Chollet, 2025) is the successor to the original Abstraction and Reasoning Corpus. ARC-AGI was cracked in late 2024 when reasoning-mode frontier models exceeded the human baseline. ARC-AGI-2 introduces significantly harder visual abstraction tasks that resist the techniques that broke the original, and retains real headroom above current models. BIG-Bench Hard is approaching saturation.

Agentic benchmarks measure multi-step task completion with tool use, a fundamentally different capability from question answering. WebArena (2023) provides 812 tasks across real web applications running in sandboxed browser environments. The agent must navigate, fill forms, search, and complete tasks without human assistance. Agents remain well short of human completion rates, leaving substantial headroom and a real performance gap between frontier models.

AgentBench (2023) covers 8 agent environments including OS tasks, game playing, database management, and web browsing, totalling 1,091 tasks. OSWorld (2024) focuses on computer-use agents that control desktop applications. SWE-bench Verified (listed in Coding above but fundamentally agentic) is the most widely cited.

See agent benchmarks deep dive →

Multimodal benchmarks have grown in importance as frontier models gained vision capability. MMMU (Yue et al., 2024) is the primary knowledge benchmark for multimodal models: college-exam questions that require understanding images alongside text, across 30 subjects. MathVista (2024) tests mathematical reasoning about visual content (geometry diagrams, data charts, scientific plots). ChartQA and DocVQA cover document understanding.

MMMU has become a standard inclusion in frontier model cards in 2026 because it captures a capability dimension that text-only benchmarks entirely miss. The gap between frontier models on it has been narrowing.

LMSYS Chatbot Arena uses blind pairwise human voting to build a Bradley-Terry Elo ranking, drawing on millions of votes that make it one of the most statistically robust human-preference signals available. The top frontier models typically sit in a tight Elo band where many query-category gaps fall within statistical noise; the live ranking is on the LMSYS leaderboard.

Chatbot Arena measures what humans prefer in open-ended conversation, a different dimension from correctness, coding ability, or agentic capability. A model can rank highly in Arena while performing only modestly on SWE-bench, and vice versa. Both are valid measures of different capabilities.

Human vs automated evaluation deep dive →

Benchmarks here are still cited in 2026 but no longer discriminate frontier models. They remain useful as historical context, not as ranking criteria.