MATH Benchmark: 12,500 Problems, 99%+ Saturation, What Replaced It

The benchmark that drove the chain-of-thought era, now functionally retired at the top.

"Mathematics benchmark" is not one test. The five that matter in 2026 span grade-school word problems through unsolved research mathematics, and most of the older ones are saturated. The table below maps each to what it measures and where it is still useful; the rest of this page is the deep dive on MATH itself, the benchmark that defined the category.

MATH was published in March 2021 by Dan Hendrycks and collaborators at UC Berkeley. It draws 12,500 problems from American math competitions (AMC 10, AMC 12, AIME, USAMO qualifiers, and similar). Problems are split 7,500 train / 5,000 test, tagged by subject and by difficulty level 1 through 5. Each ships with a full LaTeX solution.

The grading rule is exact match on the final boxed answer after the model emits its full chain-of-thought. There is no partial credit, no judge model, no human in the loop at scoring time. This makes MATH cheap to run and reproducible, two reasons it became the de facto math benchmark for three years.

Three forces compounded. First, chain-of-thought prompting plus self-consistency (Wang et al. 2022) lifted GPT-3 class models from single digits to the 50s. Second, math-specific fine-tunes (Minerva, WizardMath, DeepSeekMath) extracted another 15 to 20 points. Third, test-time compute scaling (o1, o3, Sonnet thinking modes) pushed reasoning chains long enough that the remaining error budget on MATH-500 collapsed to single percentage points.

By mid-2025 the headline number stopped moving. When every frontier model scores between 98.6% and 99.4%, the benchmark is no longer measuring capability differences; it is measuring noise plus residual contamination.

AIME 2024 and AIME 2025 are the new defaults for top-tier math comparison. Each year has 15 problems with integer answers between 0 and 999. The problems are harder than the average MATH item, so pass rates remain spread across the frontier rather than pinned at the ceiling, which is what makes them discriminating where MATH no longer is. We do not print a per-model AIME score table here; with only 15 problems per year the scores are high-variance and the cross-vendor cells cannot be responsibly sourced for the current frontier. For Olympiad-level math, Putnam-Bench and the HARP project both retain headroom.

FrontierMath (Epoch AI, late 2024) was designed explicitly to resist saturation: research-grade problems written and peer-reviewed by working mathematicians. Epoch shipped a v2 error-correction on 12 June 2026 that revised or removed problems flagged across roughly 42% of the original set, leaving 338 problems split into a 295-problem base (Tiers 1 to 3) and a 43-problem Tier 4 expansion of exceptionally hard items. It is no longer the near-zero benchmark of 2024 to 2025: frontier 2026 reasoning models now clear well over half of Tiers 1 to 3, so the discriminating headroom has migrated to Tier 4. It remains the hardest mathematics benchmark in active use; Epoch does not freeze a single SOTA figure, so read its live leaderboard rather than a reprinted number.