The "Funhouse Mirror" of AI Benchmarks: Why Progress Charts May Be Misleading
Coverage of lessw-blog
A recent analysis challenges the mathematical validity of plotting AI benchmark scores over time, arguing that without natural units, our progress curves are merely illusions.
In a recent post, lessw-blog (via LessWrong) discusses a fundamental issue in how the artificial intelligence community measures and visualizes progress. As the industry races toward Artificial General Intelligence (AGI), stakeholders ranging from engineers to policymakers rely heavily on charts showing performance metrics-such as MMLU or HumanEval scores-climbing over time. These visualizations often depict exponential growth or S-curves, influencing decisions on resource allocation and safety timelines. However, this new analysis suggests that these charts may be mathematically unsound.
The Context: The Obsession with Curves
The current AI landscape is defined by scaling laws and performance trajectories. Researchers and investors frequently look for "inflection points" or calculate the "velocity" of AI development by plotting benchmark scores against time. The assumption is that a move from 40% to 50% on a test represents a quantifiable step in intelligence that is comparable to a move from 50% to 60%. This post challenges that assumption by highlighting a critical deficiency in AI metrology: the lack of "natural units."
The Core Argument: A Lack of Natural Units
The author argues that unlike physical sciences, where measurements like meters or seconds have constant, defined values (natural units), AI benchmarks are essentially "grab-bags of tasks." A benchmark score is typically an aggregate fraction of completed tasks. Because these tasks vary wildly in difficulty and are not standardized units of "intelligence," the Y-axis on a progress chart is arbitrary.
The post uses the metaphor of a "funhouse-mirror projection." If true AI capability is a multidimensional object moving through space, benchmarks are distorted mirrors reflecting that movement. Depending on how a test is weighted (e.g., many easy questions vs. a few hard ones), the reflection might show rapid acceleration followed by a plateau, or vice versa, even if the underlying progress is constant. Therefore, analyzing the shape of these curves-looking for derivatives, speed-ups, or trends-is a category error. The shape describes the test design, not necessarily the model's evolution.
Why This Matters
If the Y-axis is arbitrary, then mathematical extrapolations regarding when AI will reach human-level performance are highly suspect. A curve that looks exponential could simply be an artifact of a benchmark that runs out of easy questions quickly. This analysis serves as a crucial reminder that while benchmarks are useful for ranking models (Model A is better than Model B), they are treacherous tools for mapping the trajectory of the technology itself.
We recommend this post to data scientists, ML evaluators, and forecasters who rely on performance metrics to model future capabilities. It prompts a necessary skepticism regarding the visualizations that currently drive the AI narrative.
Read the full post on LessWrong
Key Takeaways
- Most AI benchmarks lack "natural units," making them fundamentally different from physical measurements like distance or time.
- Plotting benchmark scores over time to find "inflection points" or "speed-ups" is mathematically invalid because the Y-axis is arbitrary.
- Benchmarks act as "funhouse mirrors," distorting the true shape of AI progress based on how tasks are weighted and distributed.
- Aggregate scores (like percentage completed) are useful for ranking models against each other but not for calculating the rate of progress.
- Misinterpreting these progress curves can lead to flawed predictions about AGI timelines and incorrect resource allocation.