The Natural Abstraction Hypothesis: Why AI and Humans May Converge on Shared Ontologies

In a recent exploration of the Natural Abstraction Hypothesis, a post on lessw-blog examines how intelligent agents model concepts probabilistically as latent variables. For the field of AI interpretability, this framework suggests that advanced models will naturally converge on human-understandable abstractions due to shared environmental pressures, providing a potential pathway for steering complex systems.

Bridging Descriptive and Philosophical Frameworks

The study of concept representation spans cognitive science, neuroscience, and machine learning. However, these fields often talk past one another, utilizing frameworks that are either strictly descriptive-focusing on the biological firing of neurons or the specific weight updates in a neural network-or highly philosophical. The lessw-blog analysis highlights a middle ground: modeling the minds of agents through a probabilistic and Bayesian lens.

Crucially, this approach does not argue that agents natively execute Bayesian code. A biological organism or a deep learning model does not explicitly update joint probability tables or maintain a pristine, Pearl-style causal directed acyclic graph (DAG). Instead, their embedded structures approximate Bayesian reasoning due to evolutionary or selection pressures. Even simple organisms like E. coli can be analyzed through an "intentional stance," where their behavior is modeled as having goals and a world model optimized by the environments their ancestors navigated. In advanced AI systems, the underlying architecture may be a transformer or a diffusion model, but the resulting behavior remains isomorphic to Bayesian updating because reality forces effective agents to build compact, causal world models.

The Mechanics of Concepts as Latent Variables

Within this probabilistic framework, concepts function as latent variables. An agent interacts with the world through noisy, high-dimensional sensory inputs-observables like raw pixels, audio frequencies, or tactile feedback. To make predictions, the agent must filter and aggregate this data into a functional world model.

The source illustrates this through the cognitive development of a toddler learning the concept of a "ball." While a ball appears to be a direct observable, it is actually a highly compressed latent variable. The toddler does not track the individual atoms of the object or process the raw photon stream continuously. Instead, the brain infers a generalized concept-a "ball" that is spherical, moves as a single unit, and bounces-based on prior sensory associations. This intuitive latent compresses the overwhelming complexity of physical reality into a manageable, predictive representation.

Not all statistical latents are intuitive. In standard statistics, a latent might simply be a mathematical artifact of data compression, akin to the unreadable structure of a gzipped file. However, "intuitive latents" are those that possess high explanatory power and are easily grasped by human cognition, largely because human brains share similar hardware and environmental training regimens.

Ontological Convergence and AI Interpretability

For PSEEDR, the most significant implication of this framework lies in AI alignment and interpretability. A persistent fear in AI safety is that artificial neural networks will develop entirely alien internal representations, making their decision-making processes opaque and unsteerable. The Natural Abstraction Hypothesis counters this by positing that "you don't get to choose the ontology."

Because the physical environment dictates the structure of reality, any highly optimized agent attempting to predict that environment will be forced to recognize the same underlying structures. This phenomenon, known as ontological convergence, suggests that humans, hypothetical alien intelligences, and advanced AI models will naturally converge on similar, interoperable semantics. If AI systems naturally form "natural latents" that map cleanly to human concepts, researchers can theoretically locate these shared latent spaces within the model's weights. This provides a theoretical foundation for mechanistic interpretability: instead of translating an alien language, alignment researchers would be mapping a shared ontology dictated by the physics of the environment.

Missing Context and Empirical Limitations

While the theoretical grounding is compelling, the framework currently exhibits notable limitations and missing context. The source introduces the concept of "natural latents," defining them mathematically as both "mediators" and "redunds" over the systems they compress. However, the precise mathematical definitions and formulas for these terms are omitted, leaving the exact mechanics of how a natural latent is formally verified ambiguous.

Furthermore, the hypothesis relies heavily on theoretical abstraction rather than empirical validation. While it is logically sound that selection pressures drive agents toward shared ontologies, empirical proof of ontological convergence in modern, frontier-scale deep learning models remains sparse. It is still an open question whether large language models (LLMs) actually form these exact natural latents, or if their vast parameter counts allow them to find non-intuitive, high-dimensional shortcuts that bypass human-legible concepts entirely. Proving that an AI's internal representation of a "ball" or "deception" mathematically aligns with a human's intuitive latent requires rigorous testing that the current literature has yet to fully provide.

Framing concepts probabilistically bridges the gap between low-level mechanistic descriptions and high-level theories of mind, offering a pragmatic lens for analyzing both biological and artificial intelligence. If the Natural Abstraction Hypothesis holds true, the alignment of superintelligent systems may rely less on forcing human values into alien architectures, and more on translating a shared, environmentally determined ontology that both humans and machines naturally compute. Understanding the mathematical boundaries of these natural latents will be critical in determining whether the future of AI remains legible to its creators.

Key Takeaways

• Agents do not need to execute explicit Bayesian code to be modeled as Bayesian; evolutionary and selection pressures force their embedded structures to approximate probabilistic reasoning.
• Concepts function as latent variables that compress high-dimensional sensory data into manageable, predictive representations of reality.
• Ontological convergence suggests that diverse intelligent agents, including advanced AI, will naturally develop similar conceptual ontologies dictated by shared environmental structures.
• If AI models naturally form human-legible 'natural latents,' researchers may be able to map shared latent spaces to improve mechanistic interpretability and alignment.