In a recent post, lessw-blog explores the computational limits of the human brain compared to artificial neural networks, suggesting that AI architectures possess a fundamental depth and speed advantage over biological intelligence.
\nThe Hook
In a recent post, lessw-blog discusses a fascinating theoretical framework termed the Biological Computing Underhang. This piece provides a rigorous comparison between the computational capabilities of the human cortex and modern artificial neural networks (ANNs), ultimately suggesting that artificial systems may possess inherent structural advantages over biological intelligence.
The Context
The intersection of neuroscience and artificial intelligence has long been a fertile ground for theoretical breakthroughs. Historically, AI development has drawn heavy inspiration from biological brains, from the basic structure of perceptrons to the complex architectures of deep learning. However, as large language models (LLMs) and foundation models scale to unprecedented sizes, a critical question emerges: are we simply replicating the brain, or are we building systems that bypass its fundamental physical limitations? Understanding the constraints of biological cognition-such as processing speed, layer depth, and synaptic plasticity-is essential for evaluating the long-term trajectory of machine learning. If biological brains operate under hard physical constraints that silicon-based systems can easily exceed, it fundamentally shifts how researchers should benchmark artificial intelligence and design future architectures.
The Gist
lessw-blog's analysis centers on the argument that the human cortex is inherently depth-limited. Specifically, its ability to represent complex abstractions in a single forward pass is constrained by biological realities, establishing a hard ceiling on the types of reasoning that are biologically possible. The author estimates that a single cortical area can be simulated by approximately 14 ReLU transforms, operating at roughly 4 milliseconds per pass.
To put this into perspective, the post contrasts these biological metrics with modern AI architectures. GPT-3, for instance, features 192 layers and can execute a forward pass in less than 2 milliseconds. This represents a massive speed and depth advantage for artificial systems. The author posits that with sufficient training data, a model matching GPT-3's depth could theoretically be trained to replicate every cortical microcircuit. Consequently, even if a human brain were given arbitrary developmental time, it should be computationally dominated by larger ANNs, assuming identical inter-region scaffolding and reward signals.
Furthermore, the publication highlights unique biological limitations that do not affect artificial models. For example, certain cognitive abilities, such as acquiring perfect pitch, are restricted to critical learning periods in humans. This is due to the physical encagement of synapses-often related to structures like perineuronal nets-during brain maturation. ANNs, free from such biological aging processes, do not suffer from these permanent physical lock-ins.
Conclusion
This analysis is highly significant for the AI and machine learning communities. It implies that a biological underhang exists, where human cognitive limits prevent certain types of learning that machines might easily master. By recognizing these biological limitations, AI researchers can push for novel architectures that exploit these theoretical advantages rather than merely mimicking human neural pathways. To explore the intricate details of corticothalamic cycles, microcircuit estimations, and the broader implications for AGI, Read the full post.
\n\n