A LessWrong post argues that treating moral governance as performative rather than infrastructural could lead to catastrophic outcomes in the Singularity.
\nIn a recent post on LessWrong, the author explores the emotional and structural weight of the "Singularity Game," presenting a counter-narrative to the idea that the arrival of Artificial General Intelligence (AGI) is merely an intellectual milestone. The piece, titled "Non-Scheming Saints (Whether Human Or Digital) Might Be Shirking Their Governance Duties," serves as a somber response to previous community discussions regarding the potential loss of intellectual puzzles in a post-AGI world.
The discourse surrounding AGI often oscillates between technical alignment problems and the philosophical implications of a post-human era. A recurring tension exists between "winning" the race to AGI and the actual quality of the future that victory secures. As the field grapples with the concept of the Singularity, the focus is shifting from merely building capable systems to ensuring that the moral and institutional frameworks-the "governance"-are robust enough to handle them. This post touches on the deep anxiety that humanity is constructing immense power without the corresponding "saintly" infrastructure to wield it responsibly.
The author articulates a sense of grief regarding the current trajectory of the Singularity. The central critique targets a specific failure mode observed in modern democracies: the tendency to treat moral sentiments and institution-building as "consumption goods" or performative "peacock feathers" rather than essential infrastructure. In this view, morality is often treated as a signal of virtue rather than a functional requirement for civilization's survival.
The post suggests that "non-scheming saints"-those who are morally inclined but avoid the "dirty work" of governance and power dynamics-are effectively shirking their duties. The argument posits that if moral actors (whether human today or digital entities tomorrow) refuse to engage in the structural engineering of civilization because they view it as beneath them, too Machiavellian, or simply "scheming," the resulting vacuum leads to tragedy. The author argues that governance is not a game or a luxury; it is the bedrock required for a functional future. If the "saints" abdicate the throne of governance, the Singularity may result in an "objective tragedy" where high-capability systems lack the moral architecture to sustain a flourishing existence.
This perspective challenges the reader to reconsider the role of power and administration in ethical systems. It suggests that true morality in the face of existential risk requires more than good intentions; it requires the willingness to build and maintain the heavy machinery of governance.
For those tracking the philosophical underpinnings of AI safety and the sociology of the alignment community, this post offers a critical look at how we value-and perhaps undervalue-the work of institutional maintenance.
Read the full post on LessWrong
\n\n