A recent analysis on LessWrong challenges the prevailing existential risk calculus, arguing that improving the quality of a potential future may yield higher expected value than marginally increasing survival probabilities.
\nIn a recent post, a LessWrong contributor discusses a critical strategic divergence in the field of existential risk (x-risk) mitigation: the trade-off between ensuring survival and ensuring a future worth living. The post, titled Focusing on Flourishing Even When Survival is Unlikely (I), investigates whether the community's intense focus on reducing extinction probabilities might be overshadowing high-leverage opportunities to improve the quality of the future.
The context for this discussion is the "astronomical waste" argument often cited in Effective Altruism and AI safety circles. This argument posits that because the potential number of future lives is vast, the expected value of the future is enormous. Consequently, even a tiny reduction in the probability of extinction (which reduces that value to zero) is typically viewed as the highest priority. This logic has historically driven a prioritization of "survival-focused" interventions-such as preventing nuclear war or solving technical AI alignment-over "flourishing-focused" interventions that aim to improve societal structures or human well-being.
The author challenges this heuristic by referencing Will MacAskill's "Better Futures" argument. The core contention is that the value of the future is not a fixed constant; it is a variable highly dependent on the trajectory humanity takes. The post argues that interventions focused on flourishing can significantly increase the expected value of the future conditional on survival. Crucially, the author extends this logic to scenarios where survival is unlikely. Even if humanity faces a 99% chance of extinction, an intervention that increases the value of the surviving 1% scenario by a factor of 100x creates more expected value than a minor increase in survival probability.
This perspective is significant for strategic forecasting. It suggests that even in dire circumstances-such as the rapid onset of potentially dangerous AGI-it may remain rational to allocate resources toward shaping how AGI integrates with society (flourishing), rather than exclusively focusing on binary safety measures. If the "safety" measures only secure a mediocre or dystopian future, the expected value calculation shifts dramatically.
The post serves as a reminder that "making it to the future" is only one variable in the equation. Ensuring that the future is structurally sound and conducive to human flourishing requires distinct interventions that may be currently undervalued.
For those involved in long-term strategy and risk assessment, this post offers a mathematical framework for rethinking resource allocation.
\n\n