In a recent post, lessw-blog examines a critical challenge to longtermist philosophy: the computational tractability of calculating the expected value of the far future.
\nIn a recent post, lessw-blog discusses a compelling critique of longtermist Effective Altruism (EA), focusing on the work of David Kinney (2022). The analysis questions whether the mathematical foundations of longtermism-specifically the calculation of expected utility for far-future events-are computationally feasible for human decision-makers.
Longtermism typically asserts that because the future could be vast and populated by trillions of humans, small reductions in existential risk today carry immense expected value. Prominent figures in the movement advocate for prioritizing interventions based on these calculations. However, this framework relies heavily on the assumption that agents can reasonably estimate the probability and utility of outcomes centuries or millennia away. The core tension lies in whether this \"arithmetic\" is actually a valid tool for guiding action, or if the complexity of the causal variables renders such calculations impossible.
The post details Kinney's argument that longtermism fails to be \"action-guiding\" due to computational intractability. To determine which intervention maximizes expected value, an agent must perform Bayesian updating on a causal model of the world. Kinney points out that exact probabilistic inference in such networks is an NP-hard problem. This means the computational resources required grow exponentially with the complexity of the variables, quickly exceeding human or machine capacity.
Furthermore, the analysis suggests that even if we overcame the inference hurdle-magically acquiring perfect probability distributions-the problem of comparing the expected values of different strategies might remain \"undecidable.\" In computational theory, an undecidable problem is one for which no algorithm can be constructed that always leads to a correct yes-or-no answer. If these claims hold, the reliance on expected value calculations to justify specific longtermist interventions may be theoretically flawed.
This discussion is vital for anyone interested in decision theory, AI safety, or the philosophical underpinnings of risk management. It challenges the community to consider whether their frameworks are rigorous tools or merely aspirational heuristics.
Read the full post on LessWrong
\n\n