# Curated Digest: How to Not Do Decision Theory Backwards

> Coverage of lessw-blog

**Published:** March 17, 2026
**Author:** PSEEDR Editorial
**Category:** risk

**Tags:** Decision Theory, AI Safety, Epistemology, Risk Management, Machine Learning

**Canonical URL:** https://pseedr.com/risk/curated-digest-how-to-not-do-decision-theory-backwards

---

A recent analysis from lessw-blog critiques 'backwards' reasoning in decision theory, highlighting the risks of justifying decision-making principles based on desired outcomes rather than objective logic.

In a recent post, lessw-blog discusses a pervasive and critical flaw in how decision theory, ethics, and epistemology are frequently constructed. The author presents a compelling critique of 'backwards' reasoning, a methodological trap where desired outcomes or intuitive verdicts are used to justify the foundational principles of decision-making, rather than allowing sound principles to dictate the outcomes.

This topic is critical because the mathematical and philosophical frameworks of decision theory are not just academic exercises; they are the blueprints for modern artificial intelligence and machine learning systems. In the rapidly evolving landscape of AI safety and risk management, the logic that governs autonomous choices is paramount. If developers and researchers build AI decision-making frameworks upon backwards justifications-where a predetermined 'safe' or 'correct' action dictates the rules of the system-it creates a fragile architecture. Such systems might perform well in testing but fail unpredictably in novel situations because their underlying logic is reverse-engineered rather than objectively sound. Ensuring that AI systems make decisions for genuinely good reasons, derived from robust forward-moving logic, is an absolute necessity for ethical deployment and long-term trustworthiness.

lessw-blog's post explores these dynamics by dissecting how some common approaches to decision theory go astray. The author points out that many theorists start with a bottom-line verdict-such as concluding 'I should definitely choose this specific action'-and then work backward to construct or validate a decision theory that outputs that exact verdict. The analysis argues that this approach is fundamentally backwards and compromises the integrity of the theoretical framework. Furthermore, the author warns that embedding this backwards move within more sophisticated and nuanced methodologies, such as reflective equilibrium, does not resolve the underlying issue. It merely obscures the logical fallacy. To vividly illustrate this flaw, the post introduces an exaggerated example involving a casino game. In this thought experiment, the author demonstrates the absurdity of justifying probability assessments or expected value calculations based purely on a desired financial outcome, rather than relying on the objective, mathematical odds of the game. This analogy perfectly encapsulates why decision-making rules must remain independent of our preferences for specific results.

For professionals engaged in AI alignment, algorithmic design, and risk evaluation, this analysis serves as a crucial reminder to audit the foundational logic of their models. Recognizing and eliminating backwards reasoning is a necessary step toward building robust, reliable systems. We highly recommend reviewing the original piece to fully grasp the nuances of the argument and the implications for future theoretical work. [Read the full post](https://www.lesswrong.com/posts/gRBH3NDC8MNQGaaBQ/how-to-not-do-decision-theory-backwards) to explore the complete analysis.

### Key Takeaways

*   Starting with a desired outcome to justify decision-making principles is a fundamentally flawed, 'backwards' approach.
*   Nuanced frameworks like reflective equilibrium do not inherently fix this logical error; they often just obscure it.
*   Objective odds and sound reasoning must drive decisions, rather than reverse-engineering rules to fit a preferred verdict.
*   Avoiding backwards reasoning is critical for AI safety, ensuring models make choices based on reliable, unbiased, and forward-moving logic.

[Read the original post at lessw-blog](https://www.lesswrong.com/posts/gRBH3NDC8MNQGaaBQ/how-to-not-do-decision-theory-backwards)

---

## Sources

- https://www.lesswrong.com/posts/gRBH3NDC8MNQGaaBQ/how-to-not-do-decision-theory-backwards