# Estimating AI Capabilities Across Job Tasks via Semantic Similarity

> Coverage of lessw-blog

**Published:** March 17, 2026
**Author:** PSEEDR Editorial
**Category:** enterprise

**Tags:** AI Capability, Workforce Transformation, Semantic Similarity, Enterprise AI, Automation

**Canonical URL:** https://pseedr.com/enterprise/estimating-ai-capabilities-across-job-tasks-via-semantic-similarity

---

A novel tool developed by a LessWrong contributor leverages vector embeddings and the Anthropic Economic Index to estimate AI's current capabilities across various job tasks, offering a nuanced view of workforce transformation.

**The Hook:** In a recent post, lessw-blog discusses the development of a novel AI capability estimator tool designed to visualize and predict artificial intelligence's proficiency across different occupations and specific job tasks. Published on the LessWrong forum, this technical breakdown offers a fascinating glimpse into how we can better measure the practical utility of AI in the modern workforce.

**The Context:** For enterprises and organizational leaders, understanding job-level AI capability is no longer just an academic exercise; it is crucial for strategic planning, workforce transformation, and identifying concrete areas for automation. Traditional metrics for measuring AI impact often rely on simple adoption data or broad industry surveys. These conventional methods can lag significantly behind actual technological capabilities, or they may fail to capture the nuanced, granular ways AI can be applied to specific daily workflows. As large language models and specialized AI tools become more adept at handling complex, multi-step processes, organizations desperately need better, more precise frameworks to evaluate where these technologies can generate immediate efficiency gains and return on investment.

**The Gist:** lessw-blog's post explores these exact dynamics by introducing a creative methodology that infers AI capability through semantic similarity, rather than relying on direct adoption metrics alone. Building upon the foundation of the Anthropic Economic Index and leveraging comprehensive occupational databases, the author created vector embeddings for various task descriptions. By finding the semantically nearest tasks across a wide array of different occupations, the tool dynamically updates capability scores based on where AI is already successfully performing comparable work. For example, if AI is highly capable of data synthesis in a financial role, the tool can infer similar capabilities for data synthesis tasks in a healthcare administration role.

To ensure the accuracy and reliability of these estimations, the developer implemented a crucial sanity check using Anthropic's Claude Haiku model. This step was necessary to prevent physical tasks-such as manual labor or operating heavy machinery-from erroneously inheriting high capability scores from purely informational, yet semantically similar, tasks. The author maintains an objective view of the project, acknowledging the tool's current limitations. For instance, certain highly technical tasks, like selecting programming languages for web developers, currently show a zero percent capability score, highlighting the ongoing challenges and edge cases in accurately mapping AI proficiency. Interestingly, the development of the tool itself was a meta-exercise in AI collaboration, as it was programmed with the assistance of Claude Opus.

**Conclusion:** This analysis provides a highly practical application of AI and machine learning concepts to solve a pressing enterprise challenge: accurately mapping the true frontier of AI utility in the workplace. For technology executives, HR strategists, and leaders focused on workforce transformation, understanding this semantic approach to capability estimation offers a significant advantage over traditional adoption metrics. [Read the full post](https://www.lesswrong.com/posts/YC8oo7ikoaCuD4EHa/i-made-a-job-level-ai-capability-estimator-by-asking-where).

### Key Takeaways

*   A new tool visualizes AI capabilities across occupations by analyzing the semantic similarity of specific job tasks.
*   The methodology uses vector embeddings and the Anthropic Economic Index to infer capability based on AI usage in comparable roles.
*   Claude Haiku was utilized as a sanity check to differentiate between physical and informational tasks, preventing skewed scoring.
*   The project demonstrates a shift from tracking simple AI adoption to estimating actual workflow capability and potential enterprise ROI.

[Read the original post at lessw-blog](https://www.lesswrong.com/posts/YC8oo7ikoaCuD4EHa/i-made-a-job-level-ai-capability-estimator-by-asking-where)

---

## Sources

- https://www.lesswrong.com/posts/YC8oo7ikoaCuD4EHa/i-made-a-job-level-ai-capability-estimator-by-asking-where