Schmidt Sciences Targets Frontier Risks with "Science of Trustworthy AI" RFP
Coverage of lessw-blog
In a recent post, LessWrong highlights a significant new funding initiative from Schmidt Sciences aimed at establishing a rigorous scientific foundation for AI safety and trustworthiness.
In a recent update shared via LessWrong, the community highlighted Schmidt Sciences' new Request for Proposals (RFP) for the "Science of Trustworthy AI." This initiative represents a substantial commitment to formalizing the study of AI safety, moving beyond theoretical debates toward empirical, funded research aimed at understanding, predicting, and controlling frontier AI systems.
As foundation models become increasingly integrated into critical infrastructure, the gap between capability and control has widened. Current evaluation methods often rely on static benchmarks that may not predict behavior under novel conditions or distribution shifts. This RFP addresses that fragility by seeking proposals that establish a "science" of trust-specifically focusing on rigorous characterization of misalignment and the development of decision-relevant constructs with predictive validity.
The scope of the solicitation is broad yet targeted. It prioritizes research into three specific aims: characterizing the drivers of misalignment, creating generalizable measurements and interventions, and designing oversight protocols for superhuman and multi-agent scenarios. The inclusion of multi-agent risks and superhuman oversight suggests a forward-looking approach, anticipating challenges that will arise as AI systems become more autonomous and capable than their human operators.
Perhaps most significant is the structure of the funding. While Tier 1 grants offer up to $1 million for smaller projects, Schmidt Sciences has explicitly expressed a preference for ambitious Tier 2 proposals ($1 million to over $5 million). This signals a strategic shift toward supporting large-scale, concentrated research efforts that can deliver robust, systemic solutions rather than incremental improvements. For researchers and labs focused on the long-term stability of advanced AI, this represents a pivotal resource.
For full details on the application process and specific research aims, we recommend reviewing the original discussion.
Read the full post on LessWrong
Key Takeaways
- Schmidt Sciences is funding research to understand, predict, and control risks from frontier AI systems.
- The agenda focuses on three pillars: misalignment forecasting, generalizable interventions, and oversight for superhuman/multi-agent systems.
- There is a stated preference for ambitious "Tier 2" proposals ($1M-$5M+) over smaller, fragmented grants.
- The initiative aims to transition AI safety from theoretical discussions to a rigorous, empirical scientific discipline.