Skip to yearly menu bar Skip to main content


Poster
in
Workshop: Workshop on Technical AI Governance

Expert Survey: AI Safety & Security Research Priorities

Joe O'Brien · Jeremy Dolan · Jeba Sania · Jay Kim · Rocio Labrador · Jonah Dykhuizen · Sebastian Becker · Jam Kraprayoon


Abstract:

As AI systems approach broadly human-level performance, safety and security research is urgently needed to prevent severe harms from AI and ensure its benefits can be safely and reliably realized. To inform strategic investment, we surveyed 53 diverse experts on the importance and tractability of 105 technical AI safety and security research areas. Our survey revealed several highly promising research directions centered around robust early warning and monitoring of AI risks. Some of the most promising sub-areas included specific capability evaluations (e.g., CBRN, cyber, and deception), understanding emergence and scaling laws, and advancing agent oversight. This study is the first to quantify expert priorities across a comprehensive taxonomy of AI safety and security research directions and to produce a data-driven ranking of their potential impact. These rankings may support evidence-based decisions about how to effectively deploy resources toward AI safety and security research.

Chat is not available.