How can we extract deeper insights from LLM evaluations?

Join experts from the UK AI Security Institute for an interactive discussion at ICML focused on improving how we analyse, interpret, and act on evaluation data for frontier AI systems. As large language models become more capable and influential, evaluations have become a cornerstone of scientific understanding, safety assessments, and deployment decisions. Yet current evaluation designs and methodologies are often poorly suited to answering the questions we care most about—such as uncovering latent capabilities, forecasting performance trajectories, and identifying dangerous failure modes.

This session will explore four key dimensions of evaluation methodology: developing tools for richer evaluation-data analysis; advancing statistical techniques for uncertainty and variability; building efficient evaluation pipelines that prioritise signal-rich tasks; and mapping evaluation results onto capability or risk thresholds. We’ll identify open research questions, promising methodological directions, and opportunities for collaboration to make evaluations more rigorous, interpretable, and decision-relevant.

Whether you you are an eval designer yourself, train your own models, or work on risks related to safety and misuse, this session will help you think critically about the importance of evaluation insights to your own work.