Some resources for physicists transitioning into AI safety

This is a list of resources that may be useful for physicists who are considering a move into AI safety.¹ For those coming from the physics world, Iliad may be a nice place to look into.

First research projects

For a first research project, I would look at:

• SPAR: A three-month, part-time, remote research programme with mentorship.
• MARS: A programme for early-career researchers, with an initial in-person week followed by part-time remote research in small teams.
• Iliad Fellowship: Three months of full-time, mentored research in applied mathematics for AI alignment.
• PIBBSS Fellowship: A funded, approximately three-month programme specifically designed to help researchers from other disciplines move into AI safety. Prior AI safety expertise is not required, although they look for research ability comparable to PhD-level work.
• Pivotal Fellowship: A funded, 15-week, full-time AI safety research fellowship, held in person in London with expert mentorship.
• CBAI Fellowship: A funded, ten-week, intensive research programme hosted in person in Cambridge, Massachusetts, with weekly mentorship. It covers interpretability, multi-agent safety, formal verification and other technical and governance topics.
• MATS: A funded research fellowship covering alignment, interpretability and security. It is more selective, but worth exploring when an individual mentor’s work connects with your background.

Taught programmes

If you would first like to build familiarity with the field, there are also some useful taught programmes:

• BlueDot: Free introductory courses, starting with a short overview of AI and progressing to technical AI safety.
• ARENA: Practical training in deep learning, transformers and interpretability. Its materials are available online for self-study. The bootcamp expects Python proficiency and a mathematical foundation.
• Iliad Intensive: A four-week taught programme specifically welcoming people from theoretical physics and mathematics, focused on foundational alignment research.

Deciding where to start

Two LessWrong posts are useful for deciding where to start:

• Roadmap through AI safety programs for early-career technical researchers: An overview of introductory courses, first research opportunities and more selective fellowships.
• How to Become a Mechanistic Interpretability Researcher by Neel Nanda: Practical advice on learning the essentials and progressing through small research projects.