Safe and Constrained AI

How can we design AI systems that are not only powerful but also provably safe and trustworthy? This advanced PhD seminar surveys algorithmic methods to enforce hard constraints in machine learning, reinforcement learning, and generative AI. Topics include classical constrained optimization (Lagrangian methods, robust and stochastic programming), safe reinforcement learning (trust regions, Lyapunov functions, reachability, shielding), hybrid ML-optimization methods (projection networks, solver-in-the-loop architectures), and alignment strategies for large language models (fine-tuning, model editing, tool use, and interactive alignment). Each week highlights a key theoretical result alongside state-of-the-art research, with applications spanning robotics, finance, healthcare, energy, and language models. Students will critically assess the strengths and limitations of these methods and develop final projects that apply or extend them in real-world domains. Prerequisites: optimization at the level of CME 307 or EE 364a.