Evaluating AI and Machine Learning Algorithms for Causal Inference - Online Course

ABSTRACT

Artificial intelligence and machine learning now shape decisions across nearly every domain—from medicine and business to education and public policy. Yet one essential question often lags behind their rapid adoption: How well do these algorithms actually work in practice?

Before AI and machine learning systems can guide consequential human decisions, we must be able to evaluate their empirical performance with rigor and transparency.

This three-part lecture introduces state-of-the-art statistical frameworks for evaluating AI and machine learning algorithms through the lens of causal inference and randomized experiments.

• Part 1 – Heterogeneous Treatment Effects (HTEs):
  
  We begin by exploring how machine learning algorithms can uncover variation in treatment effects—and, crucially, how to verify whether those identified as “most likely to benefit” truly do so. You will learn statistical tools for assessing the reliability and validity of algorithmic “discoveries.”

• Part 2 – Individualized Treatment Rules (ITRs):
  
  The second session turns to algorithms that recommend personalized actions, such as individualized medical treatments or targeted marketing interventions. We discuss how to evaluate an ITR’s practical performance: If implemented in the real world, how much better would it actually perform?

• Part 3 – Human–AI Collaboration:
  
  In many real-world settings, AI systems assist—but do not replace—human decision-makers. We examine statistical challenges that arise when humans selectively follow or ignore algorithmic advice, leading to biased feedback. New methods address this “selective labels” problem and allow us to answer a critical question: Does AI truly help humans make better decisions?
  

Throughout the lectures, empirical illustrations will be drawn from medicine, the social sciences, and public policy. Each session will include an open Q&A segment for in-depth discussion with participants.

This Distinguished Speaker Series seminar will consist of three hours of lecture and Q&A, held live* via the free video-conferencing software Zoom.

*The video recording of the seminar will be made available to registrants within 24 hours and will be accessible for four weeks thereafter. That means that you can watch all of the class content and discussion even if you cannot participate synchronously.

Closed captioning is available for all live and recorded sessions. Captions can be translated to a variety of languages including Spanish, Korean, and Italian. For more information, click here.