Using AI to Build Better Experiments - Online Course
A 3-Day Livestream Seminar Taught by
Charles CrabtreeWednesday, January 14 –
Friday, January 16, 2026
10:00am-12:30pm (convert to your local time)
1:30pm-3:30pm
This seminar teaches researchers how to leverage artificial intelligence to design, implement, and analyze experimental studies more effectively. You’ll learn practical techniques for using large language models (LLMs) to generate experimental materials, validate treatments, deploy AI-powered chatbot experiments, conduct automated text analysis, and communicate results. The course emphasizes hands-on application, with participants working through real experimental challenges using state-of-the-art AI tools integrated with R.
Throughout the course, you’ll progress through the complete experimental lifecycle: from foundational prompt engineering and material generation (Day 1), to rigorous validation and advanced applications including conversational experiments and automated text analysis (Day 2), and finally to comprehensive analysis and publication (Day 3). We’ll cover best practices for prompt engineering, validation strategies to ensure AI outputs meet scientific standards, and methods for transparent reporting of AI-assisted research.
Starting January 14, this seminar will be presented as a 3-day synchronous, livestream workshop via Zoom. Each day will feature two lecture sessions with hands-on exercises, separated by a 1-hour break. Live attendance is recommended for the best experience. But if you can’t join in real time, recordings will be available within 24 hours and can be accessed for four weeks after the seminar.
Closed captioning is available for all live and recorded sessions. Captions can be translated to a variety of languages including Spanish, Korean, and Italian. For more information, click here.
ECTS Equivalent Points: 1
Computing
To participate in the hands-on exercises, you should have:
- A computer with the most recent version of R installed.
- Rstudio (strongly recommended), a free front-end for R available for Windows, Mac, and Linux.
- The following R packages installed:
- tidyverse (for data manipulation and visualization)
- ggplot2 (for creating publication-quality graphics)
- marginaleffects (for estimating and visualizing treatment effects)
- modelsummary (for creating regression tables)
- tidytext (for text analysis and processing)
- httr and jsonlite (for API calls to LLMs)
- quanteda (for advanced text analysis)
- shiny (for building interactive chatbot interfaces)
- Access to at least one LLM platform (you will receive guidance on options):
- OpenAI API (GPT-4, GPT-4o) – recommended
- Anthropic API (Claude)
- Google API (Gemini)
- Or other LLM platforms
- Note: Most exercises can be completed with any LLM. Basic API costs are minimal (typically $5-20 for course exercises).
If you’d like to take this course but are concerned that you don’t know enough R, there are excellent online resources for learning the basics. Here are our recommendations.
To participate in the hands-on exercises, you should have:
- A computer with the most recent version of R installed.
- Rstudio (strongly recommended), a free front-end for R available for Windows, Mac, and Linux.
- The following R packages installed:
- tidyverse (for data manipulation and visualization)
- ggplot2 (for creating publication-quality graphics)
- marginaleffects (for estimating and visualizing treatment effects)
- modelsummary (for creating regression tables)
- tidytext (for text analysis and processing)
- httr and jsonlite (for API calls to LLMs)
- quanteda (for advanced text analysis)
- shiny (for building interactive chatbot interfaces)
- Access to at least one LLM platform (you will receive guidance on options):
- OpenAI API (GPT-4, GPT-4o) – recommended
- Anthropic API (Claude)
- Google API (Gemini)
- Or other LLM platforms
- Note: Most exercises can be completed with any LLM. Basic API costs are minimal (typically $5-20 for course exercises).
If you’d like to take this course but are concerned that you don’t know enough R, there are excellent online resources for learning the basics. Here are our recommendations.
Who should register?
This course is designed for researchers at the Ph.D. level or beyond who conduct experimental research in any field, including social sciences, behavioral sciences, medicine, public health, and business. Whether you’re designing survey experiments, field experiments, or lab studies, this course will help you incorporate AI tools into your workflow while maintaining scientific rigor.
No prior experience with AI or LLMs is required, though you should be comfortable with basic experimental design concepts. The course is ideal for researchers who want to increase their productivity, improve the quality of their experimental materials, and explore new possibilities for measurement and analysis.
You should also have familiarity with experimental research methods (e.g., randomized controlled trials, factorial designs, survey experiments), a basic knowledge of statistical analysis, and a working knowledge of R (ability to load data, run basic analyses, and create plots).
This course is designed for researchers at the Ph.D. level or beyond who conduct experimental research in any field, including social sciences, behavioral sciences, medicine, public health, and business. Whether you’re designing survey experiments, field experiments, or lab studies, this course will help you incorporate AI tools into your workflow while maintaining scientific rigor.
No prior experience with AI or LLMs is required, though you should be comfortable with basic experimental design concepts. The course is ideal for researchers who want to increase their productivity, improve the quality of their experimental materials, and explore new possibilities for measurement and analysis.
You should also have familiarity with experimental research methods (e.g., randomized controlled trials, factorial designs, survey experiments), a basic knowledge of statistical analysis, and a working knowledge of R (ability to load data, run basic analyses, and create plots).
Seminar outline
Day 1: Foundations and Material Generation
-
- Introduction to LLMs and API integration with R
- Prompt engineering fundamentals
- Generating experimental materials (vignettes, surveys, stimuli)
- Designing complete treatment packages
- Initial validation approaches
Day 2: Validation and Advanced Applications
-
- Manipulation checks and confound detection
- Systematic validation protocols
- Pre-testing and pilot analysis
- Building conversational AI experiments
- Automated coding and content analysis
- Large-scale text processing
Day 3: Analysis and Communication
-
- LLM-as-participant designs
- Analyzing treatment effects with R
- Heterogeneous effects and subgroup analysis
- Publication-quality tables and visualizations
- Ethical considerations and transparent reporting
Day 1: Foundations and Material Generation
-
- Introduction to LLMs and API integration with R
- Prompt engineering fundamentals
- Generating experimental materials (vignettes, surveys, stimuli)
- Designing complete treatment packages
- Initial validation approaches
Day 2: Validation and Advanced Applications
-
- Manipulation checks and confound detection
- Systematic validation protocols
- Pre-testing and pilot analysis
- Building conversational AI experiments
- Automated coding and content analysis
- Large-scale text processing
Day 3: Analysis and Communication
-
- LLM-as-participant designs
- Analyzing treatment effects with R
- Heterogeneous effects and subgroup analysis
- Publication-quality tables and visualizations
- Ethical considerations and transparent reporting
Payment information
The fee of $995 includes all course materials.
PayPal and all major credit cards are accepted.
Our Tax ID number is 26-4576270.
The fee of $995 includes all course materials.
PayPal and all major credit cards are accepted.
Our Tax ID number is 26-4576270.