Sensitivity Analysis for Causal Inference - Online Course
A 3-Day Livestream Seminar Taught by
Kenneth FrankWednesday, March 8 –
Friday, March 10, 2023
10:00am-12:30pm (convert to your local time)
1:30pm-3:30pm
The phrase “But have you controlled for …” is fundamental to social science, but can also create a quandary. Even after controlling for the most likely alternative explanations for an inferred effect, there may be some alternative explanation(s) that cannot be ruled out with observed data. Generally, the first response is to develop the best models that maximally leverage the available data. After that, sensitivity analyses can inform discourse about an inference by quantifying the unobserved conditions necessary to change the inference.
This course provides widely accessible ways, such as correlations or cases, to quantify the sensitivity of an inference. Specifically, in this course you will learn how to generate statements such as “An omitted variable would have to be correlated at with the predictor of interest and with the outcome to change the inference.” Or “To invalidate the inference, % of the data would have to be replaced with counterfactual cases for which the treatment had no effect.”
Rooted in the foundations of the general linear model and potential outcomes, these techniques can be adapted to a range of analyses, including logistic regression, propensity-based approaches, and multilevel models. As a result, they can broadly facilitate discourse among researchers who seek to make an inference, challengers of that inference, as well as policymakers and clinicians.
Starting March 8, we are offering this seminar as a 3-day synchronous*, livestream workshop held via the free video-conferencing software Zoom. Each day will consist of two lecture sessions which include hands-on exercises, separated by a 1-hour break. You are encouraged to join the lecture live, but will have the opportunity to view the recorded session later in the day if you are unable to attend at the scheduled time.
*We understand that finding time to participate in livestream courses can be difficult. If you prefer, you may take all or part of the course asynchronously. The video recordings will be made available within 24 hours of each session and will be accessible for four weeks after the seminar, meaning that you will get all of the class content and discussions even if you cannot participate synchronously.
Closed captioning is available for all live and recorded sessions.
More details about the course content
In this seminar, you will:
- Apply and understand techniques for quantifying the robustness of causal inferences.
- Comparing evidence to a threshold for inference.
- Understanding internal and external validity.
- Conduct sensitivity analyses in R or the on-line app http://konfound-it.com (Stata and Excel also available).
- Develop a deeper understanding of regression and the counterfactual as well as how threats to internal and external validity compare against the strength of evidence.
- Apply sensitivity analysis to a specific problem of interest that may require extensions or adaptations.
In this seminar, you will:
- Apply and understand techniques for quantifying the robustness of causal inferences.
- Comparing evidence to a threshold for inference.
- Understanding internal and external validity.
- Conduct sensitivity analyses in R or the on-line app http://konfound-it.com (Stata and Excel also available).
- Develop a deeper understanding of regression and the counterfactual as well as how threats to internal and external validity compare against the strength of evidence.
- Apply sensitivity analysis to a specific problem of interest that may require extensions or adaptations.
Computing
Participants in this course will learn statistical conceptualization, application, and software. The primary examples will be presented using the pkonfound command in R, but corresponding analyses can be done in Stata.
In addition to the examples presented by the instructor, participants should identify inferences from published examples or their own analyses to use for exercises. Optimally, those applying the techniques to their own data will have access to the data and their preferred software so they can consider alternative examples during the course.
Those using other software other than R or Stata (e.g., SPSS, SAS, or Python) can conduct analyses using the app http://konfound-it.com or spreadsheet.
Participants in this course will learn statistical conceptualization, application, and software. The primary examples will be presented using the pkonfound command in R, but corresponding analyses can be done in Stata.
In addition to the examples presented by the instructor, participants should identify inferences from published examples or their own analyses to use for exercises. Optimally, those applying the techniques to their own data will have access to the data and their preferred software so they can consider alternative examples during the course.
Those using other software other than R or Stata (e.g., SPSS, SAS, or Python) can conduct analyses using the app http://konfound-it.com or spreadsheet.
Who should register?
Participants should be familiar with statistical control as in the general linear model (i.e., regression).
Participants should be familiar with statistical control as in the general linear model (i.e., regression).
Seminar outline
Day 1: Robustness of Inference to Replacement (RIR) of Data
-
- Overview and introduction
- Counterfactual logic and unobserved data
- Reflection & discussion
- Quantifying the robustness of inferences as replacing observed data with unobserved data
- Reflection & discussion
- Examples for internal and external validity
- Benchmarks
- Software & exercises
- Application of the RIR to dichotomous outcomes
- Case switching and fragility
- Replacing data to create switches
- Reflection & discussion
- Examples, software & exercises
- Alternatives
Day 2: Impact Threshold of a Confounding Variable (ITCV)
-
- Logic of regression and partial correlation
- Reflection & discussion
- Reasoning behind the ITCV
- Applications and usage of the ITCV
- Examples, software & exercises
- Benchmarks
- Absorption of impact
- Alternatives
Day 3: Extensions: Supported Exploration
-
- Participant generated examples
- Extension topics based on participant preferences:
- Specifying thresholds for inference
- Alternative scenarios: Type II errors, alternative replacement data
- Extensions to econometric models: diff in diff, regression discontinuity
- Extensions to heterogeneous treatment effects
- Adaptation of ITCV to mediation
- Adaptation of RIR to Value Added Models: Violations of SUTVA
- Comparison of RIR and ITCV
Day 1: Robustness of Inference to Replacement (RIR) of Data
-
- Overview and introduction
- Counterfactual logic and unobserved data
- Reflection & discussion
- Quantifying the robustness of inferences as replacing observed data with unobserved data
- Reflection & discussion
- Examples for internal and external validity
- Benchmarks
- Software & exercises
- Application of the RIR to dichotomous outcomes
- Case switching and fragility
- Replacing data to create switches
- Reflection & discussion
- Examples, software & exercises
- Alternatives
Day 2: Impact Threshold of a Confounding Variable (ITCV)
-
- Logic of regression and partial correlation
- Reflection & discussion
- Reasoning behind the ITCV
- Applications and usage of the ITCV
- Examples, software & exercises
- Benchmarks
- Absorption of impact
- Alternatives
- Logic of regression and partial correlation
Day 3: Extensions: Supported Exploration
-
- Participant generated examples
- Extension topics based on participant preferences:
- Specifying thresholds for inference
- Alternative scenarios: Type II errors, alternative replacement data
- Extensions to econometric models: diff in diff, regression discontinuity
- Extensions to heterogeneous treatment effects
- Adaptation of ITCV to mediation
- Adaptation of RIR to Value Added Models: Violations of SUTVA
- Comparison of RIR and ITCV
Related readings
*Frank, K. A., *Lin, Q., *Maroulis, S., *Mueller, A. S., Xu, R., Rosenberg, J. M., … & Zhang, L. 2021. Hypothetical case replacement can be used to quantify the robustness of trial results. Journal of Clinical Epidemiology, 134, 150-159. *authors listed alphabetically.
Frank, K.A., Maroulis, S., Duong, M., and Kelcey, B. 2013. What would it take to Change an Inference?: Using Rubin’s Causal Model to Interpret the Robustness of Causal Inferences. Education, Evaluation and Policy Analysis. Vol 35: 437-460.
Frank, K. A. 2000. “Impact of a Confounding Variable on a Regression Coefficient.” Sociological Methods and Research, 29(2), 147-194.
*Frank, K. A. and *Min, K. 2007. Indices of Robustness for Sample Representation. Sociological Methodology. Vol 37, 349-392. *authors listed alphabetically.
Xu, R., Frank, K. A., Maroulis, S. J., & Rosenberg, J. M. 2019. konfound: Command to quantify robustness of causal inferences. The Stata Journal, 19(3), 523-550.
*Frank, K. A., *Lin, Q., *Maroulis, S., *Mueller, A. S., Xu, R., Rosenberg, J. M., … & Zhang, L. 2021. Hypothetical case replacement can be used to quantify the robustness of trial results. Journal of Clinical Epidemiology, 134, 150-159. *authors listed alphabetically.
Frank, K.A., Maroulis, S., Duong, M., and Kelcey, B. 2013. What would it take to Change an Inference?: Using Rubin’s Causal Model to Interpret the Robustness of Causal Inferences. Education, Evaluation and Policy Analysis. Vol 35: 437-460.
Frank, K. A. 2000. “Impact of a Confounding Variable on a Regression Coefficient.” Sociological Methods and Research, 29(2), 147-194.
*Frank, K. A. and *Min, K. 2007. Indices of Robustness for Sample Representation. Sociological Methodology. Vol 37, 349-392. *authors listed alphabetically.
Xu, R., Frank, K. A., Maroulis, S. J., & Rosenberg, J. M. 2019. konfound: Command to quantify robustness of causal inferences. The Stata Journal, 19(3), 523-550.
Payment information
The fee of $995 includes all course materials.
PayPal and all major credit cards are accepted.
Our Tax ID number is 26-4576270.
The fee of $995 includes all course materials.
PayPal and all major credit cards are accepted.
Our Tax ID number is 26-4576270.