Sensitivity Analysis for Causal Inference - Online Course
A 4-Day Livestream Seminar Taught by
Kenneth FrankTuesday, July 11 –
Friday, July 14, 2023
10:30am-12:30pm (convert to your local time)
1:30pm-3:00pm
The phrase “But have you controlled for …” is fundamental to social science, but can also create a quandary. Even after controlling for the most likely alternative explanations for an inferred effect, there may be some alternative explanation(s) that cannot be ruled out with observed data. Generally, the first response is to develop the best models that maximally leverage the available data. After that, sensitivity analyses can inform discourse about an inference by quantifying the unobserved conditions necessary to change the inference.
This course provides widely accessible ways, such as correlations or cases, to quantify the sensitivity of an inference. Specifically, in this course you will learn how to generate statements such as “An omitted variable would have to be correlated at with the predictor of interest and with the outcome to change the inference.” Or “To invalidate the inference, % of the data would have to be replaced with counterfactual cases for which the treatment had no effect.”
Rooted in the foundations of the general linear model and potential outcomes, these techniques can be adapted to a range of analyses, including logistic regression, propensity-based approaches, and multilevel models. As a result, they can broadly facilitate discourse among researchers who seek to make an inference, challengers of that inference, as well as policymakers and clinicians.
Starting July 11, we are offering this seminar as a 4-day synchronous*, livestream workshop held via the free video-conferencing software Zoom. Each day will consist of two lecture sessions which include hands-on exercises, separated by a 1-hour break. You are encouraged to join the lecture live, but will have the opportunity to view the recorded session later in the day if you are unable to attend at the scheduled time.
*We understand that finding time to participate in livestream courses can be difficult. If you prefer, you may take all or part of the course asynchronously. The video recordings will be made available within 24 hours of each session and will be accessible for four weeks after the seminar, meaning that you will get all of the class content and discussions even if you cannot participate synchronously.
Closed captioning is available for all live and recorded sessions. Live captions can be translated to a variety of languages including Spanish, Korean, and Italian. For more information, click here.
More details about the course content
In this seminar, you will:
- Apply and understand techniques for quantifying the robustness of causal inferences.
- Comparing evidence to a threshold for inference.
- Understanding internal and external validity.
- Conduct sensitivity analyses in R or the on-line app http://konfound-it.com (Stata and Excel also available).
- Develop a deeper understanding of regression and the counterfactual as well as how threats to internal and external validity compare against the strength of evidence.
- Apply sensitivity analysis to a specific problem of interest that may require extensions or adaptations.
In this seminar, you will:
- Apply and understand techniques for quantifying the robustness of causal inferences.
- Comparing evidence to a threshold for inference.
- Understanding internal and external validity.
- Conduct sensitivity analyses in R or the on-line app http://konfound-it.com (Stata and Excel also available).
- Develop a deeper understanding of regression and the counterfactual as well as how threats to internal and external validity compare against the strength of evidence.
- Apply sensitivity analysis to a specific problem of interest that may require extensions or adaptations.
Computing
Participants in this course will learn statistical conceptualization, application, and software. The primary examples will be presented using the pkonfound command in R, but corresponding analyses can be done in Stata.
In addition to the examples presented by the instructor, participants should identify inferences from published examples or their own analyses to use for exercises. Optimally, those applying the techniques to their own data will have access to the data and their preferred software so they can consider alternative examples during the course.
Those using other software other than R or Stata (e.g., SPSS, SAS, or Python) can conduct analyses using the app http://konfound-it.com or spreadsheet.
Participants in this course will learn statistical conceptualization, application, and software. The primary examples will be presented using the pkonfound command in R, but corresponding analyses can be done in Stata.
In addition to the examples presented by the instructor, participants should identify inferences from published examples or their own analyses to use for exercises. Optimally, those applying the techniques to their own data will have access to the data and their preferred software so they can consider alternative examples during the course.
Those using other software other than R or Stata (e.g., SPSS, SAS, or Python) can conduct analyses using the app http://konfound-it.com or spreadsheet.
Who should register?
If you want to make causal inferences from observational data with potential omitted variables or from randomized experiments with non-volunteer samples, then this course is for you. You should have a solid working knowledge of the basic principles of statistical inference (e.g., standard errors, hypothesis tests, confidence intervals). It’s also essential that you have a good understanding of the basic theory and practice of linear regression, including specification, interpretation, and inference.
If you want to make causal inferences from observational data with potential omitted variables or from randomized experiments with non-volunteer samples, then this course is for you. You should have a solid working knowledge of the basic principles of statistical inference (e.g., standard errors, hypothesis tests, confidence intervals). It’s also essential that you have a good understanding of the basic theory and practice of linear regression, including specification, interpretation, and inference.
Seminar outline
Day 1
-
- Robustness of inference to replacement (RIR)
- Internal validity
- Konfound software
Day 2
-
Day 3
-
- Correlation framework: impact threshold for a confounding variable (ITCV: internal validity)
- Multivariate extension of ITCV
- Correlational for external validity
- Alternatives: Oster’s coefficient of proportionality; Cinelli & Haslett’s robustness value; Rosenbaum’s Γ; others reviewed here.
Day 4
-
- Prep presentations
- Share presentations/specific applications
Day 1
-
- Robustness of inference to replacement (RIR)
- Internal validity
- Konfound software
- Robustness of inference to replacement (RIR)
Day 2
Day 3
-
- Correlation framework: impact threshold for a confounding variable (ITCV: internal validity)
- Multivariate extension of ITCV
- Correlational for external validity
- Alternatives: Oster’s coefficient of proportionality; Cinelli & Haslett’s robustness value; Rosenbaum’s Γ; others reviewed here.
Day 4
-
- Prep presentations
- Share presentations/specific applications
Related readings
*Frank, K. A., *Lin, Q., *Maroulis, S., *Mueller, A. S., Xu, R., Rosenberg, J. M., … & Zhang, L. 2021. Hypothetical case replacement can be used to quantify the robustness of trial results. Journal of Clinical Epidemiology, 134, 150-159. *authors listed alphabetically.
Frank, K.A., Maroulis, S., Duong, M., and Kelcey, B. 2013. What would it take to Change an Inference?: Using Rubin’s Causal Model to Interpret the Robustness of Causal Inferences. Education, Evaluation and Policy Analysis. Vol 35: 437-460.
Frank, K. A. 2000. “Impact of a Confounding Variable on a Regression Coefficient.” Sociological Methods and Research, 29(2), 147-194.
*Frank, K. A. and *Min, K. 2007. Indices of Robustness for Sample Representation. Sociological Methodology. Vol 37, 349-392. *authors listed alphabetically.
Xu, R., Frank, K. A., Maroulis, S. J., & Rosenberg, J. M. 2019. konfound: Command to quantify robustness of causal inferences. The Stata Journal, 19(3), 523-550.
*Frank, K. A., *Lin, Q., *Maroulis, S., *Mueller, A. S., Xu, R., Rosenberg, J. M., … & Zhang, L. 2021. Hypothetical case replacement can be used to quantify the robustness of trial results. Journal of Clinical Epidemiology, 134, 150-159. *authors listed alphabetically.
Frank, K.A., Maroulis, S., Duong, M., and Kelcey, B. 2013. What would it take to Change an Inference?: Using Rubin’s Causal Model to Interpret the Robustness of Causal Inferences. Education, Evaluation and Policy Analysis. Vol 35: 437-460.
Frank, K. A. 2000. “Impact of a Confounding Variable on a Regression Coefficient.” Sociological Methods and Research, 29(2), 147-194.
*Frank, K. A. and *Min, K. 2007. Indices of Robustness for Sample Representation. Sociological Methodology. Vol 37, 349-392. *authors listed alphabetically.
Xu, R., Frank, K. A., Maroulis, S. J., & Rosenberg, J. M. 2019. konfound: Command to quantify robustness of causal inferences. The Stata Journal, 19(3), 523-550.
Payment information
The fee of $995 includes all course materials.
PayPal and all major credit cards are accepted.
Our Tax ID number is 26-4576270.
The fee of $995 includes all course materials.
PayPal and all major credit cards are accepted.
Our Tax ID number is 26-4576270.