Two Key Techniques for Quantifying the Robustness of Causal Inferences - Online Course
An 8-Hour Livestream Seminar Taught by
Kenneth Frank10:30am-12:30pm (convert to your local time)
1:00pm-3:00pm
In just 8 hours (over 2 days), you will get hands-on experience with quantifying the sensitivity of a causal inference using two specialized techniques – Robustness of Inference to Replacement (RIR) and Impact Threshold for a Confounding Variable (ITCV).
The phrase “But have you controlled for …” is fundamental to social science, but can also create a quandary. Even after controlling for the most likely alternative explanations for an inferred effect, there may be some alternative explanation(s) that cannot be ruled out with observed data. Generally, the first response is to develop the best models that maximally leverage the available data. After that, sensitivity analyses can inform discourse about an inference by quantifying the unobserved conditions necessary to change the inference.
In this course, you will learn how to generate statements such as “An omitted variable would have to be correlated at ___ with the predictor of interest and with the outcome to change the inference.” Or “To invalidate the inference, __% of the data would have to be replaced with counterfactual cases for which the treatment had no effect.” Because these statements express sensitivity in terms of correlations or cases they have wide accessibility.
Rooted in the foundations of the general linear model and potential outcomes, the techniques can be adapted to a range of analyses, including logistic regression, propensity-based approaches, and multilevel models. As a result, they can broadly facilitate discourse about inferences among researchers who seek to make an inference, challengers of that inference, policymakers, and clinicians.
Starting February 29, we are offering this seminar as an 8-hour synchronous*, livestream workshop held via the free video-conferencing software Zoom. Each day will consist of two 2-hour lecture sessions which include hands-on exercises, separated by a 30-minute break. You are encouraged to join the lecture live, but will have the opportunity to view the recorded session later in the day if you are unable to attend at the scheduled time.
*We understand that finding time to participate in livestream courses can be difficult. If you prefer, you may take all or part of the course asynchronously. The video recordings will be made available within 24 hours of each session and will be accessible for four weeks after the seminar, meaning that you will get all of the class content and discussions even if you cannot participate synchronously.
Closed captioning is available for all live and recorded sessions. Live captions can be translated to a variety of languages including Spanish, Korean, and Italian. For more information, click here.
More details about the course content
In this seminar, you will:
- Apply and understand techniques for quantifying the robustness of causal inferences.
- Comparing evidence to a threshold for inference.
- Understanding internal and external validity.
- Conduct sensitivity analyses in R or the on-line app http://konfound-it.com (Stata and Excel also available).
- Develop a deeper understanding of regression and the counterfactual as well as how threats to internal and external validity compare against the strength of evidence.
- Apply sensitivity analysis to a specific problem of interest that may require extensions or adaptations.
In this seminar, you will:
- Apply and understand techniques for quantifying the robustness of causal inferences.
- Comparing evidence to a threshold for inference.
- Understanding internal and external validity.
- Conduct sensitivity analyses in R or the on-line app http://konfound-it.com (Stata and Excel also available).
- Develop a deeper understanding of regression and the counterfactual as well as how threats to internal and external validity compare against the strength of evidence.
- Apply sensitivity analysis to a specific problem of interest that may require extensions or adaptations.
Computing
You will learn statistical conceptualization, application, and software. The primary examples will be presented using the pkonfound command in R, but corresponding analyses can be done in Stata.
In addition to the examples presented by the instructor, you should identify inferences from published examples or your own analyses to use for exercises. Optimally, if you plan to apply the techniques to your own data, you will have access to the data and your preferred software so you can consider alternative examples during the course.
Those using other software other than R or Stata (e.g., SPSS, SAS, or Python) can conduct analyses using the app http://konfound-it.com or spreadsheet.
You will learn statistical conceptualization, application, and software. The primary examples will be presented using the pkonfound command in R, but corresponding analyses can be done in Stata.
In addition to the examples presented by the instructor, you should identify inferences from published examples or your own analyses to use for exercises. Optimally, if you plan to apply the techniques to your own data, you will have access to the data and your preferred software so you can consider alternative examples during the course.
Those using other software other than R or Stata (e.g., SPSS, SAS, or Python) can conduct analyses using the app http://konfound-it.com or spreadsheet.
Who should register?
If you want to make causal inferences from observational data with potential omitted variables or from randomized experiments with non-volunteer samples, then this course is for you. You should have a solid working knowledge of the basic principles of statistical inference (e.g., standard errors, hypothesis tests, confidence intervals). It’s also essential that you have a good understanding of the basic theory and practice of linear regression, including specification, interpretation, and inference.
If you want to make causal inferences from observational data with potential omitted variables or from randomized experiments with non-volunteer samples, then this course is for you. You should have a solid working knowledge of the basic principles of statistical inference (e.g., standard errors, hypothesis tests, confidence intervals). It’s also essential that you have a good understanding of the basic theory and practice of linear regression, including specification, interpretation, and inference.
Seminar outline
Day 1
-
- Robustness of Inference to Replacement (RIR)
- Overall motivation
- Conceptual introduction and calculations
- Konfound software
- External validity
- Extension to logistic regression
Day 2
-
- Correlation framework: Impact Threshold for a Confounding Variable (ITCV)
- Conceptual introduction and calculations
- Benchmarking
- Multivariate
- Participants’ specific applications
- Special topics and FAQs
Day 1
-
- Robustness of Inference to Replacement (RIR)
- Overall motivation
- Conceptual introduction and calculations
- Konfound software
- External validity
- Extension to logistic regression
- Robustness of Inference to Replacement (RIR)
Day 2
-
- Correlation framework: Impact Threshold for a Confounding Variable (ITCV)
- Conceptual introduction and calculations
- Benchmarking
- Multivariate
- Participants’ specific applications
- Special topics and FAQs
- Correlation framework: Impact Threshold for a Confounding Variable (ITCV)
Related readings
*Frank, K. A., *Lin, Q., *Maroulis, S., *Mueller, A. S., Xu, R., Rosenberg, J. M., … & Zhang, L. 2021. Hypothetical case replacement can be used to quantify the robustness of trial results. Journal of Clinical Epidemiology, 134, 150-159. *authors listed alphabetically.
Frank, K.A., Maroulis, S., Duong, M., and Kelcey, B. 2013. What would it take to Change an Inference?: Using Rubin’s Causal Model to Interpret the Robustness of Causal Inferences. Education, Evaluation and Policy Analysis. Vol 35: 437-460.
Frank, K. A. 2000. “Impact of a Confounding Variable on a Regression Coefficient.” Sociological Methods and Research, 29(2), 147-194.
*Frank, K. A. and *Min, K. 2007. Indices of Robustness for Sample Representation. Sociological Methodology. Vol 37, 349-392. *authors listed alphabetically.
Xu, R., Frank, K. A., Maroulis, S. J., & Rosenberg, J. M. 2019. konfound: Command to quantify robustness of causal inferences. The Stata Journal, 19(3), 523-550.
*Frank, K. A., *Lin, Q., *Maroulis, S., *Mueller, A. S., Xu, R., Rosenberg, J. M., … & Zhang, L. 2021. Hypothetical case replacement can be used to quantify the robustness of trial results. Journal of Clinical Epidemiology, 134, 150-159. *authors listed alphabetically.
Frank, K.A., Maroulis, S., Duong, M., and Kelcey, B. 2013. What would it take to Change an Inference?: Using Rubin’s Causal Model to Interpret the Robustness of Causal Inferences. Education, Evaluation and Policy Analysis. Vol 35: 437-460.
Frank, K. A. 2000. “Impact of a Confounding Variable on a Regression Coefficient.” Sociological Methods and Research, 29(2), 147-194.
*Frank, K. A. and *Min, K. 2007. Indices of Robustness for Sample Representation. Sociological Methodology. Vol 37, 349-392. *authors listed alphabetically.
Xu, R., Frank, K. A., Maroulis, S. J., & Rosenberg, J. M. 2019. konfound: Command to quantify robustness of causal inferences. The Stata Journal, 19(3), 523-550.
Payment information
The fee of $595 includes all course materials.
PayPal and all major credit cards are accepted.
Our Tax ID number is 26-4576270.
The fee of $595 includes all course materials.
PayPal and all major credit cards are accepted.
Our Tax ID number is 26-4576270.