Sample Size Justification - Online Course
A 3-Day Livestream Seminar Taught by
Daniel LakensWednesday, March 19 –
Friday, March 21, 2025
10:00am-12:30pm (convert to your local time)
1:30pm-3:30pm
Data collection is often costly and time-consuming. It is essential, then, to carefully consider how much data you will need to collect in order to answer your research questions. In this workshop, we will learn how to determine how much data are needed to achieve: (i) accurate estimates, (ii) informative tests of hypotheses, and (iii) optimally efficient decision making, based on the available data. Starting from the pragmatic understanding that resources are limited, throughout the workshop we will strongly focus on how to answer research questions as efficiently as possible.
We will learn how to justify the sample size to test for the presence, as well as the absence, of meaningful effect sizes, how to plan for accurate estimates, the difference between an a-priori, compromise, or sensitivity power analysis, and how to control error rates when making decisions based on limited amounts of data.
Starting March 19, we are offering this seminar as a 3-day synchronous*, livestream workshop held via the free video-conferencing software Zoom. Each day will consist of two lecture sessions which include hands-on exercises, separated by a 1-hour break. You are encouraged to join the lecture live, but will have the opportunity to view the recorded session later in the day if you are unable to attend at the scheduled time.
*We understand that finding time to participate in livestream courses can be difficult. If you prefer, you may take all or part of the course asynchronously. The video recordings will be made available within 24 hours of each session and will be accessible for four weeks after the seminar, meaning that you will get all of the class content and discussions even if you cannot participate synchronously.
Closed captioning is available for all live and recorded sessions. Captions can be translated to a variety of languages including Spanish, Korean, and Italian. For more information, click here.
More details about the course content
Collecting insufficient data means you will be unable to answer your research questions. On the other hand, collecting too much data wastes resources. Simply put, determining how much data to collect is made difficult by a number of uncertainties. An important aspect of this workshop is to provide you with the tools to make informed decisions about how much data you need to collect in the face of these uncertainties. These decisions are guided by a principled sample size justification that you can communicate to reviewers of manuscripts and grant proposals, or to managers to justify the costs of data collection.
In other instances, you have no control over the amount to data that is available–for example, analyses of existing datasets, when the data collection has been determined by someone else, or because resource limitations dictate how much data can be collected. In these situations, you will need to evaluate which questions you can answer with the available data. We will extensively discuss these common scenarios and provide you with tools to decide whether or not to move forward with a research project given the available data.
You will learn common heuristics used to determine how much data to collect, and the instructor will review best practices.
Through hands-on exercises, this seminar will review a wide range of free tools to determine required sample size.
Collecting insufficient data means you will be unable to answer your research questions. On the other hand, collecting too much data wastes resources. Simply put, determining how much data to collect is made difficult by a number of uncertainties. An important aspect of this workshop is to provide you with the tools to make informed decisions about how much data you need to collect in the face of these uncertainties. These decisions are guided by a principled sample size justification that you can communicate to reviewers of manuscripts and grant proposals, or to managers to justify the costs of data collection.
In other instances, you have no control over the amount to data that is available–for example, analyses of existing datasets, when the data collection has been determined by someone else, or because resource limitations dictate how much data can be collected. In these situations, you will need to evaluate which questions you can answer with the available data. We will extensively discuss these common scenarios and provide you with tools to decide whether or not to move forward with a research project given the available data.
You will learn common heuristics used to determine how much data to collect, and the instructor will review best practices.
Through hands-on exercises, this seminar will review a wide range of free tools to determine required sample size.
Computing
This seminar will use R for the empirical examples and exercises. To participate in the hands-on exercises, you are strongly encouraged to use a computer with the most recent version of R and RStudio installed. RStudio is a front-end for R that makes it easier to work with. This software is free and available for Windows, Mac, and Linux platforms.
A basic understanding of how to run code in R is useful, although most R-based tools that we will use have user-friendly online Shiny apps. Code for all exercises will be provided.
If you’d like to take this course but are concerned that you don’t know enough R, there are excellent online resources for learning the basics. Here are our recommendations.
This seminar will use R for the empirical examples and exercises. To participate in the hands-on exercises, you are strongly encouraged to use a computer with the most recent version of R and RStudio installed. RStudio is a front-end for R that makes it easier to work with. This software is free and available for Windows, Mac, and Linux platforms.
A basic understanding of how to run code in R is useful, although most R-based tools that we will use have user-friendly online Shiny apps. Code for all exercises will be provided.
If you’d like to take this course but are concerned that you don’t know enough R, there are excellent online resources for learning the basics. Here are our recommendations.
Who should register?
This seminar is designed for researchers and data scientists who are involved in experimental design and/or specification of research questions. The strong emphasis on dealing with resource limitations makes this course especially suitable for research projects where data collection has financial, time, or sample size limitations.
This is a beginner-level course. You should preferably have experience with, and be proficient at, analyzing data in your field of work, and have some experience in the design of studies. You should have a good working knowledge of basic principles of statistical inference (e.g., standard errors, hypothesis tests, confidence intervals), and should also have a solid understanding of the basic theory and practice of linear regression.
This seminar is designed for researchers and data scientists who are involved in experimental design and/or specification of research questions. The strong emphasis on dealing with resource limitations makes this course especially suitable for research projects where data collection has financial, time, or sample size limitations.
This is a beginner-level course. You should preferably have experience with, and be proficient at, analyzing data in your field of work, and have some experience in the design of studies. You should have a good working knowledge of basic principles of statistical inference (e.g., standard errors, hypothesis tests, confidence intervals), and should also have a solid understanding of the basic theory and practice of linear regression.
Seminar outline
- Introduction to different approaches to justifying sample size
- Collecting data from the whole population
- Planning for accuracy
- A-priori power analysis
- Planning based on cost-benefit analysis
- The problem with heuristics
- Effect size
- Minimal detectable effect sizes
- Smallest effect size of interest
- Expected effect sizes
- Effect sizes in the literature and dealing with bias
- Power analysis
- Type 1 and Type 2 error control
- A-priori, compromise, and sensitivity power analysis
- The problem of post-hoc or retrospective power
- Power analysis through simulation studies
- Best practices in power analysis
- Matching power analysis to performed tests
- The desired level of statistical power
- Expecting mixed results
- Power analysis for complex designs
- Problems with retrospective power analysis
- Planning for accuracy
- Random variation in small samples
- The relation between sample size and accuracy
- Planning for a desired uncertainty around estimates
- Specifying a smallest effect size of interest
- Why establishing a smallest effect size of interest is best practice
- Smallest effect of interest based on cost-benefit analysis
- Planning for the absence of effects
- Introduction to equivalence testing
- The importance of establishing the absence of an effect
- Performing and interpreting equivalence tests
- Introduction to different approaches to justifying sample size
- Collecting data from the whole population
- Planning for accuracy
- A-priori power analysis
- Planning based on cost-benefit analysis
- The problem with heuristics
- Effect size
- Minimal detectable effect sizes
- Smallest effect size of interest
- Expected effect sizes
- Effect sizes in the literature and dealing with bias
- Power analysis
- Type 1 and Type 2 error control
- A-priori, compromise, and sensitivity power analysis
- The problem of post-hoc or retrospective power
- Power analysis through simulation studies
- Best practices in power analysis
- Matching power analysis to performed tests
- The desired level of statistical power
- Expecting mixed results
- Power analysis for complex designs
- Problems with retrospective power analysis
- Planning for accuracy
- Random variation in small samples
- The relation between sample size and accuracy
- Planning for a desired uncertainty around estimates
- Specifying a smallest effect size of interest
- Why establishing a smallest effect size of interest is best practice
- Smallest effect of interest based on cost-benefit analysis
- Planning for the absence of effects
- Introduction to equivalence testing
- The importance of establishing the absence of an effect
- Performing and interpreting equivalence tests
Payment information
The fee of $995 includes all course materials.
PayPal and all major credit cards are accepted.
Our Tax ID number is 26-4576270.
The fee of $995 includes all course materials.
PayPal and all major credit cards are accepted.
Our Tax ID number is 26-4576270.