Scale Construction and Development - Online Course
A 4-Day Livestream Seminar Taught by
Deborah Bandalos10:30am-12:30pm (convert to your local time)
1:30pm-3:00pm
Multiple-item scales designed to measure attitudes, opinions, personality, and other attributes are ubiquitous in today’s world, and are widely used in making hiring decisions, assessing student, customer, and employee satisfaction, conducting needs assessments and program evaluations, and in research projects. Those involved in such activities often have little knowledge of how to effectively develop and evaluate the scales they need. This knowledge is crucial because data obtained from these scales are only as good as the scales themselves. Scales that are not well developed often yield data that are not usable for the intended purpose.
This workshop is designed to give you the concepts and tools to develop attitude, personality, opinion, or other noncognitive scales for any of the purposes just described.
Starting August 15, we are offering this seminar as a 4-day synchronous*, livestream workshop held via the free video-conferencing software Zoom. Each day will consist of two lecture sessions which include hands-on exercises, separated by a 1-hour break. You are encouraged to join the lecture live, but will have the opportunity to view the recorded session later in the day if you are unable to attend at the scheduled time.
*We understand that finding time to participate in livestream courses can be difficult. If you prefer, you may take all or part of the course asynchronously. The video recordings will be made available within 24 hours of each session and will be accessible for four weeks after the seminar, meaning that you will get all of the class content and discussions even if you cannot participate synchronously.
Closed captioning is available for all live and recorded sessions. Live captions can be translated to a variety of languages including Spanish, Korean, and Italian. For more information, click here.
More details about the course content
First, we will cover current theory and best practices in scale construction. We will begin by discussing how to create an item pool. We will then review current research on:
- The impact of vaguely worded or negatively worded items on scale reliability and validity.
- The optimal length of a survey.
- How many scale points to include.
- Whether scale points should be labeled or unlabeled.
- Whether to include a neutral option.
- How item order effects may impact responses.
Next, we will cover the use of exploratory factor analysis (EFA) in the scale development and revision process. We will focus on basic EFA analysis and the interpretation of model parameters, with an emphasis on best practices in using EFA-based information to inform scale development. We will also discuss common issues in EFA, such as method factors, weak factors, and highly correlated factors. There will be several examples with real data sets, using SPSS, SAS, and Mplus for the analysis.
Finally, we will introduce confirmatory factor analysis (CFA) and its use in scale development and the revision process. We will discuss estimation of CFA models and interpretation of CFA model parameters. There will be a particular emphasis on reasons for CFA model misfit, including issues with redundant and similarly worded items, cross-loading items, and method effects. We will also discuss the relation between model fit and the homogeneity of item intercorrelations. These methods will be illustrated with the Mplus program. Sample code will be provided for analyses that include:
- CFA models.
- Bifactor models.
- Calculation of coefficient omega.
- Tests of item parallelism.
First, we will cover current theory and best practices in scale construction. We will begin by discussing how to create an item pool. We will then review current research on:
- The impact of vaguely worded or negatively worded items on scale reliability and validity.
- The optimal length of a survey.
- How many scale points to include.
- Whether scale points should be labeled or unlabeled.
- Whether to include a neutral option.
- How item order effects may impact responses.
Next, we will cover the use of exploratory factor analysis (EFA) in the scale development and revision process. We will focus on basic EFA analysis and the interpretation of model parameters, with an emphasis on best practices in using EFA-based information to inform scale development. We will also discuss common issues in EFA, such as method factors, weak factors, and highly correlated factors. There will be several examples with real data sets, using SPSS, SAS, and Mplus for the analysis.
Finally, we will introduce confirmatory factor analysis (CFA) and its use in scale development and the revision process. We will discuss estimation of CFA models and interpretation of CFA model parameters. There will be a particular emphasis on reasons for CFA model misfit, including issues with redundant and similarly worded items, cross-loading items, and method effects. We will also discuss the relation between model fit and the homogeneity of item intercorrelations. These methods will be illustrated with the Mplus program. Sample code will be provided for analyses that include:
- CFA models.
- Bifactor models.
- Calculation of coefficient omega.
- Tests of item parallelism.
Computing
SPSS and SAS will be used for scale construction examples. SPSS, SAS, and Mplus will be used for Exploratory Factor Analysis (EFA) examples. Mplus will be used for all confirmatory factor analysis (CFA) examples. Prior knowledge of these programs is not essential. During the seminar participants are welcome to use a computer with any of these packages installed, although this is not required. Syntax and output for all examples, with comprehensive explanatory annotation, will be provided in the materials.
There is now a free version of SAS, called SAS OnDemand for Academics, that is available to anyone.
If you’d like to familiarize yourself with Mplus basics before the seminar begins, we recommend reading through UCLA’s short guide here.
SPSS and SAS will be used for scale construction examples. SPSS, SAS, and Mplus will be used for Exploratory Factor Analysis (EFA) examples. Mplus will be used for all confirmatory factor analysis (CFA) examples. Prior knowledge of these programs is not essential. During the seminar participants are welcome to use a computer with any of these packages installed, although this is not required. Syntax and output for all examples, with comprehensive explanatory annotation, will be provided in the materials.
There is now a free version of SAS, called SAS OnDemand for Academics, that is available to anyone.
If you’d like to familiarize yourself with Mplus basics before the seminar begins, we recommend reading through UCLA’s short guide here.
Who should register?
This seminar is designed for researchers interested in developing attitude, personality, opinion, or other noncognitive scales for use in research studies, needs assessments, program evaluations, or other purposes. Participants should be familiar with the basic principles of measurement theory, such as reliability and validity. Participants should also be familiar with basic statistics such as correlations, descriptive statistics, and introductory inferential statistics. No prior knowledge of EFA or CFA is required, although a basic knowledge of these methods will be helpful.
This seminar is designed for researchers interested in developing attitude, personality, opinion, or other noncognitive scales for use in research studies, needs assessments, program evaluations, or other purposes. Participants should be familiar with the basic principles of measurement theory, such as reliability and validity. Participants should also be familiar with basic statistics such as correlations, descriptive statistics, and introductory inferential statistics. No prior knowledge of EFA or CFA is required, although a basic knowledge of these methods will be helpful.
Seminar outline
Day 1: Item Writing
-
- Item writing background
- Why do we need scales?
- Basic item writing principles
- Theories about response processes
- Item responses as social encounters
- Response effects due to item characteristics
- Negative keying
- Vague wording
- Response effects due to item characteristics
- Order effects
- Question vs statement phrasing
Day 2: Item Writing/ Use of EFA in Scale Development
-
- Response effects due to response options
- Including a neutral option
- Number of scale points
- Option labeling
- Overview of item writing exercise
- EFA
- The EFA/CFA distinction
- Foundational concepts
- The EFA model
Day 3: Use of EFA in Scale Development
-
- Review of item writing exercise
- Estimation for EFA
- Factor extraction
- Communalities and eigenvalues
- Factor rotation
- Statistical assumptions of EFA
- Use of EFA in scale development and revision
- Reasons for cross-loadings
- Overview of EFA Exercise
Day 4: Use of CFA in Scale Development
-
- Review of EFA exercise
- Conceptual foundations
- The CFA model
- CFA Model identification
- Estimation and testing
- Residuals and modification indices
- Reasons for model lack of fit
- Proportionality constraints
- Meaning/wording similarity
- Method effects
- Distributional artifacts
- Order effects
- Need for more or fewer factors
Day 1: Item Writing
-
- Item writing background
- Why do we need scales?
- Basic item writing principles
- Theories about response processes
- Item responses as social encounters
- Response effects due to item characteristics
- Negative keying
- Vague wording
- Response effects due to item characteristics
- Order effects
- Question vs statement phrasing
- Item writing background
Day 2: Item Writing/ Use of EFA in Scale Development
-
- Response effects due to response options
- Including a neutral option
- Number of scale points
- Option labeling
- Overview of item writing exercise
- EFA
- The EFA/CFA distinction
- Foundational concepts
- The EFA model
Day 3: Use of EFA in Scale Development
-
- Review of item writing exercise
- Estimation for EFA
- Factor extraction
- Communalities and eigenvalues
- Factor rotation
- Statistical assumptions of EFA
- Use of EFA in scale development and revision
- Reasons for cross-loadings
- Overview of EFA Exercise
Day 4: Use of CFA in Scale Development
-
- Review of EFA exercise
- Conceptual foundations
- The CFA model
- CFA Model identification
- Estimation and testing
- Residuals and modification indices
- Reasons for model lack of fit
- Proportionality constraints
- Meaning/wording similarity
- Method effects
- Distributional artifacts
- Order effects
- Need for more or fewer factors
Payment information
The fee of $995 includes all course materials.
PayPal and all major credit cards are accepted.
Our Tax ID number is 26-4576270.
The fee of $995 includes all course materials.
PayPal and all major credit cards are accepted.
Our Tax ID number is 26-4576270.