Scale Construction and Development - Online Course

First, we will cover current theory and best practices in scale construction. We will begin by discussing how to create an item pool. We will then review current research on:

• • The impact of vaguely worded or negatively worded items on scale reliability and validity.
  • The optimal length of a survey.
  • How many scale points to include.
  • Whether scale points should be labeled or unlabeled.
  • Whether to include a neutral option.
  • How item order effects may impact responses.

Next, we will cover the use of exploratory factor analysis (EFA) in the scale development and revision process. We will focus on basic EFA analysis and the interpretation of model parameters, with an emphasis on best practices in using EFA-based information to inform scale development. We will also discuss common issues in EFA, such as method factors, weak factors, and highly correlated factors. There will be several examples with real data sets, using SPSS, SAS, R (psych), and Mplus for the analysis. These include:

• • Factor analysis of dichotomously scored data.
  • Parallel analysis.
  • Sources of EFA model misfit.

Finally, we will introduce confirmatory factor analysis (CFA) and its use in scale development and the revision process. We will discuss estimation of CFA models and interpretation of CFA model parameters. There will be a particular emphasis on reasons for CFA model misfit, including issues with redundant and similarly worded items, cross-loading items, and method effects. We will also discuss the relation between model fit and the homogeneity of item intercorrelations. These methods will be illustrated with the Mplus and R (lavaan) programs. Sample code will be provided for analyses that include:

• • CFA models.
  • Bifactor models.
  • Calculation of coefficient omega.
  • Tests of item parallelism.

Examples in this seminar will use SPSS, SAS, R, and Mplus. SPSS, SAS, and R will be used for scale construction examples; SPSS, SAS, R, and Mplus will be used for exploratory factor analysis (EFA); and R and Mplus will be used for confirmatory factor analysis (CFA). Prior experience with these programs is not required.

You’re welcome to use a computer with any of these packages installed during the seminar, but this is not required. However, if you wish to complete the workshop exercises you will need to use one of the programs specified. Syntax and output for all examples and exercises, along with detailed explanatory annotations, will be provided in the materials.

Those using R should install the following packages before the course: psych, lavaan, tidyverse or tidyr, dplyr, rlang, e1071, and readxl (for reading in Excel files). The tidyverse package includes haven, which can be used to read SPSS or SAS files.

If you’d like to use R for this course but don’t yet have much experience with that package, here are some excellent online resources for building your R skills.

There is now a free version of SAS, called SAS OnDemand for Academics, that is available to anyone.

If you’d like to familiarize yourself with Mplus basics before the seminar begins, we recommend reading through UCLA’s short guide here.

This seminar is designed for researchers interested in developing attitude, personality, opinion, or other noncognitive scales for use in research studies, needs assessments, program evaluations, or other purposes. You should be familiar with the basic principles of measurement theory, such as reliability and validity. You should also be familiar with basic statistics such as correlations, descriptive statistics, and introductory inferential statistics. No prior knowledge of EFA or CFA is required, although a basic knowledge of these methods will be helpful.

For broader training in measurement theory, validity, and measurement bias, see our related seminar, Psychometrics.

Day 1

Item writing

• • Why we need scales
  • Basic item writing principles
  • Detecting problems with items
  • Theories about response processes
  • Item responses as social encounters

Response effects due to item characteristics

• • Negative keying
  • Vague wording
  • Order effects

Response effects due to response options

• • Including a neutral option
  • Number of scale points
  • Option labeling

Day 2

Exploratory factor analysis

• • EFA introduction
    • The EFA/CFA distinction
    • Foundational concepts
    • The EFA model
  • Estimation for EFA
  • Factor extraction
  • Communalities and eigenvalues
  • Determining the number of factors
  • Factor rotation
  • Statistical assumptions and data requirements
  • Use of EFA in scale development and revision
    • Reasons for cross-loadings
    • Reasons for obtaining fewer factors than expected
    • Reasons for obtaining more factors than expected

Day 3

Confirmatory factor analysis

• • Intro to confirmatory factor analysis
    • Conceptual foundations
    • The CFA model
    • CFA model identification
  • Estimation
  • Model fit and testing
  • Residuals and modification indices
  • Reasons for model lack of fit
    • Proportionality constraints
    • Meaning/wording similarity
    • Method effects
    • Distributional artifacts
    • Order effects
    • Need for more or fewer factors
  • Putting it all together – making a validity argument
  • Additional topics – time permitting
    • Coefficient omega
    • Bifactor models
    • Model comparison tests

The fee of $995 USD includes all course materials.

PayPal and all major credit cards are accepted.

Our Tax ID number is 26-4576270.