UCLA

Cognitive diagnosis models (see, e.g., Rupp, Templin, & Henson, 2010) have received increasing attention within educational and psychological measurement. The popularity of these models may be largely due to their perceived ability to provide useful information concerning both examinees (classifying them according to their attribute profiles) and test items (describing the particular attributes that are relevant to or required in order to achieve a certain response). However, the validity of such information may be undermined when diagnostic models are misspecified.

This research focuses on one aspect of model misspecification: violations of the local item independence assumption. Potential causes of dependence are examined, with a particular focus on those causes unrelated to the attributes a diagnostic test is intended to measure. Ignoring such dependencies, as is the standard practice in fitting traditional diagnostic models, may lead to biased estimates of model parameters and misclassification of examinees.

An alternative to traditional diagnostic models is presented, in which random effects are included in order to account for these nuisance dependencies. This approach is already well-established in item factor analysis, serving as the basis for the testlet response model (Wainer, Bradlow, & Wang, 2007), random intercept item factor model (Maydeu-Olivares & Coffman, 2006), item bifactor model (Cai, Yang, & Hansen, 2011), and two-tier item factor model (Cai, 2010), among others.

The resulting hierarchical diagnostic item response model maintains the desirable properties of traditional diagnostic models (e.g., the classification of examinees with respect to fine-grained cognitive attributes), while allowing for greater complexity in the underlying response process. Importantly, the model may be estimated efficiently — even for models with a large number of nuisance variables — using an analytical dimension reduction technique described by Gibbons and Hedeker (1992).