Posterior predictive model checks for cognitive diagnostic models

Cognitive diagnostic models (CDMs; DiBello et al., 2007) have received increasing attention in educational measurement for the purpose of diagnosing examinees' strengths and weaknesses of their latent attributes. Despite the current popularity of a number of diagnostic models, research on assessing model-data fit has been limited. The current study applies one of the Bayesian model checking methods, namely the posterior predictive model check (PPMC) method (Rubin, 1984) to investigate model misfit. Specifically, we aim to employ the technique to investigate model-data misfit from various diagnostic models using real data and a simulation study. An important issue with the application of PPMC is the choice of discrepancy measure. This study examines the performance of three discrepancy measures for assessing different aspects of model fit: observed total-scores distribution, association of item pairs, and correlation of attribute pairs as adequate measures for the diagnostic models.