Planned Missing Data Designs in Educational Psychology Research

Although missing data are often viewed as a challenge for applied researchers, in fact missing data can be highly beneficial. Specifically, when the amount of missing data on specific variables is carefully controlled, a balance can be struck between statistical power and research costs. This article presents the issue of planned missing data by discussing specific designs (i.e., multiform designs, longitudinal wave-missing designs, and 2-method measurement designs), introducing the power and cost benefits of such scenarios to applied education and educational psychology researchers.

Within educational and psychological research, missing data seem to come with the territory. The study of learning within school settings, for example, a common focus of educational psychology research, might involve tightly designed randomized controlled trials to compare programs to facilitate learning. Also common are longitudinal investigations (possibly even within randomized controlled trials), designed to gauge whether, at what rate, and under what circumstances learning is occurring. In addition to their inferential threats arising from cohort/classroom effects and changes in context over time (e.g., students transitioning to new classrooms and/or to new school settings, such as from elementary to middle school), longitudinal designs present the very practical challenges that following individuals over time is costly and typically results in missing data. These missing data could arise for fairly simple reasons, such as school absences, but are more often part of a pattern of attrition resulting from families moving or perhaps from individual students requiring alternative educational settings more tailored to their specific learning needs. Suffice it to say, dealing with missing data, whether in cross-sectional or longitudinal designs, is a very real part of the research process, as can be the dread of having to do so.

Fortunately, with the advent of modern missing data handling methods, and the implementation of these modern methods in most statistical software, missing data are becoming far less troublesome. In particular, so long as the missingness patterns in data are not related to the missing values themselves, analyses can proceed without much trouble or fear of bias. In addition, although some patterns of missing data can result in dramatically reduced power, other patterns can largely avoid it. For example, if two variables are highly correlated, such as perceptions of academic efficacy and academic self-worth, and one is missing many observations, not much power will be lost for testing parameters associated with that variable; in contrast, if the variable with a high rate of missing data (e.g., teacher stress) is unrelated to other variables (e.g., peer norms for cooperation), much more power will be lost for testing its parameters (e.g., means, variances, regression slopes).

Planned missing data designs take advantage of these facts and impose random missingness in such a way that it does not introduce bias and it minimizes power loss. The result can be a dramatic reduction in cost, and even an increase in validity due to reducing participant burden. In this article, after a brief review of some foundations related to missing data, we present three planned missing data designs and how they could be implemented in educational research. The first type of design (multiform design) is presented in the greatest detail, laying the groundwork for the remaining two: accelerated longitudinal designs and two-method measurement designs.