Random permutation tests of nonuniform differential item functioning in multigroup item factor analysis

The purpose of the present research was to introduce and evaluate random permutation testing applied to measurement invariance testing with ordered-categorical data. The random permutation test builds a reference distribution from the observed data that is used to calculate a p value for the observed (Δ)χ² statistic. The reference distribution is built by repeatedly shuffling the grouping variable and then saving the Δχ² statistic between the two models fitted to the resulting data. The present research consisted of two Monte Carlo simulations. The first simulation was designed to evaluate random permutation testing across a variety of conditions with scalar invariance testing in comparison to an existing analytical solution: the robust mean- and variance-adjusted Δχ² test. The second simulation was designed to evaluate the random permutation test applied to testing configural invariance by evaluating overall model fit (the χ² fit statistic). Simulation results and suggestions for the use of the random permutation test are provided.