Overestimation of reliability by Guttman’s λ₄, λ₅, and λ₆, and the greatest lower bound

For methods using statistical optimization to estimate lower bounds to test-score reliability, we investigated the degree to which they overestimate true reliability. Optimization methods do not only exploit real relationships between items but also tend to capitalize on sampling error and do this more strongly as sample size is smaller and tests are longer. The optimization methods were Guttman’s λ₄, λ₅, and λ₆ and the greatest lower bound to the reliability (GLB). Method λ₂ was used as benchmark. We used a simulation study to investigate the relation of the methods’ discrepancy, bias, and sampling error with the proportion of simulated data sets in which each method overestimated true test-score reliability. Method λ₄ and the GLB often overestimated test-score reliability. When sample size exceeded 250 observations, methods λ₂, λ₅, and λ₆ provided reasonable to good statistical results, in particular when data were two-dimensional. Benchmark method λ₂ produced the best results.