A psychometric perspective on personalized assessment in education

Education has long grappled with the intricacies of providing truly personalized assessment. This dissertation addresses the limitations of conventional assessment methods, especially the widely employed linear tests. A variety of models and methods are introduced to enhance the efficiency, validity, and reliability of assessments. A robust procedure to create fair fixed-length tests, utilizing randomly selected items from an item bank, is delineated, emphasizing guidelines on item setup and procedures to adjust scores based on mean test difficulty. This approach incorporates traditional statistical measures, such as Cronbach’s alpha and the Spearman-Brown prediction formula, to ensure the reliability of tests using random item selections. Secondly, a model for computerized adaptive practice, inspired by the Elo rating system, is introduced. This model, exemplified by the Maths Garden system, updates ability and item difficulty estimations in real-time. It factors in both response time and accuracy in its scoring rule, with children's performance data substantiating its reliability, validity, and user satisfaction. Furthermore, this newly proposed scoring rule, is investigated for its performance across various domains. Findings revealed its strong reliability and validity without discernible individual performance biases. Lastly, in the realm of Elo-based adaptive systems, the importance of item quality is highlighted. Through simulations, methods to determine item discrimination were compared, with the logistic model inclusive of the intercept emerging as the most effective. This method's effectiveness was further affirmed using real-world data from the aforementioned Maths Garden system.