Towards an Empirically Validated Model for Assessment of Code Quality

We present a pilot study into developing a model of feedback on code quality in introductory programming courses. To devise such a model, we analyzed professional standards of code quality embedded in three popular software engineering handbooks and found 401 suggestions that we categorized into twenty topics. We recorded three instructors who performed a think-aloud judgment of student-submitted programs, and we interviewed them on the topics from the books, leading to 178 statements about code quality. The statements from the instructor interviews allowed us to generate a set of topics relevant to their practice of giving feedback, which we used to create criteria for the model. We used the instructor statements as well as the book suggestions to distinguish three levels of achievement for each criterion. This resulted in a total of 9 criteria for code quality. The interviews with the instructors generated a view of code quality that is very comparable to what was found in the handbooks, while the handbooks provide detailed suggestions that make our results richer than previously published grading schemes. As such, this process leads to an overview of code quality criteria and levels that can be very useful for constructing a standards-based rubric for introductory programming courses.