Automatically Scoring Dutch Summary Assignments on a Multiple-Points Scale

Abstract

Summary assignments have long been a significant component of Dutch language exams in secondary education, but human bias often affects the assigned scores. In this study, we aimed to address this issue by investigating the use of artificial intelligence (AI) for automatically scoring Dutch summary assignments on a multiple-score scale. Our research question focused on the potential application of AI in this context. We developed and compared various models, including a feature-based linear regression model, a similarity-based support vector machine (SVM), and a Long short-term memory (LSTM) model. Among these models, the LSTM demonstrated the best performance. Additionally, we conducted an expert analysis involving a Dutch test expert to assess the proximity of the model and human scorers to the scores that would be assigned by correctly using the answer key. The results showed similar performance between the model and human scorers, suggesting that automatic scoring models offer a promising approach for mitigating bias. Future research could explore the use of larger datasets and focus on enhancing the explainability of the models.