A Systematic Review of the Effects of Automatic Scoring and Automatic Feedback in Educational Settings

Automatic scoring and feedback tools have become critical components of online learning proliferation. These tools range from multiple-choice questions to grading essays using machine learning (ML). Learning environments such as massive open online courses (MOOCs) would not be possible without them. The usage of this mechanism has brought many exciting areas of study, from the design of questions to the ML grading tools’ precision and accuracy. This paper analyzes the findings of 125 studies published in journals and proceedings between 2016 and 2020 on the usages of automatic scoring and feedback as a learning tool. This analysis gives an overview of the trends, challenges, and open questions in this research area. The results indicate that automatic scoring and feedback has many advantages. The most important benefits include enabling scaling the number of students without adding a proportional number of instructors, improving the student experience by reducing the time between submission grading and feedback, and removing bias in scoring. On the other hand, these technologies have some drawbacks. These include creating a disincentive to come up with innovative answers that do not match the expected one or that have not been used to prepare the problem, training the student to answer the question instead of learning the concepts, and the potential availability of the answers on the internet. Overall, each of these drawbacks presents an opportunity to look at ways to improve technologies to make use of these tools to provide a better learning experience to students

Full open access at: Guerra-Hahn, M., Baldiris-Navarro, S.M., De-La-Fuente-Valentin, L., & Burgos, D. (2021). A Systematic Review of the Effects of Automatic Scoring and Automatic Feedback in Educational Settings. IEEE Access, 9, pp. 108190-108198. DOI: https://doi.org/10.1109/ACCESS.2021.3100890