A red button does it all : a red button nudges students to close their knowledge gaps in an intelligent tutoring system

Abstract

Background Intelligent tutoring systems (ITSs) often adaptively provide students with additional learning materials to close knowledge gaps automatically recognized by the system. In addition, students may be nudged with notification badges to close their knowledge gaps. Aim This study evaluated the effectiveness of implementing a notification badge (small red button) reminding students to close their knowledge gaps. We also investigated the association between closing knowledge gaps and performance while controlling for other relevant factors (e.g., usage behavior, system features, and teachers’ assignment types). Sample We considered 105,081 students (in 6,668 classes working on ∼25 million problems) who used an ITS for learning mathematics. Methods We employed hierarchical linear regression analysis to evaluate whether students were more likely to close their knowledge gaps in a period in which notification badges were implemented in the ITS, as compared to a period without notification badges. Moreover, we used psychological network analysis and hierarchical linear regression analysis to evaluate whether closing knowledge gaps was associated with students’ performance relative to other relevant factors. Results Students closed 28 % of their knowledge gaps when the red button notification badge nudged them, but only 5 % when there was no red button. Closing knowledge gaps predicted better student performance and was a more important predictor than students' usage behavior, other system features, and teachers’ assignment types. Conclusions Our study highlights the relevance of nudges within ITS for closing knowledge gaps and shows that closing knowledge gaps is linked to students’ performance.