Evaluating the content structure of intelligent tutor systems : a psychological network analysis

Abstract

The adoption of intelligent tutoring systems (ITSs) worldwide has led to a considerable accumulation of process data as students interact with different learning topics within these systems. Typically, these learning topics are structured within ITSs (e.g., the fraction topic includes subtopics such as a fraction number line subtopic). However, there is a lack of methods that offer quick, data-driven insights into the content structure of ITSs, particularly through easily accessible visualizations. Here, we applied psychological network analysis to process data (230,241 students; 5,365,932 problem sets) from an ITS for learning mathematics to explore performance interdependencies between 40 different subtopics. We argue that the visualization of these content interdependencies allows for a quick empirical evaluation of the validity of the existing structuring of the respective learning content. These insights allow for deriving recommendations concerning potential changes in the ITS structure and are thus highly valuable for ITS developers. Our results are also relevant for researchers as the interdependencies illustrated through psychological network analysis can contribute towards a better understanding of the interplay between mathematical skills. Together, our results indicate that psychological network analysis represents a valuable data-driven method to evaluate and optimize ITSs.