UC Irvine

Predicting learning and human behavior in general is a challenging endeavor. Machine learning driven predictive modeling have been an increasingly popular means to understand disparities in student performance. With more than a handful of approaches to predictive modeling, the current literature of predicting learning is plagued with issues such as lack of standards for predictions, lack of work to understand the lower and upper bounds to context specific predictions, and explanatory models perceived to be at odds with predictive models. To overcome these issues, I use a single predictive modeling framework across three different learning contexts that involve four key steps: using approaches that are prediction task specific while reporting all metrics; using a baseline model for comparison; using context-specific early learning to predict later learning; systematically introducing extrinsic feature sets to derive actionable insights. In my first study, I use this framework to predict learning in a working memory training context. Results suggest that later learning can be predicted from early learning behavior better when using extrinsic features. In the second study, this framework is applied in the context of a blended learning environment. Results suggest that students’ study spacing intentions, demographics, and past achievements predict later learning, while students’ click behaviors in the learning environment do not improve predictions. In the third study, this framework is applied in the context of a fully online learning environment. Results show that students’ self-reported motivational, affective, and social-emotional data are more predictive of early learning than context-agnostic click-behaviors. Overall, the current work proposes and evaluates a framework that can be used to compare results across learning contexts, between models, and approaches to make predictions that may inform future prescriptions. Specifically, the framework acts as a means to understand the relationship shared by the three key aspects to making successful predictions – ‘how well’, ‘how soon’, and ‘how much information’ – and relevance of looking beyond simple accuracies.