pyKT: A Python Library to Benchmark Deep Learning based Knowledge Tracing Models
Nov 28, 2022
Speakers
Qiongqiong Liu
Speaker · 0 followers
Jiahao Chen
Speaker · 4 followers
Shuyan Huang
Speaker · 0 followers
Jiliang Tang
Speaker · 0 followers
About
Knowledge tracing (KT) is the task of using students' historical learning interaction data to model their knowledge mastery over time so as to make predictions on their future interaction performance. Recently, remarkable progress has been made of using various deep learning techniques to solve the KT problem. However, the success behind deep learning based knowledge tracing (DLKT) approaches is still left somewhat unknown and proper measurement and analysis of these DLKT approaches remain a challenge. First, data preprocessing procedures in existing works are often private and custom, which limits experimental standardization. Furthermore, existing DLKT studies often differ in terms of the evaluation protocol and are far away real-world educational contexts. To address these problems, we introduce a comprehensive python based benchmark platform, pyKT, to guarantee valid comparisons across DLKT methods via thorough evaluations. The pyKT library consists of a standardized set of integrated data preprocessing procedures on 7 popular datasets across different domains, and 10 frequently compared DLKT model implementations for transparent experiments. Results from our fine-grained and rigorous empirical KT studies yield a set of observations and suggestions for effective DLKT, e.g., wrong evaluation setting may cause label leakage that generally leads to performance inflation; and the improvement of many DLKT approaches is minimal compared to the very first DLKT model proposed by Piech et al. <cit.>. We have open sourced pyKT and our experimental results at <https://pykt.org/>. We welcome contributions from other research groups and practitioners.Knowledge tracing (KT) is the task of using students' historical learning interaction data to model their knowledge mastery over time so as to make predictions on their future interaction performance. Recently, remarkable progress has been made of using various deep learning techniques to solve the KT problem. However, the success behind deep learning based knowledge tracing (DLKT) approaches is still left somewhat unknown and proper measurement and analysis of these DLKT approaches remain a ch…
Organizer
NeurIPS 2022
Account · 954 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
ContextNER: Contextual Phrase Generation at Scale
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Zeroth-Order Negative Curvature Finding: Escaping Saddle Points without Gradients
Watch later
Hualin Zhang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Learning General World Models in a Handful of Reward-Free Deployments
Watch later
Yingchen Xu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
End-to-end Symbolic Regression with Transformers
Watch later
Total of 1 viewers voted for saving the presentation to eternal vault which is 0.1%
Lost in Latent Space: Examining failures of disentangled models at combinatorial generalisation
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Smooth Fictitious Play in Stochastic Games with Perturbed Payoffs and Unknown Transitions
Watch later
Lucas Baudin, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%