The authors evaluated several ways of measuring item difficulty. First, they used the FSAIF1toF3 dataset, whose publisher supplied difficulty levels for each question, to assess how well Deep-IRT's estimated difficulty matched these labels. Second, as an alternative, difficulty was measured from how many students answered each question correctly versus incorrectly, counting only each student's first attempt and only questions with at least 10 respondents. Third, they used PFA together with each student's full learning trajectory to estimate difficulty, restricting this analysis to a subset of five skills. The authors suggest it would be worthwhile to test whether Deep-IRT's difficulty estimates are more accurate than these traditional approaches.

Going Deeper in Difficulty Level (Deep-IRT: Make Deep Learning Based Knowledge Tracing Explainable Using Item Response Theory)

The authors examined whether Deep-IRT's student-ability predictions exhibit two problems reported for DKT: decreases in predicted performance after a correct response and inconsistent predictions across knowledge components over time. Deep-IRT produced smooth prediction transitions, although it still generated several irrelevant predictions.

Deep-IRT Student-Ability Predictions Compared with DKT

1. Going Deeper in Difficulty Level
2. Going Deeper in Student Ability

San Diego State University

1. Introduction
2. Literature Review
3. Deep Item Response Theory
4. Experiments
5. Discussion
6. Conclusion

Deep-IRT: Make Deep Learning Based Knowledge Tracing Explainable Using Item Response Theory

1. Paper:  https://arxiv.org/pdf/1904.11738v1.pdf
2. Github: https://github.com/ckyeungac/DeepIRT

Reference for Deep-IRT: Make Deep Learning Based Knowledge Tracing Explainable Using Item Response Theory

Knowledge tracing is very important to improve personalized learning. "Knowledge tracing is the task of modeling student’s knowledge state, which is a general representation of the mastery level of KCs, e.g., a scalar value representing a student ability level, or a vector representation similar to word embedding."
In knowledge tracing based on previous interactions next interaction is predicted.
Usually we have question answer pairs as an interaction data and we want to predict the probability of answering the question correctly. 
The knowledge tracing models have 2 groups:

1. Models having direct meaningful parameters (Bayesian Knowledge Tracing (BKT), Performance Factor Analysis (PFA)).
2. Complex and general-purpose models which don't have direct interpretations.(Deep Knowledge Tracing (DKT) , Deep Key Value Memory Networdk (DKVMN)).

These groups are non-overlapping and the authors decided to create a new model, called Deep-IRT (Deep-Item Response Theory) which would make deep learning based knowledge explainable.
The main contributions of this work (as written in the paper) are:

1. The proposed Deep-IRT knowledge tracing model is capable of inferring meaningful estimation of student ability and KCs’ difficulty level while  simultaneously retains the predictive power of the deep learning based knowledge tracing model.

2. The Deep-IRT model potentially provides an alternative way for estimating KC’s difficulty level by utilizing the entire learning trajectory, rather than the traditional educational testing environment.

3. We propose to use a deep learning model to output parameters of a  psychometric model so as to leverage the deep learning capability and provide explainable psychometric parameters. This idea can be applied elsewhere apart from the knowledge tracing task.


Introduction (Deep-IRT: Make Deep Learning Based Knowledge Tracing Explainable Using Item Response Theory)

1. Item Response Theory
2. Knowledge Tracing

Literature Review (Deep-IRT: Make Deep Learning Based Knowledge Tracing Explainable Using Item Response Theory)

1. General
2. Working Mechanism of DKVMN
3. Student Ability and Difficulty Networks

Learn Before

Related

Learn After