Comparing Student Models in Different Formalisms by Predicting their Impact on Help Success

Sébastien Lalle 1, 2, 3 Jack Mostow 3 Vanda Luengo 4 Nathalie Guin 1
1 SILEX - Supporting Interaction and Learning by Experience
LIRIS - Laboratoire d'InfoRmatique en Image et Systèmes d'information
4 MeTAH
LIG - Laboratoire d'Informatique de Grenoble
Abstract : We describe a method to evaluate how student models affect ITS decision quality - their raison d'être. Given logs of randomized tutorial decisions and ensuing student performance, we train a classifier to predict tutor decision outcomes (success or failure) based on situation features, such as student and task. We define a decision policy that selects whichever tutor action the trained classifier predicts in the current situation is likeliest to lead to a successful outcome. The ideal but costly way to evaluate such a policy is to implement it in the tutor and collect new data, which may require months of tutor use by hundreds of students. Instead, we use historical data to simulate a policy by extrapolating its effects from the subset of randomized decisions that happened to follow the policy. We then compare policies based on alternative student models by their simulated impact on the success rate of tutorial decisions. We test the method on data logged by Project LISTEN's Reading Tutor, which chooses randomly which type of help to give on a word. We report the cross-validated accuracy of predictions based on four types of student models, and compare the resulting policies' expected success and coverage. The method provides a utility-relevant metric to compare student models expressed in different formalisms.
Type de document :
Communication dans un congrès
H. Chad Lane, Kalina Yacef, Jack Mostow, Philip Pavlik. AIED 2013 - 16th International Conference on Artificial Intelligence in Education, Jul 2013, Memphis, TN, United States. Springer, 7926, pp.161-170, 2013, Lecture Notes in Computer Science (LNCS). 〈10.1007/978-3-642-39112-5_17〉
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-00871563
Contributeur : Denis Bouhineau <>
Soumis le : mardi 15 octobre 2013 - 15:44:08
Dernière modification le : jeudi 19 avril 2018 - 14:38:05

Lien texte intégral

Identifiants

Citation

Sébastien Lalle, Jack Mostow, Vanda Luengo, Nathalie Guin. Comparing Student Models in Different Formalisms by Predicting their Impact on Help Success. H. Chad Lane, Kalina Yacef, Jack Mostow, Philip Pavlik. AIED 2013 - 16th International Conference on Artificial Intelligence in Education, Jul 2013, Memphis, TN, United States. Springer, 7926, pp.161-170, 2013, Lecture Notes in Computer Science (LNCS). 〈10.1007/978-3-642-39112-5_17〉. 〈hal-00871563〉

Partager

Métriques

Consultations de la notice

414