Effect of Tuned Parameters on a LSA Multiple Choice Questions Answering Model

Alain Lifchitz (1) , Sandra Jhean-Larose (2) , Guy Denhière (2)

1 MALIRE - Machine Learning and Information Retrieval
2 LPC - Laboratoire de psychologie cognitive

This paper presents the current state of a work in progress, whose objective is to better understand the effects of factors that significantly influence the performance of Latent Semantic Analysis (LSA). A difficult task, which consists in answering (French) biology Multiple Choice Questions, is used to test the semantic properties of the truncated singular space and to study the relative influence of main parameters. A dedicated software has been designed to fine tune the LSA semantic space for the Multiple Choice Questions task. With optimal parameters, the performances of our simple model are quite surprisingly equal or superior to those of 7th and 8th grades students. This indicates that semantic spaces were quite good despite their low dimensions and the small sizes of training data sets. Besides, we present an original entropy global weighting of answers' terms of each question of the Multiple Choice Questions which was necessary to achieve the model's success.

Keywords: MCQ Answering Model; Semantic similarity; Latent Semantic Analysis (LSA); Vector Space Model Dimensionality; Unsupervised Learning; LSA software

Domains: Machine Learning [cs.LG]; Artificial Intelligence [cs.AI]; Psychology; Machine Learning [stat.ML]

Alain Lifchitz, Sandra Jhean-Larose, Guy Denhière. Effect of Tuned Parameters on a LSA Multiple Choice Questions Answering Model. Behavior Research Methods, 2009, 41 (4), pp.1201--1209. ⟨10.3758/BRM.41.4.1201⟩. ⟨hal-00336126v3⟩