Designing a Multilingual Large-Scale Placement Test with a Formative Perspective: A Case Study at the University of Grenoble Alpes
Résumé
An interdisciplinary team composed of more than thirty people has been engaged in the process of conceiving, designing and validating an online placement test with formative orientation, called SELF (Système d’Evaluation en Langues à visée Formative). SELF is a large-scale assessment device, anchored on the CEFR descriptors and validated according to the ALTE cycle, with both quantitative (CTT – Classical Test Theory and IRT – Item Response Theory) and qualitative methods (questionnaires, interview and focus group), developed within the frame of the ANR IDEFI project Innovalangues . Nowadays, SELF has already placed around 90,000 students in six different languages (Italian, English, French, Spanish, Mandarin and Japanese).
Designing a multilingual test with these features is a very challenging and long process. The most challenging aspects to manage concerned i) the respect of the same communicative construct for the six different languages of the test; ii) the necessity to improve item writers’ skills in psychometrics and, more broadly, to spread a good-quality culture of evaluation; iii) the infrastructural and technical side; iv) the necessity to coordinate a large and heterogeneous team for a long time (6 years).
Facing these difficulties required the adoption of specific strategies, in order to reach the goal, such as: a very careful organization of the working team, composed of scientific manager, team coordinators and item-writers; the decision to start with two piloting languages, Italian and English, followed by the other four; the redaction and sharing of common documents to guarantee the interlinguistic transfer; the in-house conception of a multi-task platform serving as authoring tool, piloting and pretesting repository, large-scale administration device for tracking, saving and communicating final results.