Using corpora in language learning: A meta-analysis of effectiveness and efficiency
Résumé
Corpora and associated tools represent a powerful, empirical research methodology that spreads beyond the discipline of corpus linguistics. They have long been used to inform language teaching and materials (Johns, 1986) and are now used increasingly in the learning process itself. However, given the wide range of teaching/learning contexts, learner profiles, languages, corpora, software, approaches, experimental designs, etc., it can be difficult to make sense of the state of research. The traditional literature review retains a high degree of subjectivity - selecting papers to review, deciding on their relative importance, interpreting results, and producing generalisations. In contrast, a meta-analysis is as comprehensive and objective as possible. This paper presents a meta-analysis of empirical research evaluating an effect of corpus use in language learning, broadly following approaches laid out in Norris and Ortega (2000) and Spada and Tomita (2010). An initial collection of 132 papers is reduced to 21 which meet inclusion criteria for this first version meta-analysis. Preliminary results from pre/post-test designs suggest a mean gain effect of 1.68 standard deviations (d = .84); experimental/control group results are predictably somewhat lower at 1.04 (d = .73); both fall comfortably within 95% confidence limits and can be characterised as "strong" even by the exacting standards proposed by Oswald and Plonsky (2010). These results suggest not only that corpus use can be effective (within groups) but also efficient (between groups) compared to a range of conventional practices. It should be stressed that considerably more work is still required at all stages of this research (collecting, coding, collating, analysis, etc.). We expect to develop these and other aspects into a fully-fledged meta-analysis for the AILA seminar in 2014, describing limitations and nuancing the findings, focusing on a range of variables, contextualising our study against other meta-analyses in applied linguistics, and suggesting avenues in need of future exploration.