Adapting multiple-choice comprehension question formats in a test of second language listening comprehension

Date
2021-01-23
Advisor
Supervisor
Co-Advisor
Co-Supervisor
Instructor
Source Title
Language Teaching Research
Print ISSN
1362-1688
Electronic ISSN
1477-0954
Publisher
Sage Publications
Volume
Issue
Pages
1 - 25
Language
English
Type
Article
Journal Title
Journal ISSN
Volume Title
Series
Abstract

The current study explores the impact of varying multiple-choice question preview and presentation formats in a test of second language listening proficiency targeting different levels of text comprehension. In a between-participant design, participants completed a 30-item test of listening comprehension featuring implicit and explicit information comprehension questions under one of four multiple-choice question preview and presentation conditions. Interactions between preview, presentation and comprehension in the participants’ test scores were analysed using many facet Rasch analysis. The results suggest that the measurement of participants’ listening ability was directly influenced by the presentation of multiple-choice questions. Test scores were highest when participants were able to preview the multiple-choice question stems before the sound file and listened to the options after the text had completed. However, interactions between preview and presentation conditions and comprehension level were only statistically significant in an analysis of the low scoring students’ item responses, which were more frequently correct when preview of item stems was available for questions targeting comprehension of implicit information. The research underscores the importance of accounting for test design when making inferences about language learners’ listening ability and will be of interest to teachers, practitioners and researchers developing listening assessment tasks.

Course
Other identifiers
Book Title
Keywords
English as a second language, Listening comprehension, Multiple-choice questions, Rasch measurement
Citation
Published Version (Please cite this version)