Alternative interpretation of bi-level language proficiency test scores for listening and reading comprehension
Roč.15,č.1(2025)
The aim of this study was to take an alternative look at the language assessment of the listening and reading comprehension with the intention to provide language teachers at the authors’ institution with more valid and reliable feedback on their instruction. To achieve this, an alternative approach to the interpretation of test scores obtained from bi-level language proficiency tests assessing the listening and reading comprehension at the authors’ institution was analysed. This alternative approach was proposed by the authors for comparison with the currently used approach to test score interpretation at their institution. The intention of the proposed alternative approach was to increase the validity of the proficiency levels obtained based on listening and reading comprehension test scores. In this alternative approach, the authors attempted to eliminate the guessing factor employed by test takers, which may negatively affect the valid assessment of their listening and reading comprehension. To compare the alternative and the current approaches to the interpretation of test scores, proficiency levels of receptive skills (listening and reading) and productive skills (speaking and writing) were correlated with the assumption that the test score interpretation approach showing higher correlation is more valid. The rationale for this correlation was the fact that the authors considered proficiency levels of productive skills valid and reliable due to the manner in which they are tested at their institution and due to the fact that the proficiency levels of productive skills cannot be distorted by test takers’ guessing.
bi-level language proficiency test ; guessing factor; language assessment; test score interpretation; validation
Abbosov, A. A. (2023). Analysis of multilevel English language proficiency tests. Bulletin, 2023(1), 79-96.
Alderson, J. Ch. (2000). Assessing reading. Cambridge University Press.
Bachman, L. (1990). Fundamental considerations in language testing. Oxford University Press.
Bachman, L., & Palmer, A. (1996). Language testing in practice. Oxford University Press.
Bachman, L. (2002). Alternative interpretations of alternative assessments: Some validity issues in educational performance assessments. Educational Measurement: Issues and Practice, 21(3), 5–18.
Bachman, L. (2005). Building and supporting a case for test use. Language Assessment Quarterly, 2, 1–34.
Bachman, L., & Palmer, A. (2010). Language assessment in practice. Oxford University Press.
BILC. (2014). STANAG 6001. (Edition 5). [online]. Dostupné z: https://www.natobilc.org/files/file/6001EFed05.pdf
BILC. (2016). STANAG 6001 language proficiency levels. (Edition A, Version 2). [online]. Dostupné z: https://www.natobilc.org/files/ATrainP-5%20EDA%20V2%20E.pdf
BILC. (2019). Overview of language proficiency levels. (5th ed.). [online]. Dostupné z: https://www.natobilc.org/documents/TrainingResources/STANAG%206001%20Overview%20Feb%202019.pdf
Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2004). The concept of validity. Psychological Review, 111(4), 1061–1071.
Carr, N. (2011). Designing and analyzing language tests. Oxford University Press.
Cizek, G. J., & Bunch, M. (2006). Standard setting: a guide to establishing and evaluating performance standards on tests. SAGE Publications.
Cizek G. J. (2012). Defining and distinguishing validity: interpretations of score meaning and justifications of test use. Psychological Methods, 17(1), 31–43.
Clifford, R. (2016). A rationale for criterion-referenced proficiency testing. Foreign Language Annals, 49(2), 224–234.
Clifford, R., & Cox, T. (2013). Empirical validation of reading proficiency guidelines. Foreign Language Annals, 46(1), 45–61.
Cox, T., & Clifford, R. (2014). Empirical validation of listening proficiency guidelines. Foreign Language Annals, 47(3), 379–403.
Davis, A., Brown, A., Elder, C., Hill, K., Lumley, T., & McNamara, T. (1999). Dictionary of language testing. (1st ed.). Cambridge University Press.
Green, A. (2014). Exploring language assessment and testing. Routledge.
Hughes, A. (2013). Testing for language teachers. Cambridge University Press.
Kane, M. (2002). Validating high-stakes testing programs. Educational Measurement: Issues and Practice, 21(1), 31–41.
Kane, M. (2006). Validation. In R. L. Brennan (ed.), Education Measurement (4th ed.), 17–64, American Council on Education and Praeger Publishers.
Kane, M. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73.
Kostromitina, M. (2024). Is this normal? Parsing jagged profiles in language assessment. [online]. Dostupné z: https://blog.englishtest.duolingo.com/is-this-normal-parsing-jagged-profiles-in-language-assessment/
Lakshani, J. (2015). Correlational study – The role of receptive language skills in the acquisition of productive skills. [online]. Dostupné z: https://www.researchgate.net/publication/344188030_Correlational_Study_-The_Role_of_Receptive_Language_Skills_in_the_Acquisition_of_Productive_Skills
Ryan, K. (2002). Assessment validation in the context of high-stakes assessment. Educational Measurement: Issues and Practice, 21(1), 7–15.
Thompson, N. (2024). Modified-Angoff Method Study. [online]. Dostupné z: https://assess.com/conduct-a-modified-angoff-study/
University of Defence Language Centre. (2019). Manual for item development. [online]. Dostupné z: https://shorturl.at/1V2WV
University of Defence Language Centre. (2023). Test specifications. [online]. Dostupné z: https://shorturl.at/oH1gN
Xi, X. (2010). How do we go about investigating test fairness? Language Testing, 27(2), 147–170.
Yuzar, E., Rejeki, S. (2020). Correlation between productive and receptive language skills: An examination on ADFELPS test scores. SALEE: Study of Applied Linguistics and English Education, 1(2), 99–113.
Copyright © 2025 Pavel Svoboda, Lenka Marková