Benefits of using the eye-tracking method for qualitative observation of students’ multiplechoice physics tasks solution process
Bd.29,Nr.4(2019)
ENGLISH ISSUE
As previous studies have shown, the eye-tracking method is useful for monitoring cognitive processes. In this study, we focus on the benefits of analysing high school students’ problem-solving processes when completing a multiple-choice test. Specifically, we provide a qualitative analysis of eye-movements of two low prior knowledge students as they solve two physics tasks. Both students scored zero on the test and chose the same options. In a paper-pencil administration of such a test, we would receive the same results from both of them. Via analysis of the eye-tracking data, we have shown that there are several similarities and differences between their solving approaches. Particularly, they both emphasized the key terms in the stem, meaning they did not provide incorrect answers due to improper stem reading. Both students showed very typical misconceptions concerning understanding Newton’s First Law. However, they differ in their confidence. Eve arrived at her answer quicker, but Jane needed to reread the stem many times. We observed atypical saccadic eye movements when Eve read the stems of the tasks and much longer fixations when she suddenly switched to her answers. Jane showed “graph as a picture error” when interpreting an illustrative picture in the stem. Based on our analysis, we demonstrate that summarized data on areas of interest (AOIs) can be misleading when AOIs are too large (e.g., each test option). We also introduce several indicators appropriate for a qualitative analysis, and we present new visualisations for in-word and between-words saccades.
eye-tracking; physics education; regressive saccades; multiple-choice test; problem-solving
Bax, S. (2013). Readers’ cognitive processes during IELTS reading tests: Evidence from eye tracking. ELT Research Papers 13-06. London: British Council. ISBN 978-0-86355-720-0
Beichner, R., J. (1994). Testing student interpretation of kinematics graphs. American Journal of Physics, 62(8), 750–762. https://doi.org/10.1119/1.17449
Blascheck, T., Raschke, M., & Ertl, T. (2013). Circular heat map transition diagram. In Proceedings of the 2013 Conference on Eye Tracking South Africa (pp. 58–61). New York: ACM. https://doi.org/10.1145/2509315.2509326
Blascheck, T., Kurzhals, K., Raschke, M., Burch, M., Weiskopf, D., & Ertl. T. (2014). State-of-the-art of visualization for eye tracking data. In R. Borgo, R. R. Maciejewski, & I. Viola (Eds.), Proceedings of Eurographics Conference on Visualization (EuroVis).
Bojko, A. (2013). Eye tracking the user experience: A practical guide to research. New York: Rosenfeld Media.
Dimigen, O., Sommer, W., Hohlfeld, A., Jacobs, A., & Kliegl, R. (2011). Coregistration of eye movements and eeg in natural reading: Analyses and review. Journal of Experimental Psychology: General, 140 (4), 552–572. https://doi.org/10.1037/a0023885
Dunn, R., Dunn, K., & Price, G. E. (1989). Learning style inventory (LSI): Price Systems, Incorporated (PO Box 1818, Lawrence 66044).
Gegenfurtner, A., Lehtinen, E., & Saljö, R. (2011). Expertise differences in the comprehension of visualizations: A meta-analysis of eye-tracking research in professional domains. Educational Psychology Review, 23(4), 523–552. https://doi.org/10.1007/s10648-011-9174-7
Hestenes, D., Wells, M., & Swackhamer, G. (1992). Force Concept Inventory. The Physics Teacher, 30(3), 141–158. https://doi.org/10.1119/1.2343497
HyperMind project. Websites. Available from https://uedu.uni-kl.de/arbeitsfelder/ arbeitsfeld-01/hypermind/english
Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., & Van Dee Weijer, J. (2011). Eye tracking. Oxford: Oxford university press.
Kekule, M., & Viiri, J. (2018). Students’ approaches to solving R-FCI tasks observed by eye-tracking method. Scientia in educatione, 9(2), 117–130. https://doi.org/10.14712/18047106.1010
Kekule, M., & Krejčí, A. (2019, in print). Žákovské čtení textu z učebnice sledované oční kamerou a role otázek při porozumění tomuto textu. In Moderní trendy v přípravě učitelů fyziky 2019.
Kliegl, R., Nuthmann, A., & Engbert, R. (2006). Tracking the mind during reading: The influence of past, present, and future words on fixation durations. Journal of Experimental Psychology: General, 35(1), 12. https://doi.org/10.1037/0096-3445.135.1.12
Kozhevnikov, M., Motes, M., & Hegarthy, M. (2007). Spatial visualization in physics problem solving. Cognitive Science, 31(4), 549–579. https://doi.org/10.1080/15326900701399897
Kübler, T., Sippel, K., Fuhl, W., Schievelbein, G., Aufreiter, J., Rosenberg, R., Rosenstiel, W., & Kasneci, E. (2015). Analysis of eye movements with Eyetrace, In International Joint Conference on Biomedical Engineering Systems and Technologies (pp. 458–471). Cham: Springer. https://doi.org/10.1007/978-3-319-27707-3_28
Kunze, K., Kawaichi, H., Yoshimura, K., & Kise, K. (2013a). Towards inferring language expertise using eye tracking. In CHI’13 Extended Abstracts on Human Factors in Computing Systems (pp. 217–222). https://doi.org/10.1145/2468356.2468396
Kunze, K., Kawaichi, H., Yoshimura, K., & Kise, K. (2013b). The Wordometer -- estimating the number of words read using document image retrieval and mobile eye tracking. In 12th International Conference on Document Analysis and Recognition, IEEE, Washington, DC, USA. https://doi.org/10.1109/ICDAR.2013.14
Lai, M.-L., Tsai, M.-J., Yang, F.-Y., Hsu, C.-Y., Liu, T.-C., Lee, S. W.-Y., …. Tsai, C. C. (2013). A review of using eye-tracking technology in exploring learning from 2000 to 2012. Educational Research Review, 10, 90–115. https://doi.org/10.1016/j.edurev.2013.10.001
Lindner, M. A., Eitel, A., Thoma, G.-B., Dalehefte, I. M., Ihme, J. M., & Köller, O. (2014). Tracking the decision-making process in multiple-choice assessment: Evidence from eye movements. Applied Cognitive Psychology, 28(5), 738–752. https://doi.org/10.1002/acp.3060
Madsen, A. M., Larson, A. M., Loschky, L. C., & Sanjay Rebello, N. (2012). Difference in visual attention between those who correctly and incorrectly answer physics problems. Physical Review Special Topics-Physics Education Research, 8(1), 010122. https://doi.org/10.1103/PhysRevSTPER.8.010122
Mareš, J., & Slavík, V. (1990). Dotazník stylu učení. Praha: Institut pedagogicko-psychologického poradenství České republiky.
Nieminen, P., Savinainen, A., & Viiri, J. (2010). Force Concept Inventory-based multiple-choice test for investigating students’ representational consistency. Physical Review Special Topics – Physics Education Research, 6, 020109. https://doi.org/10.1103/PhysRevSTPER.6.020109
Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124(3), 372–422. https://doi.org/10.1037/0033-2909.124.3.372
Rayner, K. & Pollatsek, A. (1989). The psychology of reading. Englewood Cliffs: Prentice Hall.
Reichle, E., Warren, T., & McConnell, K. (2009). Using E-Z reader to model the effects of higher-level language processing on eye movements during reading. Psychonomic Bulletin & Review, 16(1), 1–20. https://doi.org/10.3758/PBR.16.1.1
Šmideková, Z. (2018). Eye-tracking v prostredí školskej triedy, Pedagogika, 68(1), 25–50. https://doi.org/10.14712/10.14712/23362189.2017.773
Tsai, M.-J., Hou, H.-T., Lai, M.-L., Liu, W.-Y., & Yang, F.-Y. (2012). Visual attention for solving multiple-choice science problem: An eye-tracking analysis. Computers & Education, 58, 375–385.
https://doi.org/10.1016/j.compedu.2011.07.012
Tai, R. H., Loehr, J. F., & Brigham, F. J. (2006). An exploration of the use of eye-gaze tracking to study problem-solving on standardized science assessments. International Journal of Research & Method in Education, 29(2), 185–208. https://doi.org/10.1080/17437270600891614
Viiri, J., Kekule, M., Isoniemi J., & Hautala, J. (2017). Eye-tracking the effect of representation on students’ problem solving approaches. In Proceedings of the annual symposium of the FMSERA 2016 (88–98). The Finnish Mathematics and Science Education Research Association.
Voßkühler, A., Nordmeier, V., Kuchinke, L., & Jacobs, A.M. (2008). OGAMA – OpenGazeAndMouseAnalyzer: Open source software designed to analyze eye and mouse movements in slideshow study designs. Behavior Research Methods, 40(4), 1150–1162. https://doi.org/10.3758/BRM.40.4.1150
Dieses Werk steht unter der Lizenz Creative Commons Namensnennung - Nicht-kommerziell - Keine Bearbeitungen 4.0 International.
Copyright (c) 2021 Martina Kekule, Iva Spanova, Jouni Viiri