Methodological proposal for analyzing critical thinking test scores as a tool for development in higher education

Nubia Yaneth Gómez-Velasco ,Ana Emilce Jiménez-González ,Myriam Esther Ortiz-Padilla

doi:10.24294/jipd10089

Home - JIPD - Volume 8 - Issue 15

Submit to JIPD

Journal Browser

Volume | Year

Issue

• Current lssue

View All

News and Announcements

View All

Methodological proposal for analyzing critical thinking test scores as a tool for development in higher education

Nubia Yaneth Gómez-Velasco

Ana Emilce Jiménez-González

Myriam Esther Ortiz-Padilla

Journal of Infrastructure Policy and Development 2024, 8(15), 10089; https://doi.org/10.24294/jipd10089

Submitted：05 Nov 2024

Accepted：14 Nov 2024

Published：09 Dec 2024

Download PDF(557.01KB)

XML

Abstract

The development of critical thinking (CT) enhances academic and professional opportunities. A review of literature reveals the use of fragmented analysis techniques, such as descriptive and correlational methods, among others, which hinder a deeper understanding of CT levels. This research aims to develop a methodology for analyzing Critical Thinking test scores, integrating five phases: exploratory, item analysis, scoring, gap analysis, and correlational. Using a quantitative approach, CT skills were analyzed with the Halpern Critical Thinking Assessment, which includes both open- and closed-ended questions to measure five skills: Verbal Reasoning (VR), Argument Analysis (AA), Hypothesis Testing (HT), Probability Use (PU), and Problem Solving (PS). The sample consisted of 214 students aged 18 and older. The item analysis phase categorized the items into quadrants: satisfactory, for review, or for elimination, based on difficulty and discrimination indices. The gap analysis revealed that Verbal Reasoning and open-ended formats were less satisfactory. The correlational phase, using heat maps, showed a stronger association between Verbal Reasoning and Probability Use. The methodological contributions include a variety of strategies that provide recommended procedures for analyzing tests or questionnaires in general. In today’s digital age, the development of critical thinking is not only a desirable skill but an essential necessity for the higher education system.

Keywords

References

Acosta Meza, D., Atencia Andrade, A., García Medina, M. A. & Rodríguez Sandoval, M. (2020) ) ‘Identificación del pensamiento crítico en estudiantes universitarios de segundo semestre de la Corporación Universitaria del Caribe (CECAR)’ (Spanish) , Revista Electrónica Interuniversitaria de Formación del Profesorado, 23(3). doi: https://doi.org/10.6018/reifop.435831.

Adegoke, B. A. (2013) ‘Comparison of item statistics of physics achievement test using classical test and item response theory frameworks’, Journal of Education and Practice, 4(22), pp. 87-96. Available at: https://www.semanticscholar.org/paper/Comparison-of-Item-Statistics-of-Physics-Test-using-Adegoke/731a206284fad1d2522b02337b6c5409a8e91bb5 (Accessed: 17 July 2023).

Aji, M. P., Negoro, R. A., Rusilowati, A. & Subali, B. (2023) ‘Development of Waves Critical Thinking Test: Physics Essay Test for High School Students’, Development Of Waves Critical Thinking Test: Physics Essay Test For High School Students. Available at: https://www.eu-jer.com/development-of-waves-critical-thinking-test-physics-essay-test-for-high-school-student (Accessed: 17 July 2023).

Balatova, K., Nepras, K., Kovarik, P., Kubiatko, M. & Sustekova, E. (2022) ‘The Influence of Selected Variables on University Students’ Critical Thinking Level: Preliminary Results’. doi: 10.26907/esd.17.4.03.

Barra, M., Lopez, M., Moreno, E. & Uyaguari, F. (2021) ‘Developing critical thinking in the classroom: testimonies from excellent Ecuadorian teachers’(Spanish), Areté. Digital Journal of the Doctorate in Education of the Central University of Venezuela, 8(15), pp. 161–180. doi: https://doi.org/10.55560/ARETE.2022.15.8.8.

Barreiro, M. P. R., Barreiro, J. R., Bravo, K. L. M., Colamarco, I. L., Velásquez, B. I. H. & Rivadeneira, L. (2021) ‘Critical thinking and its assessment in university education’(Spanish), Research, Society and Development, 10(3), pp. e51910313748-e51910313748. doi: https://doi.org/10.33448/rsd-v10i3.13748.

Beichner, R. & Ding, L. (2009) ‘Approaches to data analysis of multiple-choice questions’, Physical Review Special Topics-Physics Education Research, 5(2), 020103. doi: https://doi.org/10.1103/PhysRevSTPER.5.020103.

Bertea, E., & Zait, A. (2013). Scale Validity In Exploratory Stages Of Research. Management and Marketing Journal, 38-46.

Cangalaya, L. (2020) ‘Critical thinking skills in university students through research’(Spanish), From the South Magazine,12(1), pp. 141-153. doi: https://doi.org/10.21142/DES-1201-2020-0009.

Cabezas, J. A. A., Cepeda, M. P. & Cordova, R. M. Z. (2017) ‘Critical thinking applied to scientific research’ (Spanish), Atlante Magazine: Education and Development Notebooks. Available at: http://www.eumed.net/rev/atlante/2017/02/investigacion.html (Accessed: 17 July 2023).

Christensen, R., Knezek, G., Smits, A., Tondeur, J. & Voogt, J. (2023) ‘Strategies for developing digital competencies in teachers: Towards a multidimensional Synthesis of Qualitative Data (SQD) survey instrument’, Computers & Education, 193, 104674. doi: https://doi.org/10.1016/j.compedu.2022.104674.

Cioban, M. & Mihaela, H. (2022) ‘Impactul tehnologiilor educaționale moderne în formarea competenței matematice la elevii din învățământul profesional tehnic postsecundar nonterțiar (viitori învățători)’, Universitatea de stat din Tiraspol universitatea pedagogică de stat „Ion Creangă”. Available at: http://www.cnaa.md/files/theses/2022/58564/mihaela_hajdeu_thesis.pdf (Accessed: 17 July 2023).

Costa, C., Hart, C., D’Souza, D., Kimpton, A. & Ljbusic, J. (2021) ‘Exploring higher education students’ critical thinking skills through content analysis’, Thinking Skills And Creativity, 41, 100877. doi: https://doi.org/10.1016/j.tsc.2021.100877.

Covaleda, I., Fayos, L., Murillo-Ligorred, V. & Ramos-Vallecillo, N. (2023) ‘Knowledge, Integration and Scope of Deepfakes in Arts Education: The Development of Critical Thinking in Postgraduate Students in Primary Education and Master’s Degree in Secondary Education’, Education Sciences, 13(11), 1073. doi: https://doi.org/10.3390/educsci13111073.

Deborah, A. C., Peter, A. O. & Temitope, B. (2020) ‘Comparism of Item Difficulty and Discrimination of Pre And Post University Entrance Examinations in Nigeria’, The Universal Academic Research Journal, 3(1), pp. 1-9. Available at: https://dergipark.org.tr/en/pub/tuara/issue/62346/937932 (Accessed: 17 July 2023).

Díaz-Larenas, C. H., Martín, L. S., Nelly, G., Ossa-Cornejo, C. J., Palma-Luengo, M. R. & Quintana-Abello, I. M. (2017) ‘Analysis of instruments for measuring critical thinking’, Psychological Sciences, 11(1), pp. 19-28. doi: https://doi.org/10.22235/cp.v11i2.1343.

Ehido, A., Awang, Z., Halim, B., & Ibeabuchi, C. (2020). DEVELOPING ITEMS FOR MEASURING QUALITY OF WORK LIFE AMONG MALAYSIAN ACADEMICS: AN EXPLORATORY FACTOR ANALYSIS PROCEDURE. Humanities & Social Sciences Reviews. https://doi.org/10.18510/hssr.2020.83132

Elder, L. & Paul, R. (2005) Critical Thinking Competency Standards: Standards, principles, performance, indicators, and outcomes, with a master rubric on critical thinking.(Spanish) Available at: https://www.criticalthinking.org (Accessed: 17 July 2023).

Elreda, L. M. & Kohler, E. A. (2023) ‘EdTech Context Inventory: Factor analyses for ten instruments to measure edtech implementation context features’, Computers & Education, 195, 104709. doi: https://doi.org/10.1016/j.compedu.2022.104709.

Facione, P. A. (2011) Critical thinking: What it is and why it counts, Insight Assessment. Available at: https://insightassessment.com/wp-content/uploads/ia/pdf/whatwhy2018.pdf (Accessed: 17 July 2023).

Gamboa Suárez, A. A., Montes Miranda, A. & Rosales Yepes, A. (2023) ‘Effective leadership for educational quality and university accreditation in the Colombian Caribbean’(Spanish), Interuniversity Journal of Teacher Training, 98(37.1). doi: https://doi.org/10.47553/rifop.v98i37.1.98213.

George Reyes, C. E. & Glasserman Morales, L. D. (2021) ‘Technology-mediated research competencies: a systematic mapping of the literature’ (Spanish), Education in the Knowledge Society (EKS), 22, e23897. doi: https://doi.org/10.14201/eks.23897.

Gómez Velasco, N. Y., Jiménez González, A. E., & Rodríguez Gutiérrez, J. K. (Eds.). (2023). Elementos de bibliometría. Fundamentos y aplicaciones. Publicaciones Universidad de América. https://doi.org/10.29097/9786289517651 (Original work published 11 de abril de 2023)

Gómez, N. Y. & Jiménez, G. A. E. (2015) ‘Statistics as a support for university-community research projects. Reflections on an experience with research incubators’ (Spanish), Logos, Science & Technology Magazine, 7(1), pp. 27-34. doi: http://dx.doi.org/10.22335/rlct.v7i1.210.

Gómez, N. Y., Jiménez, A. E., Guerrero, S. C. & Ayala, Y. (2014) ‘Analysis of Colombian scientific production in chemistry. WoK database (2001-2012)’ (Spanish), Logos Ciencia & Tecnología Journal, 6(1), pp. 108-115.

Gutiérrez, C. F. V. (2013) ‘An interdisciplinary reflection on critical thinking’, Latin American Journal of Educational Studies (Spanish),9(2), pp. 11-39. Available at: https://www.redalyc.org/articulo.oa?id=134135724002 (Accessed: 17 July 2023).

Halpern, D. F. (2006) Halpern critical thinking assessment using everyday situations: Background and scoring standards (2nd Report), Claremont McKenna College. Available at: http://www.scielo.org.co/scielo.php?script=sci_nlinks&ref=000116&pid=S0121-3814201400020000600015&lng=en (Accessed: 17 July 2023).

Halpern, D. F. (2014) Thought and knowledge: An introduction to critical thinking (5th ed.), Psychology Press.

Halpern, D. F. (2016) Manual Halpern Critical Thinking Assessment, Schuhfried GmbH, Mödling.

Hassan,S. (2016) ‘item analysis, reliability statistics and standard error of measurement to improve the quality and impact of multiple-choice questions in undergraduate medical education in faculty of medicin at UNISZA’, Malaysian Journal of Public Health Medicine, pp.7-15. Available at: http://wprim.whocc.org.cn/admin/article/articleDetail?WPRIMID=626840&articleId=626840 (Accessed: 17 July 2023).

Hernández Sampieri, R., Fernández Collado, C. & Baptista Lucio, P. (2018) Research methodology (4th ed.), México: McGraw-Hill Interamericana.

Kaczkó, É. & Ostendorf, A. (2023) ‘Critical thinking in the community of inquiry framework: An analysis of the theoretical model and cognitive presence coding schemes’, Computers & Education, 193, 104662. doi: https://doi.org/10.1016/j.compedu.2022.104662.

Lun, V. M. C., Yeung, J. C. & Ku, K. Y. L. (2023) ‘Effects of mood on critical thinking’, Thinking Skills and Creativity, 47, 101247. doi: https://doi.org/10.1016/j.tsc.2023.101247.

Manassero-Mas, M. A. & Vázquez-Alonso, Á. (2020) ‘Assessing Critical Thinking Skills: Validation of Culture-Free Instruments’ (Spanish), Tecné, Episteme y Didaxis: TED, (47), pp. 15-32. doi: https://doi.org/10.17227/ted.num47-9801.

Moreno-Pinado, W. E. & Tejeda, M. E. V. (2017) ‘Teaching strategy to develop critical thinking’(Spanish) , REICE. Ibero-American Journal on Quality, Effectiveness and Change in Education, 15(2), pp. 53-73. Available at: https://www.redalyc.org/articulo.oa?id=55150357003 (Accessed: 17 July 2023).

Rivas, S. F. & Saiz, C. (2012) ‘Validation and psychometric properties of the PENCRISAL critical thinking test’ (Spanish), REMA Electronic Journal of Applied Methodology, 17(1), pp. 18-34. Available at: http://hdl.handle.net/10366/157532 (Accessed: 17 July 2023).

Severín, E. (2011) Technologies for Education (TEd). A Framework for Action, Inter-American Development Bank.

Sughra, U. & Usmani, A. (2022) ‘Comparison of Critical Thinking among undergraduate medical students of Conventional and Integrated curricula in Twin Cities’, Pakistan Journal of Medical Sciences, 38(6), pp. 1453. doi: 10.12669/pjms.38.6.5409.

Tjoe, H. & de la Torre, J. (2014) ‘The identification and validation process of proportional reasoning attributes: An application of a cognitive diagnosis modeling framework’, Mathematics Education Research Journal, 26

Previous
article in this issue

Next
article in this issue