The Impact of Artificial Intelligence Literacy Education on Enhancing Critical Thinking among Male Secondary School Students

Document Type : Original Article

Authors

Norouz-Ali Farahmandnia ¹

Davoud Haseli ²

Ali Azimi-Vaghar ³

¹ MA Student in Knowledge and Information Science, Kharazmi University, Tehran, Iran.

² Assistant Professor, Knowledge and Information Science, Kharazmi University, Tehran, Iran.

³ Assistant Professor, Department of Knowledge and Information Science, Kharazmi University, Tehran, Iran

10.22067/infosci.2026.96855.1263

Abstract

Introduction: Given the importance of fostering critical thinking among students and the increasing role of artificial intelligence (AI) technologies in everyday life, conducting the present study has a dual necessity: on the one hand, critical thinking is recognized as one of the key skills of the 21st century for students' academic and social success; and on the other hand, the rapid spread of AI technologies requires that the younger generation be able to understand and evaluate these technologies with a critical and informed perspective. Accordingly, the present study examines the effect of AI literacy training on improving critical thinking skills of ninth-grade male students. The main focus of the present study is on measuring changes in critical thinking components (creativity, growth, and commitment) after an eight-session AI literacy training course. The findings of this study, although limited to the sample and the tools used, can provide initial evidence that shows the importance of including AI literacy education in curricula and help teachers and educational planners design effective strategies to strengthen students’ critical thinking.
Methodology: The methodology of this study was based on a quantitative approach and a real experimental design of pre-test and post-test with a control group. The statistical population consisted of 64 ninth-grade male students from Ferdowsi High School in Tehran, who were divided into two experimental and control groups using convenience sampling. To measure students’ critical thinking, the Ricketts Critical Thinking Test (2003) was used, which has 33 questions and three subscales of creativity, maturity, and commitment. In this study, the experimental group participated in an eight-session AI literacy education program. This training was designed and developed in an interactive and project-based manner, taking into account the cognitive level and learning abilities of secondary school students. Based on the curriculum development resources, this training included five articles on the proposed curriculum topics of AI for school students, especially in secondary school (Chiu, 2021; Chiu et al., 2021; Lee et al., 2021; Michaeli et al., 2022; Steinbauer et al., 2021). The content of eight sessions and 12 hours of training was as follows: 1. Introduction to the concept of AI and its role in today's life, 2. Data and its importance in AI, 3. Algorithms and machine learning, 4. Introduction to applications of AI in different fields, 5. Challenges and ethical issues of AI, 6 and 7, practical introduction to AI tools, and 8. Review, evaluation, and future of AI. At the beginning of the first session, the pre-test questionnaire was distributed among the experimental group members and at the same time, the control group also received and completed the same questionnaire. The control group did not receive any special intervention or training during the period. At the end of the eighth session, the post-test questionnaire was distributed again among the experimental group members and at the same time, the control group also completed the same questionnaire. The reliability of the questionnaire was confirmed with a Cronbach's alpha of 0.88. Descriptive and inferential statistics, including independent t-test and analysis of covariance, were used to analyze the data to examine the effect of training and control for the effects of the pre-test.
Findings: The results indicated that the AI literacy program had a statistically significant and substantial impact on enhancing students’ overall critical thinking. Notably, the intervention led to marked improvements in creativity (effect size = 0.825), maturity (0.760), and engagement (0.876), with an overall critical thinking effect size of 0.629. In most cases, pretest scores had minimal influence on posttest outcomes, underscoring the effectiveness of the educational intervention. The adjusted R² values ranged from 76.7% to 89.7%, reflecting a high level of variance explained by the model and confirming the strong impact of AI literacy education on students’ cognitive and attitudinal development.
Discussion and Conclusion: The results of the present study clearly showed that AI literacy education, especially in the form of a structured and multifaceted educational program, can have a significant impact on improving students' critical thinking. This effect was observed not only in the overall level of critical thinking, but also in its three important components: creativity, growth, and commitment, which were also significantly and with a high effect size. Such a finding indicates that AI education, if properly designed and implemented, can go beyond the transfer of technical knowledge and help develop students' cognitive, ethical, and social skills. The educational program used in this study, by combining theoretical concepts, practical activities, group discussions, and ethical reflections, was able to provide a context in which students not only become familiar with technology, but also become sensitive to its consequences in individual and social life. This multi-layered approach led students to think creatively, responsibly, and analytically in the learning process and to reconsider their attitudes toward new technologies. Also, the use of real AI tools in training sessions provided a tangible and motivating experience for students, which played an important role in increasing their commitment and interest in learning. Given the growing trend of AI penetration in all aspects of life, from education to employment and social communication, the results of this study can be used as a scientific basis for reviewing educational and training policies. Designing new curricula in which AI literacy is taught as a basic skill alongside critical thinking, problem-solving, and digital ethics skills can play an important role in preparing the next generation to live in a technological and complex society. This study showed that AI education is not only a technological necessity, but also an educational opportunity to cultivate informed, analytical, and responsible citizens.

Keywords

Artificial Intelligence Literacy

AI Literacy Education

Critical Thinking

Male Students

Secondary Education

---

References

References

Akcaoğlu, M. Ö., Mor, E., & Külekçi, E. (2023). The mediating role of metacognitive awareness in the relationship between critical thinking and self-regulation. Thinking Skills and Creativity, 47, 101187. https://doi.org/https://doi.org/10.1016/j.tsc.2022.101187

Almulla, M. A., & Al-Rahmi, W. M. (2023). Integrated social cognitive theory with learning input factors: The effects of problem-solving skills and critical thinking skills on learning performance sustainability. Sustainability, 15(5), 3978.

Aston, K. J. (2024). ‘Why is this hard, to have critical thinking?’Exploring the factors affecting critical thinking with international higher education students. Active Learning in Higher Education, 25(3), 537–550. https://doi.org/https://doi.org/10.1177/14697874231168341

Azizpour, M., & Azizpour, A. (2023). Using Artificial Intelligence Technology in Educational Content Production to Increase Students' Learning and Creativity Second International Conference on Educational Sciences, Counseling, Psychology and Social Sciences, Hamadan. [In Persian]. https://civilica.com/doc/1757166

Chiu, T. K. (2021). A holistic approach to the design of artificial intelligence (AI) education for K-12 schools. TechTrends, 65(5), 796–807.

Chiu, T. K., Meng, H., Chai, C.-S., King, I., Wong, S., & Yam, Y. (2021). Creation and evaluation of a pretertiary artificial intelligence (AI) curriculum. IEEE Transactions on Education, 65(1), 30–39. https://doi.org/https://doi.org/10.1109/TE.2021.3085878

Deldar, H., Razavi, M. R., & Faghih Nia, M. A. (2022). The effect of media literacy training on the use of social networks and improving the emotional maturity of senior high school students. Technology of Education Journal (TEJ), 16(2), 401–412. [In Persian]. https://doi.org/https://doi.org/10.22061/tej.2022.7753.2579

Elbyaly, M. Y. H., & Elfeky, A. I. M. (2023). The impact of problem-solving programs in developing critical thinking skills. European Chemical Bulletin, 12, 6636–6642. https://doi.org/https://orcid.org/0000-0001-9208-8537

Estevez, J., Garate, G., & Graña, M. (2019). Gentle introduction to artificial intelligence for high-school students using scratch. IEEE access, 7, 179027–179036.

Facione, P. A. (2011). Critical thinking: What it is and why it counts. Insight assessment, 1(1), 1–23. https://www.insightassessment.com/wp-content/uploads/ia/pdf/whatwhy.pdf

Faqih, N. (2023). Developing literacy and critical thinking with AI: What students say. Proceeding of Annual International Conference on Islamic Education (AICIED),

Fathahillah, F., Fakhri, M. M., & Ahmar, A. S. (2023). Analysis of artificial intelligence literacy in the blended learning model in higher education. EduLine: Journal of Education and Learning Innovation, 3(4), 566–575.

Funmi, K., & Qian, X. (2020). Critical thinking an antidote to artificial intelligence threat: An innovation in teacher education practices. Proc. Int. Conf. Educ,

Halonen, J. S. (1995). Demystifying critical thinking. Teaching of psychology, 22(1), 75–81. https://doi.org/https://doi.org/10.1207/s15328023top2201_23

Huang, X., & Qiao, C. (2024). Enhancing computational thinking skills through artificial intelligence education at a STEAM high school. Science & Education, 33(2), 383–403. https://doi.org/https://doi.org/10.1007/s11191-022-00389-5

Izadifard, R., & Sepasi-Ashtiani, M. (2010). The effectiveness of cognitive-behavioral therapy with problem-solving skills training in reducing symptoms of test anxiety. Journal of Behavioral Sciences, 4(1), 23–28. [In Persian].

Jia, X.-H., & Tu, J.-C. (2024). Towards a new conceptual model of AI-enhanced learning for college students: The roles of artificial intelligence capabilities, general self-efficacy, learning motivation, and critical thinking awareness. Systems, 12(3), 74. https://doi.org/https://doi.org/10.3390/systems12030074

Kajiwara, Y., Matsuoka, A., & Shinbo, F. (2023). Machine learning role playing game: Instructional design of AI education for age-appropriate in K-12 and beyond. Computers and Education: Artificial Intelligence, 5, 100162. https://doi.org/https://doi.org/10.1016/j.caeai.2023.100162

Lee, I., Ali, S., Zhang, H., DiPaola, D., & Breazeal, C. (2021). Developing middle school students' AI literacy. Proceedings of the 52nd ACM technical symposium on computer science education,

Mason, M. (2007). Critical thinking and learning. Educational philosophy and theory, 39(4), 339–349. https://doi.org/https://doi.org/10.1111/j.1469-5812.2007.00343.x

Michaeli, T., Romeike, R., & Seegerer, S. (2022). What students can learn about artificial intelligence–recommendations for K-12 computing education. IFIP World Conference on Computers in Education,

Moghadam, Z. B., Narafshan, M. H., & Tajadini, M. (2023). The effect of implementing a critical thinking intervention program on English language learners’ critical thinking, reading comprehension, and classroom climate. Asian-Pacific Journal of Second and Foreign Language Education, 8(1), 15. https://doi.org/https://doi.org/10.1186/s40862-023-00194-z

Muthmainnah, Ibna Seraj, P. M., & Oteir, I. (2022). Playing with AI to investigate human‐computer interaction technology and improving critical thinking skills to pursue 21st century age. Education Research International, 2022(1), 6468995. https://doi.org/https://doi.org/10.1155/2022/6468995

Nappi, C., & Cuocolo, A. (2020). The machine learning approach: Artificial intelligence is coming to support critical clinical thinking. Journal of Nuclear Cardiology, 27(1), 156–158. https://doi.org/https://doi.org/10.1007/s00330-020-06918-x

Ng, D. T. K., Su, J., Leung, J. K. L., & Chu, S. K. W. (2024). Artificial intelligence (AI) literacy education in secondary schools: a review. Interactive Learning Environments, 32(10), 6204–6224. https://doi.org/https://doi.org/10.1080/10494820.2023.2255228

pakmehr, H., mirdrogi, F., ghanaei, A., & karami, M. (2013). Reliability, Validity and Factor Analysis of Ricketts' Critical Thinking Disposition Scales in High School. Quarterly of Educational Measurement, 4(1), 33–54. [In Persian]. https://jem.atu.ac.ir/article_2678.html?lang=en

Panciroli, C., & Rivoltella, P. C. (2023). Can an algorithm be fair?: intercultural biases and critical thinking in generative artificial intelligence social uses. Scholé: rivista di educazione e studi culturali: LXI, 2, 2023, 67–84.

Paul, R., & Elder, L. (1992). Critical thinking: What, why, and how. New Directions for Community Colleges, 1992, 3–24. https://doi.org/https://doi.org/10.1002/cc.36819927703

Pettersson, H. (2023). From critical thinking to criticality and back again. Journal of Philosophy of Education, 57(2), 478–494. https://doi.org/https://doi.org/10.1111/1467-9752.12744

Pirouzfar, K., Azad, R., & Moalemi, S. (2024). Application of Artificial Intelligence in Teaching and Learning. International Conference on Humanities, Educational Sciences, Law and Social Sciences. [In Persian]  https://civilica.com/doc/1669151

Qureshi, M. A., Khaskheli, A., Qureshi, J. A., Raza, S. A., & Yousufi, S. Q. (2023). Factors affecting students’ learning performance through collaborative learning and engagement. Interactive Learning Environments, 31(4), 2371–2391. https://doi.org/https://doi.org/10.1080/10494820.2020.1799036

Rădoi, M.-C., Gabor, E., Oancea, M., & Saftiuc, B. (2025). Ethical Implications of Artificial Intelligence in Higher Education. SCIENTIA MORALITAS-International Journal of Multidisciplinary Research, 10(1), 288–298. https://scientiamoralitas.com/index.php/sm/article/view/313

Ramdani, A., Jufri, A., Gunawan, G., Fahrurrozi, M., & Yustiqvar, M. (2021). Analysis of students' critical thinking skills in terms of gender using science teaching materials based on the 5E learning cycle integrated with local wisdom. Jurnal Pendidikan IPA Indonesia, 10(2), 187–199. https://doi.org/https://doi.org/10.15294/jpii.v10i2.30053

Ricketts, J. C. (2003). Critical thinking skills of selected youth leaders: The efficacy of leadership development, critical thinking dispositions, and student academic performance University of Florida].

Rusandi, M. A., Ahman, Saripah, I., Khairun, D. Y., & Mutmainnah. (2023). No worries with ChatGPT: building bridges between artificial intelligence and education with critical thinking soft skills. Journal of Public Health, 45(3), e602–e603. https://doi.org/https://doi.org/10.1093/pubmed/fdad062

Saadat-Dorabi, F., & Ahmadzadeh Meymand, P. (2024). Modeling the effect of collaborative learning on critical thinking mediated by empathy in junior high school students (case study of Nir region). Journal of New Research Approaches in Management And Accounting, 8(1), 1032–1041. [In Persian].

Saeedavi, S., & Rafiei-Taghanaki, F. (2023). vestigating the role of artificial intelligence in students' learning and performance Third National Conference on Applied Ideas in Educational Sciences, Psychology and Cultural Studies, Bushehr. [In Persian]. https://civilica.com/doc/1833981

Sanusi, I. T., Oyelere, S. S., Vartiainen, H., Suhonen, J., & Tukiainen, M. (2023). Developing middle school students’ understanding of machine learning in an African school. Computers and Education: Artificial Intelligence, 5, 100155. https://doi.org/https://doi.org/10.1016/j.caeai.2023.100155

Shaw, A., Liu, O. L., Gu, L., Kardonova, E., Chirikov, I., Li, G., Hu, S., Yu, N., Ma, L., & Guo, F. (2020). Thinking critically about critical thinking: validating the Russian HEIghten® critical thinking assessment. Studies in Higher Education, 45(9), 1933–1948. https://doi.org/https://doi.org/10.1080/03075079.2019.1665327

Shin, S. (2021). A study on the framework design of artificial intelligence thinking for artificial intelligence education. International Journal of Information and Education Technology, 11(9), 392–397. https://doi.org/https://doi.org/10.18178/ijiet.2021.11.9.1541

Siegel, H. (1980). Critical thinking as an educational ideal. The Educational Forum,

Spector, J. M., & Ma, S. (2019). Inquiry and critical thinking skills for the next generation: from artificial intelligence back to human intelligence. Smart Learning Environments, 6(1), 1–11. https://doi.org/https://doi.org/10.1186/s40561-019-0093-6

Steinbauer, G., Kandlhofer, M., Chklovski, T., Heintz, F., & Koenig, S. (2021). A differentiated discussion about AI education K-12. KI-Künstliche Intelligenz, 35(2), 131–137. https://doi.org/https://doi.org/10.1007/s13218-021-00724-8

Stolpe, K., & Hallström, J. (2024). Artificial intelligence literacy for technology education. Computers and Education Open, 6, 100159. https://doi.org/https://doi.org/10.1016/j.caeo.2024.100159

Su, J., Ng, D. T. K., & Chu, S. K. W. (2023). Artificial intelligence (AI) literacy in early childhood education: The challenges and opportunities. Computers and Education: Artificial Intelligence, 4, 100124. https://doi.org/https://doi.org/10.1016/j.caeai.2023.100124

Tuhtaboevich, T. A. (2023). Developing critical thinking skills for university success in the age of digital technology. JournalNX, 9(3), 125–132.

Walter, Y. (2024). Embracing the future of Artificial Intelligence in the classroom: the relevance of AI literacy, prompt engineering, and critical thinking in modern education. International Journal of Educational Technology in Higher Education, 21(1), 15. https://doi.org/https://doi.org/10.1186/s41239-024-00404-9

Weimann-Sandig, N. (2023). Digital literacy and artificial intelligence–does chat gpt introduce the end of critical thinking in higher education? EDULEARN23 Proceedings,

Zhang, H., Lee, I., Ali, S., DiPaola, D., Cheng, Y., & Breazeal, C. (2023). Integrating ethics and career futures with technical learning to promote AI literacy for middle school students: An exploratory study. International Journal of Artificial Intelligence in Education, 33(2), 290–324.