Bilah Samping Artikel

Telah diserahkan
May 23, 2025
Diterima
July 19, 2025
Diterbitkan
October 25, 2025
Unduh
PDF (English)
Statistik
Baca Penghitung : 152 Unduh : 226

Isi Artikel Utama

Abstrak

The integration of artificial intelligence (AI) in education has accelerated, yet its pedagogical impact remains uneven and theoretically underexplored, particularly in science education. Existing studies often emphasize technical features or user satisfaction, with limited focus on how instructional design and learning context shape learning outcomes. This meta-analysis evaluated the effectiveness of AI-powered chatbots in improving student achievement in science education and identified key moderating factors influencing their impact. Using PRISMA guidelines, 26 empirical studies published between 2020 and 2024 were systematically reviewed and analyzed with a random-effects model. The overall effect size was statistically significant and moderate (Hedges’ g=0.610, p< 0.001), suggesting that chatbot-supported instruction outperformed traditional methods in many cases. However, substantial heterogeneity was observed (I²=96.58%), indicating that effectiveness varied significantly based on socio-economic context, subject area, pedagogical design, and learner experience. Chatbots were most effective in lower-middle-income countries and in subjects like computer science and natural sciences, especially when implemented through scaffolded or personalized learning strategies. Gains in engagement and satisfaction were common, while effects on self-efficacy and navigation were mixed. These findings challenge uniform assumptions about AI’s role in education and call for theory-informed, context-sensitive integration strategies. Importantly, this study extends existing learning theories by showing that AI-driven dialogue systems act not merely as tools but as active mediators of both cognitive and affective processes. Future research should pursue longitudinal designs, hybrid human–AI teaching models, and ethical frameworks to guide equitable and sustainable implementation across educational contexts.

Kata Kunci

artificial intelligence chatbots science education learning outcomes student achievement

Rincian Artikel

Lisensi

Hak Cipta (c) 2025 Jermae Dizon, Maricar S. Prudente

Creative Commons License

Artikel ini berlisensiCreative Commons Attribution-ShareAlike 4.0 International License.

Author retain copyright and grant the journal right of first publication with the simultaneously licenced under A Creative Commons Attribution (CC-By-SA) 4.0 License that allows others to share the work with an acknowledgment of the worsk's authorship and initial publication in this journal.

Cara Mengutip
Dizon, J., & Prudente , M. S. . (2025). AI-Powered Chatbots in Science Education: Meta-Analysis of Pedagogical Integration and Student Achievement. IJCER (International Journal of Chemistry Education Research), 9(2), 134–150. https://doi.org/10.20885/ijcer.vol9.iss2.art4
Unduh Sitasi
Endnote/Zotero/Mendeley (RIS)
BibTeX
Crossref
Scopus
Google Scholar
Europe PMC

Referensi

  1. R. Luckin, W. Holmes, M. Griffiths, and L. B. Forcier, Intelligence Unleashed: An Argument for AI in Education. London, UK: Pearson Education, 2016. ISBN: 9780992424886.
  2. L. Chen, F. Chen, and Z. Lin, "Artificial Intelligence in Education: A Review," IEEE Access, vol. 8, pp. 75264–75278, 2020, doi: https://doi.org/10.1109/ACCESS.2020.2988510. DOI: https://doi.org/10.1109/ACCESS.2020.2988510
  3. S. Russell and P. Norvig, Artificial Intelligence: A Modern Approach, 4th ed. Hoboken, NJ: Pearson, 2021. eBook ISBN: 9781292401171.
  4. W. Holmes, M. Bialik, and C. Fadel, Artificial Intelligence in Education: Promise and Implications for Teaching and Learning. Boston, MA: Center for Curriculum Redesign, 2019. ISBN: 978-1794293700.
  5. K. VanLehn, "The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems," Educational Psychologist, vol. 46, no. 4, pp. 197–221, 2011, doi: https://doi.org/10.1080/00461520.2011.611369. DOI: https://doi.org/10.1080/00461520.2011.611369
  6. H. F. Li, "Effects of a ChatGPT-based flipped learning guiding approach on learners’ courseware project performances and perceptions," Australasian Journal of Educational Technology, vol. 39, no. 5, pp. 40–58, 2023, doi: https://doi.org/10.14742/ajet.8923. DOI: https://doi.org/10.14742/ajet.8923
  7. L. S. Vygotsky, Mind in Society: The Development of Higher Psychological Processes. Cambridge, MA: Harvard University Press, 1978. ISBN: 9780674576292.
  8. S. Alneyadi and Y. Wardat, "ChatGPT: Revolutionizing student achievement in the electronic magnetism unit for eleventh-grade students in Emirates schools," Contemporary Educational Technology, vol. 15, no. 4, p. ep448, 2023, doi: https://doi.org/10.30935/cedtech/13417. DOI: https://doi.org/10.30935/cedtech/13417
  9. C. W. Okonkwo and A. Ade-Ibijola, "Chatbots applications in education: A systematic review," Computers and Education: Artificial Intelligence, vol. 2, p. 100033, 2021, doi: https://doi.org/10.1016/j.caeai.2021.100033. DOI: https://doi.org/10.1016/j.caeai.2021.100033
  10. S. Beltozar-Clemente and E. Díaz-Vega, "Physics XP: Integration of ChatGPT and Gamification to Improve Academic Performance and Motivation in Physics 1 Course," International Journal of Engineering Pedagogy (iJEP), vol. 14, no. 6, pp. 82–92, 2024, doi: https://doi.org/10.3991/ijep.v14i6.47127. DOI: https://doi.org/10.3991/ijep.v14i6.47127
  11. A. D. Topal, C. D. Eren, and A. K. Geçer, "Chatbot Application in a 5th Grade Science Course," Education and Information Technologies, vol. 26, pp. 6241–6265, 2021, doi: https://doi.org/10.1007/s10639-021-10627-8. DOI: https://doi.org/10.1007/s10639-021-10627-8
  12. Y. Xu, J. Zhu, M. Wang, F. Qian, Y. Yang, and J. Zhang, "The Impact of a Digital Game-Based AI Chatbot on Students’ Academic Performance, Higher-Order Thinking, and Behavioral Patterns in an Information Technology Curriculum," Applied Sciences, vol. 14, no. 6418, 2024, doi: https://doi.org/10.3390/app14156418. DOI: https://doi.org/10.3390/app14156418
  13. H. Kumar, R. Xiao, B. Lawson, I. Musabirov, J. Shi, X. Wang, H. Luo, J. J. Williams, A. N. Rafferty, J. Stamper, and M. Liut, “Supporting self-reflection at scale with large language models: Insights from randomized field experiments in classrooms,” in Proc. 11th ACM Conf. Learning @ Scale (L@S ’24), Atlanta, GA, USA, 2024, pp. 86–97. doi: https://doi.org/10.1145/3657604.3662042. DOI: https://doi.org/10.1145/3657604.3662042
  14. T. Ekukinam, I. N. Udosen, and N. I. Udoh, "The Difference in Academic Performance of Senior Secondary School Biology Students Exposed to Chatbot AI or Expository Method Based on Their Gender," Universal Academic Journal of Education, Science and Technology, vol. 6, no. 2, pp. 134–138, Aug. 2024, [Online]. Available: https://www.globalacademicstar.com/article/the-difference-in-academic-performance-of-senior-secondary-school-biology-students-exposed-to-chatbot-ai-or-expository-method-based-on-their-gender-88841.
  15. M. Fırat and S. Kuleli, "GPT vs. Google: A comparative study of self-code learning in ODL students," Journal of Educational Technology & Online Learning, vol. 7, no. 3, pp. 308–319, 2024, doi: https://doi.org/10.31681/jetol.1508675. DOI: https://doi.org/10.31681/jetol.1508675
  16. D. Moher, A. Liberati, J. Tetzlaff, D. G. Altman, and PRISMA Group, “Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement,” PLoS Med., vol. 6, no. 7, p. e1000097, Jul. 2009, doi: https://doi.org/10.1371/journal.pmed.1000097. DOI: https://doi.org/10.1371/journal.pmed.1000097
  17. M. Borenstein, L. V. Hedges, J. P. T. Higgins, and H. R. Rothstein, Introduction to Meta-Analysis. Chichester, UK: John Wiley & Sons, 2009, doi: https://doi.org/10.1002/9780470743386. DOI: https://doi.org/10.1002/9780470743386
  18. J. P. T. Higgins et al., "The Cochrane Collaboration's tool for assessing risk of bias in randomised trials," BMJ, vol. 343, p. d5928, Oct. 2011, doi: https://doi.org/10.1136/bmj.d5928. DOI: https://doi.org/10.1136/bmj.d5928
  19. J. Yin, T. T. Goh, B. Yang, and Y. Xiaobin, "Conversation Technology With Micro-Learning: The Impact of Chatbot-Based Learning on Students’ Learning Motivation and Performance," Journal of Educational Computing Research, vol. 0, no. 0, pp. 1–23, 2020, doi: https://doi.org/10.1177/0735633120952067. DOI: https://doi.org/10.1177/0735633120952067
  20. H. B. Essel, D. Vlachopoulos, A. Tachie-Menson, E. E. Johnson, and P. K. Baah, "The Impact of a Virtual Teaching Assistant (Chatbot) on Students' Learning in Ghanaian Higher Education," International Journal of Educational Technology in Higher Education, vol. 19, p. 57, 2022, doi: https://doi.org/10.1186/s41239-022-00362-6. DOI: https://doi.org/10.1186/s41239-022-00362-6
  21. United Nations, World Economic Situation and Prospects 2025. New York: United Nations, 2025. [Online]. Available: https://desapublications.un.org/file/20954/download.
  22. R. Van der Spoel, A. Noroozi, L. E. van Ginkel, and M. E. Mulder, "Teachers’ online teaching expectations and experiences during the COVID-19 pandemic in the Netherlands," European Journal of Teacher Education, vol. 43, no. 4, pp. 623–638, 2020, doi: https://doi.org/10.1080/02619768.2020.1821185. DOI: https://doi.org/10.1080/02619768.2020.1821185
  23. H. Y. Sung, G. D. Hwang, and C. Xie, "Artificial intelligence in education: Learning assessment, teacher professional development, and future challenges,"Interactive Learning Environments, vol. 31, no. 5, pp. 742–761, 2023, doi: https://doi.org/10.1080/10494820.2021.1952615. DOI: https://doi.org/10.1080/10494820.2021.1952615
  24. J. R. Aguilar-Mejía, S. Tejeda, C. V. Ramirez-Lopez, and C. L. Garay-Rondero, "Design and Use of a Chatbot for Learning Selected Topics of Physics," in Technology-Enabled Innovations in Education. Singapore: Springer Nature, 2022, ch. 13, pp. 175–188, doi: https://doi.org/10.1007/978-981-19-3383-7_13. DOI: https://doi.org/10.1007/978-981-19-3383-7_13
  25. H. B. Essel, D. Vlachopoulos, H. Nunoo-Mensah, and J. O. Amankwa, "Exploring the Impact of VoiceBots on Multimedia Programming Education Among Ghanaian University Students," British Journal of Educational Technology, vol. 00, pp. 1–20, 2024, doi: https://doi.org/10.1111/bjet.13504. DOI: https://doi.org/10.1111/bjet.13504
  26. M. Tran, "Generative artificial intelligence as the ‘more knowledgeable other’: Extending Vygotsky’s Zone of Proximal Development," Frontiers in Education, vol. 10, p. 12254308, 2025, doi: https://doi.org/10.3389/feduc.2025.12254308.
  27. B. Graefen and N. Fazali, "GPTEACHER: Examining the Efficacy of ChatGPT as a Tool for Public Health Education," European Journal of Education Studies, vol. 10, no. 8, pp. 254–259, 2023, doi: https://doi.org/10.46827/ejes.v10i8.4926. DOI: https://doi.org/10.46827/ejes.v10i8.4926
  28. F. E. Çiçek, M. Ülker, M. Özer, and Y. S. Kıyak, "ChatGPT versus expert feedback on clinical reasoning questions and their effect on learning: A randomized controlled trial," Postgraduate Medical Journal, pp. 1–6, 2024, doi: https://doi.org/10.1093/postmj/qgae170. DOI: https://doi.org/10.1093/postmj/qgae170
  29. H. Liu, "Applicability of ChatGPT in Online Collaborative Learning: Evidence Based on Learning Outcomes," Proc. Int. Acad. Conf. Educ., vol. 1, no. 1, pp. 33–43, 2024, doi: https://doi.org/10.33422/iaceducation.v1i1.656. DOI: https://doi.org/10.33422/iaceducation.v1i1.656
  30. S. Al Kahf et al., "Chatbot-based serious games: A useful tool for training medical students? A randomized controlled trial," PLoS ONE, vol. 18, no. 3, p. e0278673, Mar. 2023, doi: https://doi.org/10.1371/journal.pone.0278673. DOI: https://doi.org/10.1371/journal.pone.0278673
  31. S. Challapalli and J. Leddo, "Comparing the Relative Effectiveness of Chat GPT-generated Content and Human-generated Videos for Teaching Students Calculus," International Journal of Social Science and Economic Research, vol. 9, no. 11, pp. 5434–5442, Nov. 2024, doi: https://doi.org/10.46609/IJSSER.2024.v09i11.031. DOI: https://doi.org/10.46609/IJSSER.2024.v09i11.031
  32. M. Zawacki-Richter et al., "Systematic Review of Research on Artificial Intelligence Applications in Higher Education – Where Are the Educators?" International Journal of Educational Technology in Higher Education, vol. 16, no. 1, p. 39, 2019, doi: https://doi.org/10.1186/s41239-019-0171-0. DOI: https://doi.org/10.1186/s41239-019-0171-0
  33. J. Fowlin, "Inventing Distributed Cognition With Artificial Intelligence: Toward a New Theory of Cognition," Education Sciences, vol. 15, no. 3, p. 393, 2025, doi: https://doi.org/10.3390/educsci15030393. DOI: https://doi.org/10.3390/educsci15030393
  34. A. P. Bhatia, A. Lambat, and T. Jain, "A Comparative Analysis of Conventional and Chat-Generative Pre-trained Transformer-Assisted Teaching Methods in Undergraduate Dental Education," Cureus, vol. 16, no. 5, p. e60006, May 2024, doi: https://doi.org/10.7759/cureus.60006. DOI: https://doi.org/10.7759/cureus.60006
  35. M. Hakiki, R. Fadli, A. D. Samala, A. Fricticarani, P. Dayurni, K. Rahmadani, A. D. Astiti, and A. Sabir, “Exploring the impact of using Chat-GPT on student learning outcomes in technology learning: The comprehensive experiment,” Adv. Mobile Learn. Educ. Res., vol. 3, no. 2, pp. 859–872, 2023, doi: https://doi.org/10.25082/AMLER.2023.02.013. DOI: https://doi.org/10.25082/AMLER.2023.02.013
  36. G. Huesca et al., "Effectiveness of Using ChatGPT as a Tool to Strengthen Benefits of the Flipped Learning Strategy," Education Sciences, vol. 14, no. 660, 2024, doi: https://doi.org/10.3390/educsci14060660. DOI: https://doi.org/10.3390/educsci14060660
  37. M. Koć-Januchta, K. J. Schöonborn, L. A. E. Tibell, V. K. Chaudhri, and H. C. Heller, "Engaging with Biology by Asking Questions: Investigating Students’ Interaction and Learning with an Artificial Intelligence-Enriched Textbook," Journal of Educational Computing Research, vol. 58, no. 6, pp. 1190–1224, 2020, doi: https://doi.org/10.1177/0735633120921581. DOI: https://doi.org/10.1177/0735633120921581
  38. A. A. K. Kusuma et al., "Effectiveness of Artificial Intelligent Independent Learning (AIIL) with Physics Chatbot of Global Warming Concept," Momentum: Physics Education Journal, vol. 8, no. 1, pp. 42–54, 2024, doi: https://doi.org/10.21067/mpej.v8i1.8942. DOI: https://doi.org/10.21067/mpej.v8i1.8942
  39. Y. T. Lin and J.-H. Ye, "Development of an Educational Chatbot System for Enhancing Students’ Biology Learning Performance," Journal of Internet Technology, vol. 24, no. 2, pp. 275–278, Mar. 2023, doi: https://doi.org/10.53106/160792642023032402006. DOI: https://doi.org/10.53106/160792642023032402006
  40. R. Mellado-Silva, A. Faúndez-Ugalde, and M. Blanco-Lobos, "Effective Learning of Tax Regulations using Different Chatbot Techniques," Adv. Sci. Technol. Eng. Syst. J., vol. 5, no. 6, pp. 439–446, Nov. 2020, doi: https://doi.org/10.25046/aj050652. DOI: https://doi.org/10.25046/aj050652
  41. Z. A. Pardos and S. Bhandari, "ChatGPT-Generated Help Produces Learning Gains Equivalent to Human Tutor-Authored Help on Mathematics Skills," PLoS ONE, vol. 19, no. 5, p. e0304013, May 2024, doi: https://doi.org/10.1371/journal.pone.0304013. DOI: https://doi.org/10.1371/journal.pone.0304013
  42. T. T. Wu, H. Y. Lee, P. H. Chen, C. J. Lin, and Y. M. Huang, "Integrating peer assessment cycle into ChatGPT for STEM education: A randomised controlled trial on knowledge, skills, and attitudes enhancement,"J. Comput. Assist. Learn., vol. 40, no. 1, Oct. 2024. [Online]. Available: https://doi.org/10.1111/jcal.13085. DOI: https://doi.org/10.1111/jcal.13085
  43. Y. Xue, H. Chen, G. R. Bai, R. Tairas, and Y. Huang, "Does ChatGPT Help With Introductory Programming? An Experiment of Students Using ChatGPT in CS1," in Proc. 46th Int. Conf. on Software Engineering: Software Engineering Education and Training (ICSE-SEET '24), Apr. 2024, Lisbon, Portugal, doi: https://doi.org/10.1145/3639474.3640076. DOI: https://doi.org/10.1145/3639474.3640076
  44. J. A. Kulik and J. D. Fletcher, "Effectiveness of Intelligent Tutoring Systems: A Meta-Analytic Review," Review of Educational Research, vol. 86, no. 1, pp. 42–78, 2016, doi: https://doi.org/10.3102/0034654315581420. DOI: https://doi.org/10.3102/0034654315581420
  45. L. Labadze, M. Grigolia, and L. Machaidze, "Role of AI chatbots in education: A systematic literature review," International Journal of Educational Technology in Higher Education, vol. 20, no. 56, 2023, doi: https://doi.org/10.1186/s41239-023-00426-1. DOI: https://doi.org/10.1186/s41239-023-00426-1
  46. C. Y. Chang, G. J. Hwang, and M. L. Gau, "Promoting students' learning achievement and self-efficacy: A mobile chatbot approach for nursing training," British Journal of Educational Technology, vol. 53, no. 1, pp. 171–188, 2022, doi: https://doi.org/10.1111/bjet.13158. DOI: https://doi.org/10.1111/bjet.13158
  47. D. H. Schunk and J. A. Greene, Handbook of Self-Regulation of Learning and Performance, 2nd ed. New York: Routledge, 2021, doi: https://doi.org/10.4324/9781315697048. DOI: https://doi.org/10.4324/9781315697048
  48. B. Kim and T. C. Reeves, "Reframing research on learning with technology: In search of the meaning of cognitive tools," Instructional Science, vol. 35, no. 3, pp. 207–256, 2007, doi: https://doi.org/10.1007/S11251-006-9005-2. DOI: https://doi.org/10.1007/s11251-006-9005-2