Analysis of ChatGPT and Gemini Mental Models of Apparent Motion of the Sun and Stars

Nurhikma Safitri; Jusman  Mansyur; Haeruddin Haeruddin; Nurul  Kami Sani; Andi Ulfah  Khuzaimah

doi:10.24239/koordinat.v7i1.195

Nurhikma Safitri Universitas Tadulako
Jusman Mansyur Universitas Tadulako
Haeruddin Haeruddin Universitas Tadulako
Nurul Kami Sani Universitas Tadulako
Andi Ulfah Khuzaimah Universitas Tadulako

DOI: https://doi.org/10.24239/koordinat.v7i1.195

Keywords: Apparent Motion of Stars, Apparent Motion of the Sun, ChatGPT, Gemini, Mental Model

Abstract

mental models regarding the apparent motion of the Sun and stars. The method used is a qualitative approach with an exploratory descriptive study using the reflective thematic analysis method according to Braun, Clarke, and Rance (2014). The subjects of this study were ChatGPT and Gemini, who also acted as respondents. The instrument used was 10 multiple-choice questions about the apparent motion of celestial bodies. Each answer was analyzed thematically to identify reasoning patterns, conceptual structures, and conformity with scientific mental models. The results show that ChatGPT predominantly displays a scientific mental model consistent with a heliocentric understanding and scientific explanation of apparent motion, while Gemini tends to display a hybrid mental model with a mixture of scientific concepts and preconceptions. The comparison of the two shows differences in conceptual depth, consistency of reasoning, and accuracy of scientific representation. These findings contribute to the development of the study of artificial intelligence mental models and open up opportunities for their use in AI-based science learning

References

Anderson, J., Smith, L., & Doe, R. (2023). Improving Student Engagement and Learning Outcomes with AI-Powered Chatbots. Journal of Educational Technology, 15(2), 45-67.

AlZu'bi, S., Mughaid, A., Quiam, F., & Hendawi, S. (2023). Exploring the Capabilities and Limitations of ChatGPT and Alternative Big Language Models. Artificial Intelligence and Applications. DOI: 10.47852/bonviewAIA3202820

Brown, T., Harris, M., & Walker, K. (2023). AI-Powered Tutoring Systems: The Role of ChatGPT in Personalized Education. Journal of Innovative Education, 11(2), 150-170.

Braun, Virginia, Victoria Clarke, and Nicola Rance. 2014. How to use thematic analysis with790 interview data. In The counselling and psychotherapy research handbook, eds.791 Andreas Vossler, and Naomi Moller, 183-197. London, UK: Sage Publications.

Brown, A. (2021). The role of AI in science education: Building mental models through technology. Artificial Intelligence in Education, 48(2), 123-134.

Bekaert, H., Van Winckel, H., Van Dooren, W., Steegen, A., & De Cock, M. (2022). Identifying students’ mental models of the apparent motion of the Sun and stars. Physical Review Physics Education Research, 18(1), 010130.

Brown, J., & Thompson, R. (2022). Enhancing problem-solving skills through generative AI in educational settings. Journal of Educational Technology, 20(4), 45-60.

Duit, R., & Treagust, D. F. (2003). Conceptual change: A powerful framework for improving science teaching and learning. International Journal of Science Education, 25(6), 671–688.

Fletcher, R. J. (2022). Analysis of Solar Motion in Relation to Stellar Motion. Astronomy and Astrophysics Review, 30(4), 315-330.

Gupta, R., & Singh, A. (2023). Predicting Academic Performance with Gemini: A Machine Learning Approach. Educational Data Science Journal, 9(4), 210-225.

Khan, M. A., & Yousuf, M. (2019). Exploring the Role of Artificial Intelligence in Science Education: A Case Study in Astronomy Learning. Journal of Educational Technology Systems, 48(2), 214-227.

Khurshid, M. A. (2019). Solar and Stellar Motion: A Comparative Study. Journal of Modern Physics, 10(5), 500-510.

Martinez, A., Gonzalez, P., & Lee, S. (2022). Enhancing Creative Writing through AI: A Study on the Impact of ChatGPT on Writing Quality. International Journal of Creative Writing, 8(3), 120-135.

Nyaaba, M. (2023). Comparing Human and AI’s (GPT-4 and Gemini) Understanding of the Nature of Science. SSRN Electronic Journal.

Patel, K. R. (2018). The Role of the Earth’s Rotation in the Apparent Motion of Stars. International Journal of Astronomy, 6(3), 45-55.

Plummer, J. D. (2009). Children’s understanding of the apparent motion of celestial bodies. Physical Review Special Topics - Physics Education Research, 5(2), 020103.

Wang, X., Zhao, L., & Chen, H. (2023). Leveraging Gemini AI for Early Diagnosis of Learning Disabilities. Journal of Learning Disabilities Research, 12(1), 78-92.

Russell, S., & Norvig, P. (2020). Artificial Intelligence: A Modern Approach (4th ed.). Boston: Pearson. ISBN: 978-0134610993.

Smith, D., & Jones, R. (2020). Student misconceptions in understanding apparent motion of celestial bodies. International Journal of Science Education, 42(1), 47-63.

Smith, J., & Johnson, A. (2023). Comparative Analysis of AI Models: ChatGPT, Gemini, and Perplexity AI. Journal of Artificial Intelligence Research, 45(2), 123-145.

Schulze Balhorn, L., Weber, J. M., Buijsman, S., Hildebrandt, J. R., Ziefle, M., & Schweidtmann, A. M. (2024). Empirical assessment of ChatGPT’s answering capabilities in natural science and engineering. Scientific Reports, 14(1), 4998.

Tran, L. H. (2021). Understanding Apparent Motion in Celestial Mechanics. Celestial Mechanics and Dynamical Astronomy, 139(1), 87-102.

Vosniadou, S., & Brewer, W. F. (1992). Mental Models of the Earth: A Study of Conceptual Change in Childhood. In The development of scientific reasoning (pp. 81-94). Lawrence Erlbaum Associates.

Vosniadou, S. (1994). Capturing and modeling the process of conceptual change. Learning and Instruction, 4(1), 45–69.

Williams, J. D. S. (2020). The Apparent Motion of the Sun and Stars. Astrophysics Journal, 150(2), 123-130.