Beyond Familiarity: LLM Use, Perceived Accuracy, and Student-Generated Improvements in a University Setting

Mohammad Mahyoob Albuhairy

doi:10.53286/fdxrq953

Authors

Mohammad Mahyoob Albuhairy Associate Professor of Computational Linguistics, Department of Languages and Translation, Faculty of Arts and Humanities, Taibah University, Madinah, Saudi Arabia. https://orcid.org/0000-0002-6664-1017

DOI:

https://doi.org/10.53286/fdxrq953

Keywords:

large language models, higher education, student perceptions, human-AI interaction, coding assistance

Abstract

Large language models (LLMs) are transforming higher education, yet the factors shaping students’ adoption and trust remain insufficiently understood, particularly in underrepresented educational contexts. This study investigates university students’ awareness, usage patterns, perceptions, and recommendations regarding LLMs through a cross-sectional survey of engineering, computer science, and data science students. Descriptive and inferential statistical analyses examined adoption patterns and student perceptions. Although 90% of students were familiar with LLMs and 62% had used them for coding tasks, only 51% reported regular daily or weekly use, indicating that perceived usefulness and ease of use alone do not fully explain adoption. They also recognized their value for saving time and supporting learning. Students consistently identified accuracy, source transparency, and verifiability as essential for responsible use, highlighting trust as a key determinant of continued engagement. Building on these findings, the study proposes the Trust-Calibrated Technology Acceptance Model (TC-TAM), extending the Technology Acceptance Model by incorporating perceived trustworthiness as a critical factor influencing LLM adoption. The findings provide practical recommendations for educators, developers, and policymakers to support transparent, equitable, and responsible AI integration while contributing empirical evidence from an underexplored educational setting to advance research on generative AI in higher education.

Downloads

Download data is not yet available.

References

Akpinar, N. J., Avula, S., Lee, C. J., Dang, B., Razat, K., & Murdock, V. (2026, April). LLM or Human? Perceptions of Trust and Quality in Research Summaries. In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (pp. 1–25).

Al-Garaady, J., & Albuhairy, M. M. (2023). ChatGPT's capabilities in spotting and analyzing writing errors experienced by EFL learners. Arab World English Journals, Special Issue on CALL, (9).

Albuhairy, M. M., Al-Garaady, J., & Alblwi, A. (2023). A proposed framework for human-like language processing of ChatGPT in academic writing. International Journal of Emerging Technologies in Learning (iJET), 18(14).

Bernabei, M., Colabianchi, S., Falegnami, A., & Costantino, F. (2023). Students’ use of large language models in engineering education: A case study on technology acceptance, perceptions, efficacy, and detection chances. Computers and Education: Artificial Intelligence, 5, 100172.

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS quarterly, 13(3), 319–34.

Glikson, E., & Woolley, A. W. (2020). Human trust in artificial intelligence: Review of empirical research. Academy of Management Annals, 14(2), 627–660.

Hossain, S., Khanam, S., Haniya, S., & Nasr, N. R. (2025). AI Literacy and LLM Engagement in Higher Education: A Cross-National Quantitative Study. arXiv preprint arXiv:2507.03020.

Jang, C. (2024). Coping with vulnerability: The effect of trust in AI and privacy-protective behaviour on the use of AI-based services. Behaviour & Information Technology, 43(11), 2388-2400

Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., ... & Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and individual differences, 103, 102274.

Kazemitabaar, M., Chow, J., Ma, C. K. T., Ericson, B. J., Weintrop, D., & Grossman, T. (2023, April). Studying the effect of AI code generators on supporting novice learners in introductory programming. In Proceedings of the 2023 CHI conference on human factors in computing systems (pp. 1-23).

Lee, H., Atif, A., & Kang, K. (2026). Analyzing AI utilization in education through learner question types: A constructivist approach. Australasian Journal of Educational Technology, 42(1), 79–96. https://doi.org/10.14742/ajet.10657

Li, R., Li, M., & Qiao, W. (2025). Engineering students’ use of large language model tools: An empirical study based on a survey of students from 12 universities. Education Sciences, 15(3), 280.

Mirabile, M., Corazza, G. E., & Alonso-Moral, J. M. (2026). Trust in human–AI collaboration in finance: a bibliometric–systematic literature review. AI & SOCIETY, 1–30.

Lyu, W., Wang, Y., Chung, T., Sun, Y., & Zhang, Y. (2024, July). Evaluating the effectiveness of llms in introductory computer science education: A semester-long field study. In Proceedings of the eleventh ACM conference on learning@ scale (pp. 63-74).

O’Neill, S., Mulgrew, D., & Bagdasar, O. (2025). On the Use of Large Language Models for Improving Student and Staff Experience in Higher Education. Open Education Studies, 7(1), 20250086.

Peng, Q., Zhang, C., Mangal, R., Pasareanu, C., & Jia, L. (2025, April). Random Perturbation Attack on LLMs for Code Generation. In 2025 IEEE/ACM 4th International Conference on AI Engineering–Software Engineering for AI (CAIN) (pp. 285–287). IEEE.

Perkins, M. (2023). Academic integrity considerations of AI large language models in the post-pandemic era: ChatGPT and beyond. Journal of University Teaching and Learning Practice, 20(2), 1-24.

Raihan, N., Siddiq, M. L., Santos, J. C., & Zampieri, M. (2025, February). Large language models in computer science education: A systematic literature review. In Proceedings of the 56th ACM technical symposium on computer science education V. 1 (pp. 938–944).

Rasnayaka, S., Wang, G., Shariffdeen, R., & Iyer, G. N. (2024, April). An empirical study on usage and perceptions of llms in a software engineering project. In Proceedings of the 1st international workshop on large language models for code (pp. 111-118).

Singal, A., & Goyal, S. (2025). Comparative evaluation of AI platforms “Google Gemini 2.5 Flash, Google Gemini 2.0 Flash, DeepSeek V3 and ChatGPT 4o” in solving multiple-choice questions from different subtopics of anatomy. Surgical and Radiologic Anatomy, 47(1), 193.

Scherer, R., Siddiq, F., & Tondeur, J. (2019). The technology acceptance model (TAM): A meta-analytic structural equation modeling approach to explaining teachers’ adoption of digital technology in education. Computers & education, 128, 13-35.

Xu, J., Shen, R., Li, J., & Hu, D. (2025). Artificial intelligence literacy among Chinese Master of Education students: Current situation, influencing factors and generative pathways. Australasian Journal of Educational Technology, 41(6), 1–17. https://doi.org/10.14742/ajet.10656.