IRO Journals
Journal of Information Technology and Digital World

A Structural and Behavioral Hybrid Framework for Detecting AI-Generated Code in Programming Education

Open Access
Published: 17 October, 2026
Pages: 394-409
  • Arockia Irudayaraja M. , Arockia Irudayaraja M.
    Assistant Professor, Department of Computer Science and Engineering, NPR College of Engineering and Technology, Dindigul, India
    Assistant Professor, Department of Computer Science and Engineering, NPR College of Engineering and Technology, Dindigul, India
  • Rexton George R. , Rexton George R.
    Student, Department of Computer Science and Engineering, NPR College of Engineering and Technology, Dindigul, India
    Student, Department of Computer Science and Engineering, NPR College of Engineering and Technology, Dindigul, India
  • Nishanth K. , Nishanth K.
    Student, Department of Computer Science and Engineering, NPR College of Engineering and Technology, Dindigul, India
    Student, Department of Computer Science and Engineering, NPR College of Engineering and Technology, Dindigul, India
  • Sakthivel P. Sakthivel P.
    Student, Department of Computer Science and Engineering, NPR College of Engineering and Technology, Dindigul, India
    Student, Department of Computer Science and Engineering, NPR College of Engineering and Technology, Dindigul, India
view PDF
view PDF

How to Cite

M., Arockia Irudayaraja, Rexton George R., Nishanth K., and Sakthivel P. 2026. “A Structural and Behavioral Hybrid Framework for Detecting AI-Generated Code in Programming Education”. Journal of Information Technology and Digital World 8 (4): 394-409. https://doi.org/10.36548/jitdw.2026.4.008.
ACM ACS APA ABNT Chicago Harvard IEEE MLA Turabian Vancouver AMA

Download Citation

Endnote/Zotero/Mendeley (RIS) BibTeX

Keywords

AI-generated Code Detection
Abstract Syntax Tree Analysis
Semantic Execution Fingerprinting
Lexical Stylometry
Multidimensional Feature Fusion
XGBoost Classification
Source-Code Obfuscation

Abstract

The Large Language Models (LLMs) for generating source code creates a need for code-aware detection mechanisms capable of analyzing characteristics beyond token-level statistics. This research proposes CodeGuardX, a hybrid framework of structure and behavior, which combines Abstract Syntax Tree inspection, runtime sandboxing, and lexical stylometry in the detection of AI-generated codes. A vector of 33 dimensions containing 17 structural, 6 behavioral, and 10 stylometric features is used for classification using XGBoost with probability calibration and threshold optimization based on F1 metric. Evaluation of the model is performed using 6,800 code samples from the domains of Python, Java, C, and C++; a test set of 1,200 code samples is used for testing purposes. CodeGuardX achieves an accuracy score of 93.8% and F1 score of 92.9%. Ablation study demonstrates increasing F1 scores of 77.8%, 87.3%, and 92.9%. The combination of source code obfuscation, the framework retains 89.4% F1-score, demonstrating the robustness to lexical transformations.

References

  1. Mitchell, Eric, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. "Detectgpt: Zero-Shot Machine-Generated Text Detection using Probability Curvature." In International Conference on Machine Learning, Proceedings of Machine Learning Research, 2023, 24950-24962.
  2. Pan, Wei Hung, Ming Jie Chok, Jonathan Leong Shan Wong, Yung Xin Shin, Yeong Shian Poon, Zhou Yang, Chun Yong Chong, David Lo, and Mei Kuan Lim. "Assessing Ai Detectors in Identifying Ai-Generated Code: Implications for Education." In Proceedings of the 46th International Conference on Software Engineering: Software Engineering Education and Training, 2024, 1-11.
  3. Farrelly, Tom, and Nick Baker. "Generative Artificial Intelligence: Implications and Considerations for Higher Education Practice." Education Sciences, 2023, vol. 13, no. 11: 1109.
  4. Chan, Cecilia Ka Yuk. "A Comprehensive AI Policy Education Framework for University Teaching and Learning." International Journal of Educational Technology in Higher Education, 2023, vol. 20, no. 1: 38.
  5. Otterbacher, Jahna. "Why Technical Solutions for Detecting AI-Generated Content in Research and Education are Insufficient." Patterns, 2023, vol. 4, no. 7.
  6. Chaka, Chaka. "Detecting Ai Content in Responses Generated by Chatgpt, Youchat, and Chatsonic: The Case of Five Ai Content Detection Tools." Journal of Applied Learning Teaching, 2023, vol. 6, no. 2: 94-104.
  7. Abuhamad, Mohammed, Ji-su Rhim, Tamer AbuHmed, Sana Ullah, Sanggil Kang, and DaeHun Nyang. "Code Authorship Identification using Convolutional Neural Networks." Future Generation Computer Systems, 2019, vol. 95: 104-115.
  8. Kirchenbauer, John, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. "A Watermark for Large Language Models." In International Conference on Machine Learning, Proceedings of Machine Learning Research, 2023, 17061-17084.
  9. Becker, Brett A., Paul Denny, James Finnie-Ansley, Andrew Luxton-Reilly, James Prather, and Eddie Antonio Santos. "Programming is Hard-or At Least it used to Be: Educational Opportunities and Challenges of Ai Code Generation." In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V, 2023, vol. 1: 500-506.
  10. Gehrmann, Sebastian, Hendrik Strobelt, and Alexander M. Rush. "Gltr: Statistical Detection and Visualization of Generated Text." In Proceedings of the 57th Annual Meeting of The Association for Computational Linguistics: System Demonstrations, 2019, 111-116.
  11. Chen, Tianqi, and Carlos Guestrin. "Xgboost: A Scalable Tree Boosting System." In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016, 785-794.
  12. Alsulami, Bander, Edwin Dauber, Richard Harang, Spiros Mancoridis, and Rachel Greenstadt. "Source Code Authorship Attribution using Long Short-term Memory-based Networks." In European Symposium on Research in Computer Security, Cham: Springer International Publishing, 2017, 65-82.
  13. Caliskan-Islam, Aylin, Richard Harang, Andrew Liu, Arvind Narayanan, Clare Voss, Fabian Yamaguchi, and Rachel Greenstadt. "De-Anonymizing Programmers Via Code Stylometry." In 24th USENIX Security Symposium, 2015, 255-270.
  14. Boukili, G., El Garouani, S., and Riffi, J., “A Dataset for Human-Written and AI-Generated Code Source Classification,” Data in Brief, 2025 vol. 65, 112527. Available: https://data.mendeley.com/datasets/kjh95n54f8/1
  15. GitHub, “Github Copilot X: The AI-Powered Developer Experience,” GitHub Research Report, 2023. Available: https://github.blog/ 2023-03-22-github-copilot-x-the-ai-powered-developer-experience/