IRO Journals
Journal of Information Technology and Digital World

Multiclass Software Defect Prediction Using an Optimized LLM CodeSentinel-BERT Based Framework

Open Access
Published: 17 October, 2026
Pages: 382-393
  • Vijayarajan S.M. , Vijayarajan S.M.
    Associate Professor, Department of Computer Science and Engineering, NPR College of Engineering and Technology, Natham, Dindigul, India
    Associate Professor, Department of Computer Science and Engineering, NPR College of Engineering and Technology, Natham, Dindigul, India
  • Ponnalagar V. , Ponnalagar V.
    Student, Department of Computer Science and Engineering, NPR College of Engineering and Technology, Natham, Dindigul, India
    Student, Department of Computer Science and Engineering, NPR College of Engineering and Technology, Natham, Dindigul, India
  • Ravikrishnan R. , Ravikrishnan R.
    Student, Department of Computer Science and Engineering, NPR College of Engineering and Technology, Natham, Dindigul, India
    Student, Department of Computer Science and Engineering, NPR College of Engineering and Technology, Natham, Dindigul, India
  • Sarguna Gokul R. Sarguna Gokul R.
    Student, Department of Computer Science and Engineering, NPR College of Engineering and Technology, Natham, Dindigul, India
    Student, Department of Computer Science and Engineering, NPR College of Engineering and Technology, Natham, Dindigul, India
view PDF
view PDF

How to Cite

S.M., Vijayarajan, Ponnalagar V., Ravikrishnan R., and Sarguna Gokul R. 2026. “Multiclass Software Defect Prediction Using an Optimized LLM CodeSentinel-BERT Based Framework”. Journal of Information Technology and Digital World 8 (4): 382-93. https://doi.org/10.36548/jitdw.2026.4.007.
ACM ACS APA ABNT Chicago Harvard IEEE MLA Turabian Vancouver AMA

Download Citation

Endnote/Zotero/Mendeley (RIS) BibTeX

Keywords

Software Defect Prediction
Source Code Analysis
Transformer Models
CodeBERT
Multiclass Classification
ROC-AUC Analysis
Real-Time Defect Detection

Abstract

The prediction of software defects is important to detect defects in source codes prior to their deployment but traditional techniques use selected features and limited binary classification. This research work proposes DefectGuard, a transformer-based multiclass software defect prediction system for C/C++ source code that utilizes CodeBERT fine-tuning. This custom dataset consists of 2,000 samples of manually collected and labeled C/C++ source codes and includes nine classes of defects, namely syntax, semantic, logic, linker, and runtime-related defects. The proposed pipeline includes preprocessing, stratified splitting, sub-word tokenization, CodeBERT representation learning, and multiclass classification. The experimental results show the accuracy, precision, recall, and F1-score of 98.8%, 96.4%, 95.8%, and 96.1%, respectively, on the test set. Additionally, ROC-AUC and confusion-matrix metrics are used to evaluate the class-level discrimination. Moreover, the proposed model is implemented in a real-time Flask-based web application, which can classify the source code into defect labels with confidence scores and language detection to provide the interpretation feedback.

References

  1. Feng, Zhangyin, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou et al. “Codebert: A Pre-trained Model for Programming and Natural Languages.” In Findings of the Association for Computational Linguistics: EMNLP, 2020, 1536-1547.
  2. Liu, Yinhan, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. “Roberta: A Robustly Optimized Bert Pretraining Approach.” arXiv preprint, 2019, 1-13.
  3. Radford, Alec, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. “Language Models are Unsupervised Multitask Learners.” OpenAI Blog, 2019, vol. 1, no. 8: 9.
  4. Menzies, Tim, Jeremy Greenwald, and Art Frank. “Data Mining Static Code Attributes to Learn Defect Predictors.” IEEE Transactions on Software Engineering, 2007, vol. 33, no. 1: 2-13.
  5. Lessmann, Stefan, Bart Baesens, Christophe Mues, and Swantje Pietsch. “Benchmarking Classification Models for Software Defect Prediction: A Proposed Framework and Novel Findings.” IEEE Transactions on Software Engineering, 2008, vol. 34, no. 4: 485-496.
  6. Zou, Deqing, Sujuan Wang, Shouhuai Xu, Zhen Li, and Hai Jin. “$mu $ μ VulDeePecker: A Deep Learning-Based System for Multiclass Vulnerability Detection.” IEEE Transactions on Dependable and Secure Computing, 2019, vol. 18, no. 5: 2224-2236.
  7. Guo, Daya, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou et al. “Graphcodebert: Pre-training Code Representations with Data Flow.”arXiv preprint, 2020.
  8. Husain, Hamel, Ho-Hsiang Wu, Tiferet Gazit, Miltiadis Allamanis, and Marc Brockschmidt. “Codesearchnet Challenge: Evaluating the State of Semantic Code Search.” arXiv preprint, 2019.
  9. White, Martin, Michele Tufano, Christopher Vendome, and Denys Poshyvanyk. “Deep Learning Code Fragments for Code Clone Detection.” In Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering, 2016, 87-98.
  10. Alon, Uri, Meital Zilberstein, Omer Levy, and Eran Yahav. “code2vec: Learning Distributed Representations of Code.” Proceedings of the ACM on Programming Languages, 2019, vol. 3, no. POPL: 1-29.
  11. Gregory Tassey. “The Economic Impacts of Inadequate Infrastructure for Software Testing.” National Institute of Standards and Technology, 2002, vol. 1, no. 2002: 1.
  12. Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. “Bert: Pre-training of Deep Bidirectional Transformers for Language Understanding.” In Proceedings of the 2019 Conference of The North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019, vol. 1: 4171-4186.