IRO Journals
Journal of Innovative Image Processing

A Hierarchical Swin Transformer-Based Framework for Knee Osteoarthritis Severity Classification Using Radiographic Images

Open Access
Published: 05 August, 2026
Pages: 1290-1312
  • Zahoor M. Aydam , Zahoor M. Aydam
    Department of Information Technology, College of Computer Science and Mathematics, University of Thi-Qar, Thi-Qar, Iraq
    Department of Information Technology, College of Computer Science and Mathematics, University of Thi-Qar, Thi-Qar, Iraq
  • Baidaa Mutasher Rashed , Baidaa Mutasher Rashed
    Department of Computers, Faculty of Computer Science and Mathematics, University of Thi-Qar, Nasiriyah, Thi-Qar, Iraq
    Department of Computers, Faculty of Computer Science and Mathematics, University of Thi-Qar, Nasiriyah, Thi-Qar, Iraq
  • Mohanad A. Kadhim Mohanad A. Kadhim
    Department of Artificial Intelligence, College of Science, University of Alkafeel, Najaf, Iraq
    Department of Artificial Intelligence, College of Science, University of Alkafeel, Najaf, Iraq
view PDF
view PDF

How to Cite

Aydam, Zahoor M., Baidaa Mutasher Rashed, and Mohanad A. Kadhim. 2026. “A Hierarchical Swin Transformer-Based Framework for Knee Osteoarthritis Severity Classification Using Radiographic Images”. Journal of Innovative Image Processing 8 (3): 1290-1312. https://doi.org/10.36548/jiip.2026.3.025.
ACM ACS APA ABNT Chicago Harvard IEEE MLA Turabian Vancouver AMA

Download Citation

Endnote/Zotero/Mendeley (RIS) BibTeX

Keywords

Knee Osteoarthritis
Swin Transformer
Deep Learning
Medical Image Classification
Radiographic Grading
Class Imbalance and Hierarchical Vision Transformer

Abstract

Knee osteoarthritis (KOA), one of the leading causes of musculoskeletal disease worldwide, results in severe pain, decreased physical activity, and poor quality of life particularly in the elderly population. Early evaluation of KOA staging is critical for treatment planning and decision-making for patients and clinicians. Currently, KOA grading mostly depends on subjective diagnosis based on orthopedic experience with radiograph analysis, which is inefficient and prone to inter-observer variability. In this work, a hierarchical deep learning framework based on Swin Transformer is presented for an automated KOA classification task from knee radiographs. The framework leverages hierarchical shifted-window self-attention to extract image features from local and global perspectives. It also utilizes a weighted random sampling strategy, a class-weighted cross-entropy loss function, and label smoothing to alleviate the class imbalance problem. The experiment is performed on a publicly available dataset with 8,260 knee radiographs containing five Kellgren-Lawrence (KL) grading categories: healthy, doubtful, minimal, moderate and severe knee osteoarthritis. In the experiment, a patient-wise dataset partitioning strategy is applied to address possible data leakage issues and provide a more reasonable assessment. The results showed that the proposed framework achieved 93.4% accuracy, 92.8% precision, 92.5% recall, and a 92.6% F1-score. Grad-CAM analysis indicates that the proposed framework attends to key KOA-related anatomical structures, which supports interpretation. Although the performance on the evaluated dataset shows it can achieve a competitive result, validation on multicenter datasets from different sources is still needed to fully demonstrate its generalization capability and robustness.

References

  1. World Health Organization. “Osteoarthritis.” WHO Fact Sheets. 2023. https://www.who.int/news-room/fact-sheets/detail/osteoarthritis.
  2. Kellgren, Jonas H., and JS1006995 Lawrence. "Radiological Assessment of Osteo-Arthrosis." Ann Rheum Dis 16, no. 4 (1957): 494-502.
  3. LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. "Deep Learning." nature 521, no. 7553 (2015): 436-444.
  4. He, Kaiming, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. "Deep Residual Learning for Image Recognition." In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, 770-778.
  5. Dosovitskiy, Alexey, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani et al. "An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale." arXiv preprint arXiv:2010.11929 (2020).
  6. Liu, Ze, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. "Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows." In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, 10012-10022.
  7. Tiulpin, Aleksei, Jérôme Thevenot, Esa Rahtu, Petri Lehenkari, and Simo Saarakkala. "Automatic Knee Osteoarthritis Diagnosis from Plain Radiographs: A Deep Learning-Based Approach." Scientific Reports 8, no. 1 (2018): 1727.
  8. Sekhri, Aymen, Mohamed A. Kerkouri, Aladine Chetouani, Marouane Tliba, Yassine Nasser, Rachid Jennane, and Alessandro Bruno. "Automatic Diagnosis of Knee Osteoarthritis Severity Using Swin Transformer." In Proceedings of the 20th International Conference on Content-Based Multimedia Indexing, 2023, 41-47.
  9. Antony, Joseph, Kevin McGuinness, Noel E. O'Connor, and Kieran Moran. "Quantifying Radiographic Knee Osteoarthritis Severity Using Deep Convolutional Neural Networks." In 2016 23rd international Conference on Pattern Recognition (ICPR), IEEE, 2016, 1195-1200.
  10. Górriz, Marc, Joseph Antony, Kevin McGuinness, Xavier Giró-i-Nieto, and Noel E. O’Connor. "Assessing Knee OA Severity with CNN Attention-Based End-to-End Architectures." In International conference on medical imaging with deep learning, PMLR, 2019, 197-214.
  11. Thomas, Kevin A., Łukasz Kidziński, Eni Halilaj, Scott L. Fleming, Guhan R. Venkataraman, Edwin HG Oei, Garry E. Gold, and Scott L. Delp. "Automated Classification of Radiographic Knee Osteoarthritis Severity Using Deep Neural Networks." Radiology: Artificial Intelligence 2, no. 2 (2020): e190065.
  12. Chen, Pingjun, Linlin Gao, Xiaoshuang Shi, Kyle Allen, and Lin Yang. "Fully Automatic Knee Osteoarthritis Severity Grading Using Deep Neural Networks with a Novel Ordinal Loss." Computerized Medical Imaging and Graphics 75 (2019): 84-92.
  13. Shamshad, Fahad, Salman Khan, Syed Waqas Zamir, Muhammad Haris Khan, Munawar Hayat, Fahad Shahbaz Khan, and Huazhu Fu. "Transformers in Medical Imaging: A Survey." Medical image analysis 88 (2023): 102802.
  14. Panwar, Punita, Sandeep Chaurasia, Jayesh Gangrade, and Ashwani Bilandi. "Early Diagnosis of Knee Osteoarthritis Severity Using Vision Transformer." BMC Musculoskeletal Disorders 26, no. 1 (2025): 884.
  15. Patel, Unnati, Sanskruti Patel, Dharmendra Patel, Niky Jain, Suchita Patel, and Ronesh Gangavani. 2026. “Hybrid CNN-Transformer for Knee Osteoarthritis Severity Grading”. Journal of Innovative Image Processing 8 (2): 617-43. https://doi.org/10.36548/jiip.2026.2.010.
  16. Sekhri, Aymen, Marouane Tliba, Mohamed Amine Kerkouri, Yassine Nasser, Aladine Chetouani, Alessandro Bruno, and Rachid Jennane. "Shifting Focus: From Global Semantics to Local Prominent Features in Swin-Transformer for Knee Osteoarthritis Severity Assessment." In 2024 32nd European Signal Processing Conference (EUSIPCO), IEEE, 2024, 1686-1690.
  17. Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. "Attention is All You Need." Advances in neural information processing systems 30 (2017).
  18. Szegedy, Christian, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. "Rethinking the Inception Architecture for Computer Vision." In Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, 2818-2826.
  19. Chen, Peng. Knee Osteoarthritis Severity Grading Dataset. Mendeley Data, Version 1, 2018. https://doi.org/10.17632/56rmx5bjcr.1.
  20. Ahmad, Isah Salim, Jingjing Dai, Yaoqin Xie, and Xiaokun Liang. "Deep Learning Models for CT Image Classification: A Comprehensive Literature Review." Quantitative Imaging in Medicine and Surgery 15, no. 1 (2025): 962-1011.