IRO Journals
Journal of Innovative Image Processing

Attentive Dual-Branch CNN–Transformer Fusion for Cross-Dataset Breast Histopathology Classification

Open Access
Published: 22 September, 2026
Pages: 1341-1369
  • Mradul Kumar Jain , Mradul Kumar Jain
    Department of Computer Science and Engineering, Noida International University, Gautam Budh Nagar, Uttar Pradesh, India
    Department of Computer Science and Engineering, Noida International University, Gautam Budh Nagar, Uttar Pradesh, India
  • Veerendra Yadav , Veerendra Yadav
    Department of Computer Science and Engineering, Noida International University, Gautam Budh Nagar, Uttar Pradesh, India
    Department of Computer Science and Engineering, Noida International University, Gautam Budh Nagar, Uttar Pradesh, India
  • Anu Chaudhary Anu Chaudhary
    Department of Computer Science and Engineering, Ajay Kumar Garg Engineering College, Ghaziabad, Uttar Pradesh, India
    Department of Computer Science and Engineering, Ajay Kumar Garg Engineering College, Ghaziabad, Uttar Pradesh, India
view PDF
view PDF

How to Cite

Jain, Mradul Kumar, Veerendra Yadav, and Anu Chaudhary. 2026. “Attentive Dual-Branch CNN–Transformer Fusion for Cross-Dataset Breast Histopathology Classification”. Journal of Innovative Image Processing 8 (4): 1341-69. https://doi.org/10.36548/jiip.2026.4.002.
ACM ACS APA ABNT Chicago Harvard IEEE MLA Turabian Vancouver AMA

Download Citation

Endnote/Zotero/Mendeley (RIS) BibTeX

Keywords

Attention Mechanism
Breast Cancer
Deep Learning
Histopathology
Transformer

Abstract

In general, breast histopathology classifiers exhibit significant performance degradation when applied to images acquired using different staining protocols, magnification rates, scanner equipment, or institutional environments. This study proposes an attentive dual-branch CNN–Transformer framework intended to learn representations that remain informative with respect to heterogeneity in histopathological data. The convolutional path captures fine cellular details, such as nuclear texture and local glandular structures, whereas the dilated convolution path captures more generalized tissue structures without extensive downscaling of spatial dimensions. Attention operations were applied separately to the two paths to diminish the impact of background areas and structures with low discriminative values. The generated feature maps were then represented in terms of spatial tokens, which were processed by a transformer encoder to represent the long-range dependencies between distant regions in the tissue structure. A learnable fusion gate regulates the contribution of the local and contextually informative parts of the input features for each image to the output. The institution-specific classification layers handle the distinct diagnostic classes in the BACH and BreaKHis datasets. The evaluation procedure included stratified testing on BACH and patient-disjoint partitioning on BreaKHis to exclude any potential data leakage at the subject level. For BACH, our model achieved an accuracy of 94.75%, a macro F1-score of 93.90%, and an area under the receiver operating characteristic curve of 96.35%, with 18.9 million trainable parameters and an average inference time of 16.1 ms per image. A component-wise ablation study, calibration evaluation, confidence interval estimation, and attention-driven visual explanations were conducted to investigate the contribution and reliability of the proposed fusion method. The resulting architecture represents a trade-off between prediction performance and computational costs but does not claim superior efficiency or applicability. Multi-institutional and prospective independent validation is necessary for further clinical application.

References

  1. Dasegowda, Mutthuraj, M. Brunda, M. Sajana, and Kanthesh M. Basalingappa. ”Breast Cancer Basics: Understanding the Disease and Its Challenges.” In Nano Theragnostics in Breast Cancer: Advances, Challenges, and Future Prospects, Singapore: Springer Nature Singapore, 2026, 51-88.
  2. Verghese, Gregory, Jochen K. Lennerz, Danny Ruta, Wen Ng, Selvam Thavaraj, Kalliopi P. Siziopikou, Threnesan Naidoo et al. ”Computational Pathology in Cancer Diagnosis, Prognosis, and Prediction–Present Day and Prospects.” The Journal of pathology 260, no. 5 (2023): 551-563.
  3. Yan, Shen, Menglong Feng, and Yue Cai. ”An Integrated Deep Learning Model with Enhanced EfficientNetB0 and MobileNetV1 for Diabetic Retinopathy Grading.” Biomedical Signal Processing and Control 113 (2026): 108915.
  4. Wang, Yaoli, Yaojun Deng, Yuanjin Zheng, Pratik Chattopadhyay, and Lipo Wang. ”Vision Transformers for Image Classification: A Comparative Survey.” Technologies 13, no. 1 (2025): 32.
  5. Balasubramanian, Aadhi Aadhavan, Salah Mohammed Awad Al-Heejawi, Akarsh Singh, Anne Breggia, Bilal Ahmad, Robert Christman, Stephen T. Ryan, and Saeed Amal. ”Ensemble Deep Learning-Based Image Classification for Breast Cancer Subtype and Invasiveness Diagnosis from Whole Slide Image Histopathology.” Cancers 16, no. 12 (2024): 2222.
  6. Tummala, Sudhakar, Jungeun Kim, and Seifedine Kadry. ”BreaST-Net: Multi-class Classification of Breast Cancer from Histopathological Images Using Ensemble of Swin Transformers.” Mathematics 10, no. 21 (2022): 4109.
  7. Wang, Xiaoli, Guowei Wang, Luhan Li, Hua Zou, and Junpeng Cui. ”MFF-ClassificationNet: CNN-transformer Hybrid with Multi-Feature Fusion for Breast Cancer Histopathology Classification.” Biosensors 15, no. 11 (2025): 718.
  8. Ashoka, D. V. ”Effivit: Hybrid CNN-Transformer for Retinal Imaging.” Computers in Biology and Medicine 191 (2025): 110164.
  9. Mehmood, Shahid, Muhammad Zubair, Sagheer Abbas, Rowida Mohammed Alharbi, Mai Alduailij, Muhammad Adnan Khan, and Taher M. Ghazal. ”Automated Bone Marrow Cell Classification Using Ensemble Learning: Performance, Generalization, And Clinical Interpretability.” Frontiers in Medicine 13 (2026): 1787948.
  10. Jahanifar, Mostafa, Manahil Raza, Kesi Xu, Trinh Thi Le Vuong, Robert Jewsbury, Adam Shephard, Neda Zamanitajeddin et al. ”Domain Generalization in Computational Pathology: Survey and Guidelines.” ACM Computing Surveys 57, no. 11 (2025): 1-37.
  11. Vulli, Srinivas, Ming Liu, and Zhenyu Huang. ”Reliability-Aware Cross-Modal Fusion of Sentinel-1 SAR and Sentinel-2 Optical Imagery for Robust Patch-Level Land Cover Scene Classification.” Remote Sensing 18, no. 15 (2026): 2484.
  12. Tekale, Aditya, and Mohammad Masum. ”Clinically Aligned Long-Context Transformers for Cross-Platform Mental Health Risk Detection.” Electronics 15, no. 7 (2026): 1403.
  13. Santos, Luís Otávio, Aldo von Wangenheim, Felipe Soares Muylaert Barroso, Fabiana Botelho de Miranda Onofre, Alexandre Sherlley Casimiro Onofre, and Felipe Perozzo Daltoé. ”A Systematic Literature Review on Vision Transformers Applications in Histopathology and Cytopathology: Advances in Cellular Analysis.” (2024).
  14. Kuppusamy, Uma. ”A Review of Current Imaging Techniques for Histopathology-Based Breast Cancer Diagnosis with Comparative Insights Using Machine Learning and Deep Learning Models and its Challenges and Future Directions.” PeerJ Computer Science 12 (2026): e3645.
  15. Zhou, Xiaomin, Chen Li, Md Mamunur Rahaman, Yudong Yao, Shiliang Ai, Changhao Sun, Qian Wang et al. ”A Comprehensive Review for Breast Histopathology Image Analysis Using Classical and Deep Neural Networks.” IEEE Access 8 (2020): 90931-90956.
  16. Rakhlin, Alexander, Alexey Shvets, Vladimir Iglovikov, and Alexandr A. Kalinin. ”Deep Convolutional Neural Networks for Breast Cancer Histology Image Analysis.” In International conference image analysis and recognition, Cham: Springer International Publishing, 2018, 737-744.
  17. Liu, Suxing, Galib Muhammad Shahriar Himel, and Jiahao Wang. ”Breast Cancer Classification with Enhanced Interpretability: Dalaresnet50 and Dt Grad-Cam.” IEEE Access 12 (2024): 196647-196659.
  18. Baroni, Giulia Lucrezia, Laura Rasotto, Kevin Roitero, Angelica Tulisso, Carla Di Loreto, and Vincenzo Della Mea. ”Optimizing Vision Transformers for Histopathology: Pretraining and Normalization in Breast Cancer Classification.” Journal of Imaging 10, no. 5 (2024): 108.
  19. Yan, Yuting, Ruidong Lu, Jian Sun, Jianxin Zhang, and Qiang Zhang. ”Breast Cancer Histopathology Image Classification Using Transformer with Discrete Wavelet Transform.” Medical Engineering & Physics 138, no. 1 (2025): 104317.
  20. Sreelekshmi, V., Keechilat Pavithran, and Jyothisha J. Nair. ”Swincnn: An Integrated Swin Transformer and CNN for Improved Breast Cancer Grade Classification.” IEEE Access 12 (2024): 68697-68710.
  21. Wang, Xiaoli, Guowei Wang, Luhan Li, Hua Zou, and Junpeng Cui. ”MFF-ClassificationNet: CNN-transformer Hybrid with Multi-Feature Fusion for Breast Cancer Histopathology Classification.” Biosensors 15, no. 11 (2025): 718.
  22. Liu, Suxing, and Byungwon Min. ”DCS-ST for Classification of Breast Cancer Histopathology Images with Limited Annotations.” Applied Sciences 15, no. 15 (2025): 8457.
  23. Mahmood, Tariq, Tanzila Saba, Shaha Al‐Otaibi, Noor Ayesha, and Ahmed S. Almasoud. ”AI‐Driven microscopy: Cutting‐Edge Approach for Breast Tissue Prognosis Using Microscopic Images.” Microscopy Research and Technique 88, no. 5 (2025): 1335-1359.
  24. Yao, Xiang, and Fujun Liu. ”Weak Feature Masking for Fine-Grained Breast Cancer Histopathological Image Classification Under Small Dataset Constraints.” Biomedical Signal Processing and Control 126 (2026): 111001.