IRO Journals
Journal of Innovative Image Processing

Facial Expression Recognition using Label-and Scale-Distribution-aware Loss-assisted Cross Vision Transformer

Open Access
Published: 16 October, 2026
Pages: 1509-1531
  • Bharathi Manchenahalli Gopalakrishna Rao , Bharathi Manchenahalli Gopalakrishna Rao
    Assistant Professor, Department of Computer Science, Vijaya College, Basavanagudi, Bangalore, India
    Assistant Professor, Department of Computer Science, Vijaya College, Basavanagudi, Bangalore, India
  • Divya Savithramma Ramakrishna Reddy Divya Savithramma Ramakrishna Reddy
    Assistant Professor, Department of Computer Science, Vijaya College, Basavanagudi, Bangalore, India
    Assistant Professor, Department of Computer Science, Vijaya College, Basavanagudi, Bangalore, India
view PDF
view PDF

How to Cite

Rao, Bharathi Manchenahalli Gopalakrishna, and Divya Savithramma Ramakrishna Reddy. 2026. “Facial Expression Recognition Using Label-and Scale-Distribution-Aware Loss-Assisted Cross Vision Transformer”. Journal of Innovative Image Processing 8 (4): 1509-31. https://doi.org/10.36548/jiip.2026.4.009.
ACM ACS APA ABNT Chicago Harvard IEEE MLA Turabian Vancouver AMA

Download Citation

Endnote/Zotero/Mendeley (RIS) BibTeX

Keywords

Attention Mechanism
Cross Vision Transformer
Facial Expression Recognition
Global Facial Structure
Label- and Scale- Distribution-aware Loss

Abstract

In recent years, Facial Expression Recognition (FER) has gained attention as a crucial area of computer vision. However, FER methods depend on single-source facial data that often experience reduced accuracy and instability because of challenges such as facial occlusion and variations in illumination. Moreover, existing FER approaches fail to understand the salient facial regions, resulting in poor interpretability and performance. Hence, this research proposes the Label- and Scale- Distribution-aware Loss-assisted Cross Vision Transformer (LSDL-CrossViT) to solve the issues in FER. The proposed model captures the global facial structure and local expression-specific cues through cross-attention-based feature learning. By incorporating the attention mechanism through discriminative loss design, the pipeline improves class separability and reduces classification errors across multiple facial expression categories. This research considers three benchmarks such as RAF-DB, AffectNet-8, and CK+, to estimate the complete performance of the proposed approach. The experimental results illustrate that the proposed LSDL-CrossViT gains a balanced accuracy of 93.18%, 71.84%, and 99.01% on the RAF-DB, AffectNet-8, and CK+ datasets, respectively, as compared to the reproduced Distract your Attention Network (DAN) and Self Learning Weight Network related to Label Distribution Training (SLW-LDT) models.

References

  1. Liao, Jun, Yuanchang Lin, Tengyun Ma, Songxiying He, Xiaofang Liu, and Guotian He. “Facial Expression Recognition Methods in the Wild Based on Fusion Feature of Attention Mechanism and LBP.” Sensors 23, no. 9 (2023): 4204.
  2. Qin, Lixiong, Mei Wang, Chao Deng, Ke Wang, Xi Chen, Jiani Hu, and Weihong Deng. “Swinface: A Multi-Task Transformer for Face Recognition, Expression Recognition, Age Estimation and Attribute Estimation.” IEEE Transactions on Circuits and Systems for Video Technology 34, no. 4 (2023): 2223-2234.
  3. Anand, M., Babu, S. “Predicting the Facial Expression Recognition Using Novel Enhanced Gated Recurrent Unit-Based Kookaburra Optimization Algorithm.” International Journal of Intelligent Engineering & Systems 17, no. 3 (2024): 159-170.
  4. Foteinopoulou, Niki Maria, and Ioannis Patras. “Emoclip: A Vision-Language Method for Zero-Shot Video Facial Expression Recognition.” In 2024 IEEE 18th international conference on automatic face and gesture recognition (FG), IEEE, 2024, 1-10.
  5. Hossain, Sanoar, Saiyed Umer, Ranjeet Kr Rout, and M. Tanveer. “Fine-grained Image Analysis for Facial Expression Recognition Using Deep Convolutional Neural Networks with Bilinear Pooling.” Applied Soft Computing 134 (2023): 109997.
  6. Putro, Muhamad Dwisnanto, Joko Hariyono, Oktavian A. Lantang, and Danilo Cáceres Hernández. “FER-MOTION: Facial Expression Recognizer in Multi-resolution Images Using a Lightweight Large Receptive Residual Network.” International Journal of Intelligent Engineering & Systems 18, no. 2 (2025): 650.
  7. Zhang, Fan, Gongguan Chen, Hua Wang, and Caiming Zhang. “CF-DAN: Facial-Expression Recognition Based on Cross-Fusion Dual-Attention Network.” Computational Visual Media 10, no. 3 (2024): 593-608.
  8. Chen, Dongliang, Guihua Wen, Huihui Li, Rui Chen, and Cheng Li. “Multi-relations Aware Network for in-the-Wild Facial Expression Recognition.” IEEE Transactions on Circuits and Systems for Video Technology 33, no. 8 (2023): 3848-3859.
  9. Aly, Mohammed. “Revolutionizing Online Education: Advanced Facial Expression Recognition for Real-Time Student Progress Tracking via Deep Learning Model.” Multimedia Tools and Applications 84, no. 13 (2025): 12575-12614.
  10. Zhang, Xiaoqin, Min Li, Sheng Lin, Hang Xu, and Guobao Xiao. “Transformer-based Multimodal Emotional Perception for Dynamic Facial Expression Recognition in the Wild.” IEEE Transactions on Circuits and Systems for Video Technology 34, no. 5 (2023): 3192-3203.
  11. Febrian, Rio, Benedic Matthew Halim, Maria Christina, Dimas Ramdhan, and Andry Chowanda. “Facial Expression Recognition Using Bidirectional LSTM-CNN.” Procedia Computer Science 216 (2023): 39-47.
  12. Pan, Hang, Lun Xie, and Zhiliang Wang. “C3DBed: Facial Micro-Expression Recognition with Three-Dimensional Convolutional Neural Network Embedding in Transformer Model.” Engineering Applications of Artificial Intelligence 123 (2023): 106258.
  13. Thuseethan, Selvarajah, Sutharshan Rajasegarar, and John Yearwood. ”Deep3DCANN: A Deep 3DCNN-ANN Framework for Spontaneous Micro-Expression Recognition.” Information Sciences 630 (2023): 341-355.
  14. Meena, Gaurav, Krishna Kumar Mohbey, Ajay Indian, Mohammad Zubair Khan, and Sunil Kumar. “Identifying Emotions from Facial Expressions Using a Deep Convolutional Neural Network-Based Approach.” Multimedia Tools and Applications 83, no. 6 (2024): 15711-15732.
  15. Sun, Zhe, Hehao Zhang, Jiatong Bai, Mingyang Liu, and Zhengping Hu. “A Discriminatively Deep Fusion Approach with Improved Conditional GAN (im-cGAN) for Facial Expression Recognition.” Pattern Recognition 135 (2023): 109157.
  16. AL-Abboodi, Rana H., and Ayad A. AL-Ani. “Facial Expression Recognition Based on GSO Enhanced Deep Learning in IOT Environment.” International Journal of Intelligent Engineering & Systems 17, no. 3 (2024): 445.
  17. Jabbooree, Abbas Issa, Leyli Mohammad Khanli, Pedram Salehpour, and Shahin Pourbahrami. “Geometrical Facial Expression Recognition Approach Based on Fusion CNN-SVM.” Int. J. Intell. Eng. Syst 17, no. 1 (2024): 457-468.
  18. Zhang, Saining, Yuhang Zhang, Ye Zhang, Yufei Wang, and Zhigang Song. “A Dual-Direction Attention Mixed Feature Network for Facial Expression Recognition.” Electronics 12, no. 17 (2023): 3595.
  19. Wen, Zhengyao, Wenzhong Lin, Tao Wang, and Ge Xu. “Distract Your Attention: Multi-Head Cross Attention Network for Facial Expression Recognition.” Biomimetics 8, no. 2 (2023): 199.
  20. Chen, Dan, Yu Cao, and Xu Cheng. “Research on Facial Expression Recognition Method Based on Improved ConvNeXt.” IET Image Processing 19, no. 1 (2025): e70118.
  21. Yousafzai, Samia Nawaz, Inzamam Mashood Nasir, Oumaima Saidani, Refka Ghodhbani, Yeonghyeon Gu, Muhammad Syafrudin, and Norma Latif Fitriyani. “A Multi-Scale Simplicial Transformer with Graph Attention for Facial Emotion Recognition.” Ain Shams Engineering Journal 16, no. 10 (2025): 103584.
  22. Chen, Yangbo, Chunyan Peng, Xuan Wang, and Yuhui Zheng. “Self‐learning Weight Network Based on Label Distribution Training for Facial Expression Recognition.” IET Image Processing 19, no. 1 (2025): e13326.
  23. Li, Shan, Weihong Deng, and JunPing Du. “Reliable Crowdsourcing and Deep Locality-Preserving Learning for Expression Recognition in the Wild.” In Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, 2852-2861.
  24. Mollahosseini, Ali, Behzad Hasani, and Mohammad H. Mahoor. “Affectnet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild.” IEEE transactions on affective computing 10, no. 1 (2017): 18-31.
  25. Lucey, Patrick, Jeffrey F. Cohn, Takeo Kanade, Jason Saragih, Zara Ambadar, and Iain Matthews. “The Extended Cohn-Kanade Dataset (ck+): A Complete Dataset for Action Unit and Emotion-Specified Expression.” In 2010 ieee computer society conference on computer vision and pattern recognition-workshops, IEEE, 2010, 94-101.