Abstract
The research integrates Natural Language Processing (NLP) and facial micro-expressions recognition methods for analyzing deceptive behavior. Lie behavior analysis is enhanced by the incorporation of both verbal and non-verbal communication in the assessment as subtle non-verbal cues are hard to detect during scrutiny. Different machine learning algorithms were evaluated based on their ability to detect lies in this study. Several classic models like Nearest Neighbors, Linear SVM, Decision Tree, Random Forest and Extra Trees Classifier were tested using the Real-Life Deception Detection and Own Dataset student viva scenario data. Various accuracies were generated by different traditional ML models until researchers developed a lightweight Convolutional Neural Network (CNN) model designed to efficiently detect deception. The lite-CNN model achieved a successful 96% accuracy in both tests on the dataset. The lite-CNN model identifies deceptions through its high performance by combining verbal speech and facial behavioral patterns. It has been found that deception detection is successful when using NLP with facial expressions providing reasonable solutions in the fields of security, psychology, and human-computer interaction. The proposed lightweight CNN model is a proven solution compared to traditional models, as it is effective yet consumes fewer computing resources.
References
Wang, Hanpu, Ju Zhou, Xinyu Liu, Yingjuan Jia, and Tong Chen. "A Cross-Database Micro-Expression Recognition Framework based on Meta-Learning." Applied Intelligence 55, no. 1 (2025): 58.
Tseng, Philip, and Tony Cheng. "Artificial Intelligence in Lie Detection: Why do Cognitive Theories Matter?." New Ideas in Psychology 76 (2025): 101128.
Delmas, Hugues, Vincent Denault, Judee K. Burgoon, and Norah E. Dunbar. "A Review of Automatic Lie Detection from Facial Features." Journal of Nonverbal Behavior 48, no. 1 (2024): 93-136.
Nikbin, Sohiel, and Yanzhen Qu. "A Study on the Accuracy of Micro Expression Based Deception Detection with Hybrid Deep Neural Network Models." European Journal of Electrical Engineering and Computer Science 8, no. 3 (2024): 14-20.
Satpathi, Saswata, K. Mohamed Ismail Yasar Arafath, Aurobinda Routray, and Partha Sarathi Satpathi. "Analysis of Thermal Videos for Detection of Lie During Interrogation." EURASIP Journal on Image and Video Processing 2024, no. 1 (2024): 9.
D’Ulizia, Arianna, Alessia D’Andrea, Patrizia Grifoni, and Fernando Ferri. "Analysis, Evaluation, and Future Directions on Multimodal Deception Detection." Technologies 12, no. 5 (2024): 71.
King, Sayde L., and Tempestt Neal. "Applications of AI-Enabled Deception Detection Using Video, Audio, and Physiological Data: A Systematic Review." IEEE Access (2024).
Yadav, Rahul, Priyanka, and Priyanka Kacker. "AutoMEDSys: Automatic Facial Micro-Expression Detection System Using Random Fourier Features Based Neural Network." International Journal of Information Technology 16, no. 2 (2024): 1073-1086.
Kumar Tataji, Kadimi Naveen, Mukku Nisanth Kartheek, and Munaga VNK Prasad. "CC-CNN: A Cross Connected Convolutional Neural Network Using Feature Level Fusion for Facial Expression Recognition." Multimedia Tools and Applications 83, no. 9 (2024): 27619-27645.
Ahmed Khan, Hammad Ud Din, Usama Ijaz Bajwa, Naeem Iqbal Ratyal, Fan Zhang, and Muhammad Waqas Anwar. "Deception Detection in Videos Using the Facial Action Coding System." Multimedia Tools and Applications 84, no. 9 (2025): 6429-6443.
Manalu, Haposan Vincentius, and Achmad Pratama Rifai. "Detection of Human Emotions Through Facial Expressions Using Hybrid Convolutional Neural Network-Recurrent Neural Network Algorithm." Intelligent Systems with Applications 21 (2024): 200339.
Cash, Daniella K., Kayla D. Spenard, and Tiffany D. Russell. "Examining the Role of Speaker Familiarity and Statement Practice on Deception Detection." Journal of Social and Personal Relationships 41, no. 4 (2024): 931-951.
Talaat, Fatma M. "Explainable Enhanced Recurrent Neural Network for Lie Detection Using Voice Stress Analysis." Multimedia Tools and Applications 83, no. 11 (2024): 32277-32299.
Dinges, Laslo, Marc-André Fiedler, Ayoub Al-Hamadi, Thorsten Hempel, Ahmed Abdelrahman, Joachim Weimann, and Dmitri Bershadskyy. "Automated Deception Detection from Videos: Using End-To-End Learning Based High-Level Features and Classification Approaches." arXiv preprint arXiv:2307.06625 (2023).
Chauhan, Anjaly, and Shikha Jain. "FMeAR: FACS Driven Ensemble Model for Micro-Expression Action Unit Recognition." SN Computer Science 5, no. 5 (2024): 598.
De Marsico, Maria, Giordano Dionisi, and Donato Francesco Pio Stanco. "FTM: The Face Truth Machine—Hand-crafted Features from Micro-Expressions to Support Lie Detection." Computer Vision and Image Understanding 249 (2024): 104188.
Zhou, Yan, and Feng Bu. "Lie Detection Technology of Bimodal Feature Fusion Based on Domain Adversarial Neural Networks." IET Signal Processing 2024, no. 1 (2024): 7914185.
Abdulridha, Fahad, and Baraa M. Albaker. "Non-Invasive Real-Time Multimodal Deception Detection Using Machine Learning and Parallel Computing Techniques." Social Network Analysis and Mining 14, no. 1 (2024): 97.
Preethi, Thakkalapally, Saila Ram Choudalla, Sudeepthi Govathoti, K. Rajasri, Karrar Shareef Mohsen, and Dudi Bhanu Prakash. "The Future of Multimedia: Micro Facial Recognition in Advanced Systems." In 2024 IEEE 9th International Conference for Convergence in Technology (I2CT), IEEE, 2024, 1-6.
Sen, Monica, and Rébecca Deneckère. "Unmasking Lies: A Literature Review on Facial Expressions and Machine Learning for Deception Detection." Procedia Computer Science 246 (2024): 1925-1935.
Li, Yanfeng, Jincheng Bian, and Rencheng Song. "Video-based Deception Detection Using Wrapper-Based Feature Selection." In 2024 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications (CIVEMSA), IEEE, 2024, 1-5.
Stathopoulos, Anastasis, Ligong Han, Norah Dunbar, Judee K. Burgoon, and Dimitris Metaxas. "Deception Detection in Videos Using Robust Facial Features with Attention Feedback." In Handbook of Dynamic Data Driven Applications Systems: Volume 2, Cham: Springer International Publishing, 2023, 725-741.
Chebbi, Safa, and Sofia Ben Jebara. "Deception Detection Using Multimodal Fusion Approaches." Multimedia Tools and Applications 82, no. 9 (2023): 13073-13102.
Constâncio, Alex Sebastião, Denise Fukumi Tsunoda, Helena de Fátima Nunes Silva, Jocelaine Martins da Silveira, and Deborah Ribeiro Carvalho. "Deception Detection with Machine Learning: A Systematic Review and Statistical Analysis." Plos one 18, no. 2 (2023): e0281323.
D’Ulizia, Arianna, Alessia D’Andrea, Patrizia Grifoni, and Fernando Ferri. "Detecting Deceptive Behaviours Through Facial Cues from Videos: A Systematic Review." Applied Sciences 13, no. 16 (2023): 9188.
Nam, Borum, Joo Young Kim, Beomjun Bark, Yeongmyeong Kim, Jiyoon Kim, Soon Won So, Hyung Youn Choi, and In Young Kim. "FacialCueNet: Unmasking Deception-an Interpretable Model for Criminal Interrogation Using Facial Expressions: IY Kim et al." Applied Intelligence 53, no. 22 (2023): 27413-27427.
Alaskar, Haya. "Hybrid Metaheuristics with Deep Learning Enabled Automated Deception Detection and Classification of Facial Expressions." Computers, Materials & Continua 75, no. 3 (2023).
Yildirim, S., Chimeumanu, M.S., Rana, Z.A.: The Influence of Micro-Expressions on Deception Detection. Multimedia Tools and Applications. 82, 29115–29133 (2023).
Pérez-Rosas, V., Mihalcea, R.: Real-Life Deception Detection Dataset. University of Michigan. (2016). https://web.eecs.umich.edu/~mihalcea/downloads/RealLifeDeceptionDetection.2016.zip.
Patel, T., Vekairya, D.: Own Dataset (Student Viva). (2025). https://drive.google.com/uc?id=14i-A-ogp3Pc2RDL1Dqt9ENl0bz2qVHYr.