Multimodal Lie Detection Using Linguistic and Visual Cues: A Fusion of NLP and Facial Micro-Feature Analysis - Patel, Twisha; Vekariya, Daxa (Journal of Innovative Image Processing, 2025)
The research integrates Natural Language Processing (NLP) and facial micro-expressions recognition methods for analyzing deceptive behavior. Lie behavior analysis is enhanced by the incorporation of both verbal and non-verbal communication in the assessment as subtle non-verbal cues are hard to detect during scrutiny. Different machine learning algorithms were evaluated based on their ability to detect lies in this study. Several classic models like Nearest Neighbors, Linear SVM, Decision Tree, Random Forest and Extra Trees Classifier were tested using the Real-Life Deception Detection and Own Dataset student viva scenario data. Various accuracies were generated by different traditional ML models until researchers developed a lightweight Convolutional Neural Network (CNN) model designed to efficiently detect deception. The lite-CNN model achieved a successful 96% accuracy in both tests on the dataset. The lite-CNN model identifies deceptions through its high performance by combining verbal speech and facial behavioral patterns. It has been found that deception detection is successful when using NLP with facial expressions providing reasonable solutions in the fields of security, psychology, and human-computer interaction. The proposed lightweight CNN model is a proven solution compared to traditional models, as it is effective yet consumes fewer computing resources.
Keywords: Lie Detection; Natural Language Processing; Facial Micro-Features; Machine Learning; Convolutional Neural Network