Abstract
Parkinson’s Disease (PD) is a neurological disorder that causes patients with Parkinson’s Disease (PPD) to have difficulty with body balancing. Thus, PPD rely on caretakers to fulfill their daily needs. Vision-based assistive systems could be useful for PPD to communicate with caretakers. The work presented here is a hybrid YOLOv8n-seg framework with DETR. In this framework, the traditional YOLOv8n-seg model’s head is replaced with DETR as the head for hand gesture (HG) segmentation for PPD. Since no public dataset exists for PPD gestures, a dataset of 4,583 raw hand gesture images was collected with a webcam under realistic home and clinical environments (such as poor light, cluttered background, and motion blur) and expanded via augmentation to 11,230 gestures. This dataset was divided into an 80% training set, a 15% validation set, and a 5% testing set with 9 classes (e.g., hungry, attention, call, toilet) to ensure robust evaluation. The baseline YOLOv8n-seg model and Transformer-based variant, DETR (DEtection TRansformer), were tested on the custom PPD hand gesture dataset. Compared to the baseline YOLOv8n-seg, the implemented hybrid model achieved superior performance across all evaluation metrics, with ~1% improvement in precision (99% vs. 98%), recall (97% vs. 96%), F1 score (98% vs. 97%), and dice score (98% vs. 97%), with almost the same mAP@50 (97% vs. 97% for all), while improving inference speed by +3.0% (55.1 FPS vs. 53.5 FPS). On the same custom dataset, the conventional U-Net achieved 88% precision, 92% recall, and a 0.9 dice score, whereas the proposed hybrid model reached 99% precision, 98% recall, and a 0.98 dice score. This confirms the superior performance of the hybrid model over the conventional U-Net architecture for HG segmentation. The Raspberry Pi 4B is used as an edge device for HGR of PPD. These enhancements demonstrate that the hybrid approach achieves both higher accuracy and faster real-time performance, which is useful for assistive systems deployment on the embedded edge device. To our knowledge, this is the first work combining YOLOv8n-seg with a DETR head for PPD hand gesture segmentation.
References
Telepatil, Avadhoot, and Jayashree Vaddin. "Various Optimizers' Performances for CNN-Based Hand Gesture Recognition for PDP Assistance." In 2023 14th International Conference on Computing Communication and Networking Technologies (ICCCNT), IEEE, (2023): 1-7.
Di Libero, Tommaso, Elisa Langiano, Chiara Carissimo, Maria Ferrara, Pierluigi Diotaiuti, and Angelo Rodio. "Technological support for people with Parkinson’s disease: A narrative review." Journal of Gerontology and Geriatrics 71 (2023): 87-101.
Deb, Ranadeep, Sizhe An, Ganapati Bhat, Holly Shill, and Umit Y. Ogras. "A systematic survey of research trends in technology usage for Parkinson’s disease." Sensors 22, no. 15 (2022): 5491.
Duffy, Orla, Jonathan Synnott, Roisin McNaney, Paola Brito Zambrano, and W. George Kernohan. "Attitudes toward the use of voice-assisted technologies among people with Parkinson disease: findings from a web-based survey." JMIR rehabilitation and assistive technologies 8, no. 1 (2021): e23006.
Kour, Navleen, Sunanda Gupta, and Sakshi Arora. "Sensor technology with gait as a diagnostic tool for assessment of Parkinson’s disease: a survey." Multimedia Tools and Applications 82, no. 7 (2023): 10211-10247.
Dineshkumar, V., D. Raveena Judie Dolly, D. J. Jagannath, and J. Dinesh Peter. "Assistive methodologies for Parkinson's disease tremor management—a health opinion." Frontiers in Public Health 10 (2022): 850805.
Deb, Ranadeep, Ganapati Bhat, Sizhe An, Holly Shill, and Umit Y. Ogras. "Trends in technology usage for Parkinson’s disease assessment: a systematic review." MedRxiv (2021): 2021-02.
Wang, Peng, Hong Deng, Jiaxu Guo, Siqi Ji, Dan Meng, Jun Bao, and Peng Zuo. "Leaf segmentation using modified YOLOv8-seg models." Life 14, no. 6 (2024): 780.
Alsuwaylimi, Amjad A. "Enhanced YOLOv8-Seg instance segmentation for real-time submerged debris detection." IEEE Access (2024).
Hassanudin, Wahyu Maulana, Victor Gayuh Utomo, and Riski Apriyanto. "Fine-Grained Analysis of Coral Instance Segmentation using YOLOv8 Models." Sinkron: jurnal dan penelitian teknik informatika 8, no. 2 (2024): 1047-1055.
Casas, Edmundo, Leo Ramos, Cristian Romero, and Francklin Rivas-Echeverria. "A comparative study of YOLOv5 and YOLOv8 for corrosion segmentation tasks in metal surfaces." Array 22 (2024): 100351.
Bai, Ruihan, Mingkang Wang, Zhiping Zhang, Jiahui Lu, and Feng Shen. "Automated construction site monitoring based on improved YOLOv8-seg instance segmentation algorithm." IEEE Access 11 (2023): 139082-139096.
Sapkota, Ranjan, Dawood Ahmed, and Manoj Karkee. "Comparing YOLOv8 and Mask R-CNN for instance segmentation in complex orchard environments." Artificial Intelligence in Agriculture 13 (2024): 84-99.
Li, Xiangtai, Henghui Ding, Haobo Yuan, Wenwei Zhang, Jiangmiao Pang, Guangliang Cheng, Kai Chen, Ziwei Liu, and Chen Change Loy. "Transformer-based visual segmentation: A survey." IEEE transactions on pattern analysis and machine intelligence (2024).
Carion, Nicolas, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. "End-to-end object detection with transformers." In European conference on computer vision, Cham: Springer International Publishing, (2020): 213-229.
Tian, Hao, Yu Cao, and P. Y. Mok. "Detr-based layered clothing segmentation and fine-grained attribute recognition." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, (2023): 3535-3539.
Zhang, Baocan, Xiaolu Jiang, and Wei Zhao. "An enhanced mask transformer for overlapping cervical cell segmentation based on DETR." IEEE Access (2024).
Jiang, Jiaran, Xin Zuo, Xin Shu, Dan Xu, and Ping Qian. "UW-DETR: Underwater Image Instance Segmentation with Co-DETR." In 2024 7th International Conference on Pattern Recognition and Artificial Intelligence (PRAI), IEEE, (2024): 125-130.
Kadi, Hocine, Théo Sourget, Marzena Kawczynski, Sara Bendjama, Bruno Grollemund, and Agnés Bloch-Zupan. "Segmentation, and Numbering in Oral Rare Diseases: Focus on Data Augmentation and Inpainting Techniques." In 2023 International Conference on Computational Science and Computational Intelligence (CSCI), IEEE, (2023): 1358-1363.
Wang, Yucheng, Huiping Wang, Hang Mao, Suwei Gao, Qiaofeng Wei, Shujing Li, Rangtong Liu, and Boyang Xu. "Image Segmentation Method for Water Transport Feature Detection in Fabrics via Target-Located Strategy." IEEE Access (2025).
Tang, Chunming, Xinguang Dai, Jinghong Liu, and Yan Li. "SFED-Former: A Symmetrical Feature Enhanced DETR for Video Lane Detection." In 2023 2nd International Conference on Artificial Intelligence, Human-Computer Interaction and Robotics (AIHCIR), IEEE, (2023): 121-125.
Bai, Ruihan, Mingkang Wang, Zhiping Zhang, Jiahui Lu, and Feng Shen. "Automated construction site monitoring based on improved YOLOv8-seg instance segmentation algorithm." IEEE Access 11 (2023): 139082-139096.