A Multimodal Approach to Detect the Tuberculosis Using Deep Learning Technique - G., Saranya K.; A., Akalya (Journal of Soft Computing Paradigm, 2026)
The biggest challenges faced byTB (tuberculosis) patients are difficulty in managing TB particularly for those living in low socioeconomic areas and especially in rural communities. Chest X-rays are valuable in diagnosing TB because they are inexpensive and non-invasive for obtaining the data needed. There are many limitations to existing applications using deep learning algorithms to detect TB, including a limited number of images available for use (i.e., an X-ray or radiograph image of the chest), and these applications may require significant computational power. Additionally, many applications do not have a variety of characteristics to support the for diagnosis of TB. Existing deep learning TB detection methods do not provide interpretable output because they produce high rates of false negatives, especially when the disease is clinically unclear. This paper proposes a hybrid multimodal ensemble approach that uses both X-ray images and a systematic set of clinical symptoms to improve the accuracy of clinical-quality TB detection. By combining the advantages of the DenseNet and MobileNet architectures, this model is able to create additional radiographic features. It also uses graph neural networks to learn additional features related to the clinical symptoms, allowing the model to learn the contextual and clinical connections within the input data. The multimodal representations will be used to create overall diagnostic predictions using an attention-based ensemble to combine these representations. The overall prediction accuracy for the multimodal approach is 96% with precision/recall statistics nearly the same resulting in more robust and reduced false positive/false negative predictions compared to predictions from unimodal, image-based approaches. Examples of XAI techniques may be implemented to support transparency and build trust in clinical decisions using visualizations (e.g. Grad-CAM) to illustrate areas of the lung that contribute to the diagnosis of the disease. The Unified Multimodal TB Screening method will provide useful, interpretable and computationally efficient solutions to help automate TB diagnoses with the technologies needed to implement screening in real world, resource-constrained settings.