Adaptive Markov Decision Process-Based Congestion Control Using Deep Deterministic Policy Gradient in Distributed MQTT Brokers
Snowlin Preethi Janani , Immanuel Johnraja J., Getzi Jeba Leelipushpam P.
Open Access
Volume - 8 • Issue - 1 • march 2026
Abstract
The publish-subscribe variant of the Message Queuing Telemetry Transport (MQTT) protocol is lightweight enough to permit the asynchronous sending of messages. Nonetheless, the protocol is experiencing issues related to congestion, which is caused by the IoT traffic characteristics in the distributed broker networks in the IoT environment. As a result, delay occur resulting in a degradation of QoS. However, using classical AIMD and static rate-limiting techniques for congestion control in the IoT environment is becoming difficult. The proposed work provides an adaptive Reinforcement Learning (RL)-based system to handle congestion in the IoT environment. This system can be represented using a Markov Decision Process (MDP). The DDPG algorithm will allow the RL agent to discover optimal strategies in the IoT environment, and it is beneficial in managing non-stationary IoT traffic. Based on the agent clusters’ simulation of the brokers, the RL-based system outperforms the traditional system in the IoT setting. It achieves this as the message drop rate is reduced by 38-40%, the latency is reduced by 27-32%, and the fairness of the load is increased by 18-24%.
Cite this article
Janani, Snowlin Preethi, Immanuel Johnraja J., and Getzi Jeba Leelipushpam P.. "Adaptive Markov Decision Process-Based Congestion Control Using Deep Deterministic Policy Gradient in Distributed MQTT Brokers." Journal of Trends in Computer Science and Smart Technology 8, no. 1 (2026): 89-113. doi: 10.36548/jtcsst.2026.1.005
Copy Citation
Janani, S. P., J., I. J., & P., G. J. L. (2026). Adaptive Markov Decision Process-Based Congestion Control Using Deep Deterministic Policy Gradient in Distributed MQTT Brokers. Journal of Trends in Computer Science and Smart Technology, 8(1), 89-113. https://doi.org/10.36548/jtcsst.2026.1.005
Copy Citation
Janani, Snowlin Preethi, et al. "Adaptive Markov Decision Process-Based Congestion Control Using Deep Deterministic Policy Gradient in Distributed MQTT Brokers." Journal of Trends in Computer Science and Smart Technology, vol. 8, no. 1, 2026, pp. 89-113. DOI: 10.36548/jtcsst.2026.1.005.
Copy Citation
Janani SP, J. IJ, P. GJL. Adaptive Markov Decision Process-Based Congestion Control Using Deep Deterministic Policy Gradient in Distributed MQTT Brokers. Journal of Trends in Computer Science and Smart Technology. 2026;8(1):89-113. doi: 10.36548/jtcsst.2026.1.005
Copy Citation
S. P. Janani, I. J. J., and G. J. L. P., "Adaptive Markov Decision Process-Based Congestion Control Using Deep Deterministic Policy Gradient in Distributed MQTT Brokers," Journal of Trends in Computer Science and Smart Technology, vol. 8, no. 1, pp. 89-113, Mar. 2026, doi: 10.36548/jtcsst.2026.1.005.
Copy Citation
Janani, S.P., J., I.J. and P., G.J.L. (2026) 'Adaptive Markov Decision Process-Based Congestion Control Using Deep Deterministic Policy Gradient in Distributed MQTT Brokers', Journal of Trends in Computer Science and Smart Technology, vol. 8, no. 1, pp. 89-113. Available at: https://doi.org/10.36548/jtcsst.2026.1.005.
Copy Citation
@article{janani2026,
author = {Snowlin Preethi Janani and Immanuel Johnraja J. and Getzi Jeba Leelipushpam P.},
title = {{Adaptive Markov Decision Process-Based Congestion Control Using Deep Deterministic Policy Gradient in Distributed MQTT Brokers}},
journal = {Journal of Trends in Computer Science and Smart Technology},
volume = {8},
number = {1},
pages = {89-113},
year = {2026},
publisher = {IRO Journals},
doi = {10.36548/jtcsst.2026.1.005},
url = {https://doi.org/10.36548/jtcsst.2026.1.005}
}
Copy Citation
Keywords
MQTT
Internet of Things
Distributed Broker Networks
Congestion Control
Reinforcement Learning
Markov Decision Process
Quality of Service
Traffic Management
References
- Al-Sarawi, Shadi, Mohammed Anbar, Rosni Abdullah, and Ahmad B. Al Hawari. "Internet of Things Market Analysis Forecasts, 2020–2030." In 2020 Fourth World Conference on smart trends in systems, security and sustainability (WorldS4), IEEE, 2020, 449-453.
- Lakshminarayana, Sujitha, Amit Praseed, and P. Santhi Thilagam. "Securing the IoT Application Layer from an MQTT Protocol Perspective: Challenges and Research Prospects." IEEE Communications Surveys & Tutorials 26, no. 4 (2024): 2510-2546.
- Abujassar, Radwan S. "A Highly Effective Algorithm for Mitigating and Identifying Congestion Through Continuous Monitoring of IoT Networks, Improving Energy Consumption." Wireless Networks 30, no. 5 (2024): 3161-3180.
- Roy, Deepsubhra Guha, Bipasha Mahato, Debashis De, and Rajkumar Buyya. "Application-Aware End-To-End Delay and Message Loss Estimation in Internet of Things (IoT)—MQTT-SN Protocols." Future Generation Computer Systems 89 (2018): 300-316.
- Kurdi, Hassan, and Vijey Thayananthan. "Authentication Mechanisms for IoT system Based on Distributed MQTT Brokers: Review and Challenges." Procedia Computer Science 194 (2021): 132-139.
- D’Ortona, Cristian, Daniele Tarchi, and Carla Raffaelli. "Open-Source MQTT-Based End-To-End IoT System for Smart City Scenarios." Future Internet 14, no. 2 (2022): 57.
- Alshammari, Hamoud H. "The Internet of Things Healthcare Monitoring System Based on MQTT Protocol." Alexandria Engineering Journal 69 (2023): 275-287.
- Rodríguez Aguilar, Manuel José, Ismael Abad Cardiel, and José Antonio Cerrada Somolinos. "IIoT System for Intelligent Detection of Bottleneck in Manufacturing Lines." Applied Sciences 14, no. 1 (2023): 323.
- Schrab, Karl, Maximilian Neubauer, Robert Protzmann, Ilja Radusch, Stamatis Manganiaris, Panagiotis Lytrivis, and Angelos J. Amditis. "Modeling An Its Management Solution for Mixed Highway Traffic with Eclipse Mosaic." IEEE Transactions on Intelligent Transportation Systems 24, no. 6 (2022): 6575-6585.
- Vlahakis, Eleftherios, Raphaël Jungers, Nikolaos Athanasopoulos, and Seán McLoone. "AIMD-Inspired Switching Control of Computing Networks." IEEE Transactions on Control of Network Systems 11, no. 2 (2023): 683-695.
- Kanellopoulos, Dimitris, and Varun Kumar Sharma. "Dynamic Load Balancing Techniques in the IoT: A Review." Symmetry 14, no. 12 (2022): 2554.
- Böhm, Sebastian, and Guido Wirtz. "Cloud-Edge Orchestration for Smart Cities: A Review of Kubernetes-Based Orchestration Architectures." EAI Endorsed Transactions on Smart Cities 6, no. 18 (2022): e2.
- Alotaibi, Nouf Saeed, Hassan I. Sayed Ahmed, Samah Osama M. Kamel, and Ghada Farouk ElKabbany. "Secure Enhancement for MQTT Protocol Using Distributed Machine Learning Framework." Sensors 24, no. 5 (2024): 1638.
- Nguyen, Lam Tran Thanh, Son Xuan Ha, Trieu Hai Le, Huong Hoang Luong, Khanh Hong Vo, Khoi Huynh Tuan Nguyen, Tuan Anh Dao, and Hy Vuong Khang Nguyen. "BMDD: A Novel Approach for IoT Platform (Broker-Less and Microservice Architecture, Decentralized Identity, and Dynamic Transmission Messages)." PeerJ Computer Science 8 (2022): e950.
- Li, Zhuoran, Xing Wang, Ling Pan, Lin Zhu, Zhendong Wang, Junlan Feng, Chao Deng, and Longbo Huang. "Network Topology Optimization via Deep Reinforcement Learning." IEEE Transactions on Communications 71, no. 5 (2023): 2847-2859.
- Sumiea, Ebrahim Hamid, Said Jadid Abdulkadir, Hitham Seddig Alhussian, Safwan Mahmood Al-Selwi, Alawi Alqushaibi, Mohammed Gamal Ragab, and Suliman Mohamed Fati. "Deep Deterministic Policy Gradient Algorithm: A Systematic Review." Heliyon 10, no. 9 (2024).
- Azzedin, Farag, and Turki Alhazmi. "Secure Data Distribution Architecture in IoT Using MQTT." Applied Sciences 13, no. 4 (2023): 2515.
- Pinto Neto, Euclides Carlos, Somayeh Sadeghi, Xichen Zhang, and Sajjad Dadkhah. "Federated Reinforcement Learning in IoT: Applications, Opportunities and Open Challenges." Applied Sciences 13, no. 11 (2023): 6497.
- Usama, Muhammad, Ubaid Ullah, Zaid Muhammad, Muhammad Bux, Inam Ullah, Muhammad Rouf, and Taminul Islam. "Data Traffic Management in AI-IoT Network to Reduce Congestion." In artificial intelligence for intelligent systems, CRC Press, 2024, 145-189.
- Hintaw, Ahmed J., Selvakumar Manickam, Mohammed Faiz Aboalmaaly, and Shankar Karuppayah. "MQTT Vulnerabilities, Attack Vectors and Solutions in the Internet of Things (IoT)." IETE Journal of Research 69, no. 6 (2023): 3368-3397.
- Serdaroglu, Kemal Cagri, Sebnem Baydere, Boonyarith Saovapakhiran, and Chalermpol Charnsripinyo. "Q-IoT: QoS-Aware Multilayer Service Architecture for Multiclass IoT Data Traffic Management." IEEE Internet of Things Journal 11, no. 17 (2024): 28330-28340.
- Palmese, Fabio, Alessandro EC Redondi, and Matteo Cesana. "Adaptive Quality of Service Control for Mqtt-Sn." Sensors 22, no. 22 (2022): 8852.
- Buenrostro-Mariscal, Raymundo, Pedro C. Santana-Mancilla, Osval Antonio Montesinos-López, Mabel Vazquez-Briseno, and Juan Ivan Nieto-Hipolito. "Prioritization-Driven Congestion Control in Networks for the Internet of Medical Things: A Cross-Layer Proposal." Sensors 23, no. 2 (2023): 923.
- Detienne, Martin. "Master Thesis: MQTT broker with In-Line, Real-Time Data Visualiser for the Internet of Things (IoT)." (2022).
- Li, Mei, and Jing Ai. "Energy-Aware Clustering in the Internet of Things using Tabu Search and Ant Colony Optimization Algorithms." International Journal of Advanced Computer Science & Applications 14, no. 12 (2023).
