Optimal Resource Allocation in Cloud-Fog Environment by PSO-based DDQN Hierarchical Structure

Subject Areas : Information Technology in Engineering Design (ITED) Journal

Seyed Danial Alizadeh Javaheri ¹ , Reza Ghaemi ^{2 *} , Hossein Monshizadeh Naeen ³

1 - Department of Computer Engineering, Ne.C., Islamic Azad University, Neyshabur, Iran
2 - Department of Computer Engineering, Qu.C., Islamic Azad University, Quchan, Iran
3 - Department of Computer Engineering, Ne.C., Islamic Azad University, Neyshabur, Iran

Received: 2025-02-24 Accepted : 2025-05-17 Published : 2025-08-02

Keywords: Keywords: Resource Allocation, Reinforcement Learning, Double Q-Network, Particle Swarm Optimization, Cloud-Fog Space,

Abstract :

Abstract

The Internet of Things (IoT) technology has significantly expanded its presence in areas such as traffic management and health monitoring, increasing reliance on sensor data. This technology requires rapid and effective data processing, as delays in processing can reduce system efficiency. Utilizing cloud space for managing requests, particularly latency-sensitive requests, comes with challenges. Therefore, leveraging fog computing and user-side resources has been proposed as a solution to reduce latency and increase response speed. However, fog nodes have limited capacity, making optimal request management essential.

In this research, a deep reinforcement learning algorithm based on a double Q-network is used, with its hyperparameters updated by a particle swarm optimization algorithm. The results show that the average error function has decreased by 0.0005 at each stage, the request processing completion rate has increased, energy consumption has remained stable, and the exploration rate has decreased. These findings affirm the high efficiency of the proposed approach and highlight the key role of advanced algorithms in optimizing IoT networks. Employing this method could provide an effective infrastructure for managing requests in IoT systems.

References:

[1] Capponi, Andrea, Claudio Fiandrino, Burak Kantarci, Luca Foschini, Dzmitry Kliazovich, and Pascal Bouvry. "A survey on mobile crowdsensing systems: Challenges, solutions, and opportunities." IEEE communications surveys & tutorials 21, no. 3 (2019): 2419-2465.
[2] Lu, An-qi, and Jing-hua Zhu. "Worker recruitment with cost and time constraints in mobile crowd sensing." Future Generation Computer Systems 112 (2020): 819-831.
[3] Chen, Jianwei, Huadong Ma, Dong Zhao, and David SL Wei. "Participant density-independent location privacy protection for data aggregation in mobile crowd-sensing." Wireless Personal Communications 98 (2018): 699-723.
[4] Wang, J., Wang, Y., Zhang, D., Wang, F., Xiong, H., Chen, C., Lv, Q., Qiu, Z., 2018a. Multi-task allocation in mobile crowd sensing with individual task quality assurance. IEEE Transactions on Mobile Computing 17, 2101–2113. doi:10.1109/TMC.2018.2793908.
[5] Sutton, Richard S., and Andrew G. Barto. Reinforcement learning: An introduction. MIT press, 2018.
[6] Van Seijen, Harm, Mehdi Fatemi, and Arash Tavakoli. "Using a logarithmic mapping to enable lower discount factors in reinforcement learning." Advances in Neural Information Processing Systems 32 (2019).
[7] Pitis, Silviu. "Rethinking the discount factor in reinforcement learning: A decision theoretic approach." In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, pp. 7949-7956. 2019.
[8] Zhang, Daqing, Haoyi Xiong, Leye Wang, and Guanling Chen. "CrowdRecruiter: Selecting participants for piggyback crowdsensing under probabilistic coverage constraint." In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 703-714. 2014.
[9] Jie, Yingmo, Mingchu Li, Cheng Guo, and Ling Chen. "Game-theoretic online resource allocation scheme on fog computing for mobile multimedia users." China Communications 16, no. 3 (2019): 22-31.
[10] Guo, Cheng, Ningqi Luo, Md Zakirul Alam Bhuiyan, Yingmo Jie, Yuanfang Chen, Bin Feng, and Muhammad Alam. "Key-aggregate authentication cryptosystem for data sharing in dynamic cloud storage." Future Generation Computer Systems 84 (2018): 190-199.
[11] Ning, Zhaolong, Jun Huang, and Xiaojie Wang. "Vehicular fog computing: Enabling real-time traffic management for smart cities." IEEE Wireless Communications 26, no. 1 (2019): 87-93.
[12] Nassar, Almuthanna, and Yasin Yilmaz. "Reinforcement learning for adaptive resource allocation in fog RAN for IoT with heterogeneous latency requirements." IEEE Access 7 (2019): 128014-128025.
[13] Subbaraj, Saroja, Revathi Thiyagarajan, and Madavan Rengaraj. "A smart fog computing based real-time secure resource allocation and scheduling strategy using multi-objective crow search algorithm." Journal of Ambient Intelligence and Humanized Computing 14, no. 2 (2023): 1003-1015.
[14] Fan, Qiang, Jianan Bai, Hongxia Zhang, Yang Yi, and Lingjia Liu. "Delay-aware resource allocation in fog-assisted IoT networks through reinforcement learning." IEEE Internet of Things Journal 9, no. 7 (2021): 5189-5199.
[15] Leontiou, Nikolaos, Dimitrios Dechouniotis, Spyros Denazis, and Symeon Papavassiliou. "A hierarchical control framework of load balancing and resource allocation of cloud computing services." Computers & Electrical Engineering 67 (2018): 235-251.
[16] Huang, Liang, Xu Feng, Cheng Zhang, Liping Qian, and Yuan Wu. "Deep reinforcement learning-based joint task offloading and bandwidth allocation for multi-user mobile edge computing." Digital Communications and Networks 5, no. 1 (2019): 10-17
[17] Sundar, Sowndarya, and Ben Liang. "Offloading dependent tasks with communication delay and deadline constraint." In IEEE INFOCOM 2018-IEEE Conference on Computer Communications, pp. 37-45. IEEE, 2018.
[18] Hammoud, Ahmad, Azzam Mourad, Hadi Otrok, Omar Abdel Wahab, and Haidar Harmanani. "Cloud federation formation using genetic and evolutionary game theoretical models." Future Generation Computer Systems 104 (2020): 92-104.
[19] Sheng, Shuran, Peng Chen, Zhimin Chen, Lenan Wu, and Yuxuan Yao. "Deep reinforcement learning-based task scheduling in iot edge computing." Sensors 21, no. 5 (2021): 1666.
[20] Song, Zheng, Chi Harold Liu, Jie Wu, Jian Ma, and Wendong Wang. "QoI-aware multitask-oriented dynamic participant selection with budget constraints." IEEE Transactions on Vehicular Technology 63, no. 9 (2014): 4618-4632.
[21] Zhu, Xiaoyu, Yueyi Luo, Anfeng Liu, Wenjuan Tang, and Md Zakirul Alam Bhuiyan. "A deep learning-based mobile crowdsensing scheme by predicting vehicle mobility." IEEE Transactions on Intelligent Transportation Systems (2020).
[22] Li, Hanshang, Ting Li, and Yu Wang. "Dynamic participant recruitment of mobile crowd sensing for heterogeneous sensing tasks." In 2015 IEEE 12th International Conference on Mobile Ad Hoc and Sensor Systems, pp. 136-144. IEEE, 2015.
[23] Kalui, Dorothy Mwongeli, Dezheng Zhang, Geoffrey Muchiri Muketha, and Jared Okoyo Onsomu. "Simulation of trust-based mechanism for enhancing user confidence in mobile crowdsensing systems." IEEE Access 8 (2020): 20870-20883.
[24] Wu, Chu-ge, Wei Li, Ling Wang, and Albert Y. Zomaya. "Hybrid evolutionary scheduling for energy-efficient fog-enhanced internet of things." IEEE Transactions on Cloud Computing 9, no. 2 (2018): 641-653.
[25] Ding, Xuyang, Ruizhao Lv, Xiaoyi Pang, Jiahui Hu, Zhibo Wang, Xu Yang, and Xiong Li. "Privacy-preserving task allocation for edge computing-based mobile crowdsensing." Computers & Electrical Engineering 97 (2022): 107528.
[26] Buhussain, A. A., R. E. D. Grande, and A. Boukerche. "Performance analysis of Bio-Inspired scheduling algorithms for cloud." In IEEE International parallel and distributed processing symposium workshops, pp. 776-785. 2016.
[27] Verma, Abhishek, Luis Pedrosa, Madhukar Korupolu, David Oppenheimer, Eric Tune, and John Wilkes. "Large-scale cluster management at Google with Borg." In Proceedings of the tenth european conference on computer systems, pp. 1-17. 2015.
[28] Tirmazi, Muhammad, Adam Barker, Nan Deng, Md E. Haque, Zhijing Gene Qin, Steven Hand, Mor Harchol-Balter, and John Wilkes. "Borg: the next generation." In Proceedings of the fifteenth European conference on computer systems, pp. 1-14. 2020.
[29] Chen, Yanpei, Archana Sulochana Ganapathi, Rean Griffith, and Randy H. Katz. "Analysis and lessons from a publicly available google cluster trace." EECS Department, University of California, Berkeley, Tech. Rep. UCB/EECS-2010-95 94 (2010).
[30] Wang, Liang, Zhiwen Yu, Daqing Zhang, Bin Guo, and Chi Harold Liu. "Heterogeneous multi-task assignment in mobile crowdsensing using