Solving the Multi-Objective Problem of IoT Service Placement in Fog Computing Using Reinforcement Learning Approaches
Subject Areas : Multimedia Processing, Communications Systems, Intelligent Systems
Zahra Saadati
1
,
Mani Zarei
2
1 - MSc. Student, Department of Computer Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran
2 - Assistant Professor, Department of Computer Engineering, Shahr-e-Qods Branch, Islamic Azad University, Tehran, Iran
Keywords: Internet of Things, Fog Computing, Service Placemen, Reinforcement Learning, Autonomous Planning.,
Abstract :
Introduction: The data generated in the Internet of Things (IoT) ecosystem requires continuous and timely processing. Transferring generated data to cloud data centers is costly and unsuitable for real-time applications. To increase the speed of service delivery, resources should be placed as close as possible to the user, i.e. at the edge of the network. A new paradigm called fog computing was introduced and added as a layer in the IoT architecture to meet this challenge. Fog computing provides the processing and storage of IoT data locally in the vicinity of IoT devices rather than in the cloud. Fog computing can provide less latency and better service quality for real-time applications than cloud computing. In general, there are theoretical foundations for fog computing, but the issue of locating IoT services to fog nodes remains a challenge and has attracted a great deal of research.
Method: In this research, a conceptual computing framework based on cloud-fog control software is proposed to optimally locate IoT services. The proposed model is formulated as an autonomous planning model for managing service requests due to some constraints, considering the heterogeneity of programs and resources. To solve the problem of locating IoT services, an autonomous evolutionary approach based on enhanced learning approaches has been proposed with the aim of making maximum use of fog-based resources and improving service quality. A heterogeneous advantage operator-criterion algorithm is used as a new reinforcement learning approach aimed at maximizing long-term cumulative reward.
Results: The results of the comparisons showed that the proposed reinforcement learning-enabled framework performs better than the advanced methods of the literature. The results of the proposed method compared to FSP-ODMA, SPP-GWO, CSA-FSPP, and GA-FSP methods indicate 4.6%, 2.4%, 3.4%, and 1.1% improvement, respectively.
Discussion: Experimental studies were performed on a simulated artificial environment based on various metrics including fog usage, services performed, response time, and service delay. The proposed reinforcement learning-enabled framework outperforms the previous works and shows better scalability. Analysis of parallel heuristic algorithms to find a more accurate localization than evolutionary approaches is another aspect of future work. We intend to consider new reinforcement learning approaches such as the Asynchronous Advantage Actor Critic (A3C) algorithm along with the long-term cumulative reward maximization policy for locating services. Also, future efforts will explore reinforcement learning approaches for failure recovery towards Cloud-Fog-IoT architecture, where parallel processing architecture of IoT services can be considered in the location process.
[1] Hassan, H. O., Azizi, S., & Shojafar, M. (2020). Priority, network and energy-aware placement of IoT-based application servicesin fog-cloud environments. IET communications, 14(13), 2117-2129.
[2] Brogi, A., Forti, S., & Ibrahim, A. (2018, March). Optimising QoS-assurance, resource usage and cost of fog application deployments. In International Conference on Cloud Computing and Services Science (pp. 168-189). Springer, Cham.
[3] Chen, Y., Li, Z., Yang, B., Nai, K., & Li, K. (2020). A Stackelberg game approach to multiple resources allocation and pricing in mobile edge computing. Future Generation Computer Systems, 108, 273-287.
[4] Dastjerdi, A. V., & Buyya, R. (2016). Fog computing: Helping the Internet of Things realize its potential. Computer, 49(8), 112-116.
[5] Ghobaei‐Arani, M., Souri, A., Safara, F., & Norouzi, M. (2020). An efficient task scheduling approach using moth‐flame optimization algorithm for cyber‐physical system applications in fog computing. Transactions on Emerging Telecommunications Technologies, 31(2), e3770.
[6] Goswami, P., Mukherjee, A., Maiti, M., Tyagi, S. K. S., & Yang, L. (2021). A Neural Network Based Optimal Resource Allocation Method for Secure IoT Network. IEEE Internet of Things Journal.
[7] Askarzadeh, A. (2016). A novel metaheuristic method for solving constrained engineering optimization problems: crow search algorithm. Computers & Structures, 169, 1-12.
[8] Ibrahim, A., Noshy, M., Ali, H. A., & Badawy, M. (2020). PAPSO: A power-aware VM placement technique based on particle swarm optimization. IEEE Access, 8, 81747-81764.
[9] Jia, B., Hu, H., Zeng, Y., Xu, T., & Yang, Y. (2018). Double-matching resource allocation strategy in fog computing networks based on cost efficiency. Journal of Communications and Networks, 20(3), 237-246.
[10] Karatas, F., & Korpeoglu, I. (2019). Fog-based data distribution service (F-DAD) for Internet of Things (IoT) applications. Future Generation Computer Systems, 93, 156-169.
[11] Etemadi, M., Ghobaei-Arani, M., & Shahidinejad, A. (2020). Resource provisioning for IoT services in the fog computing environment: An autonomic approach. Computer Communications, 161, 109-131.
[12] Nayeri, Z. M., Ghafarian, T., & Javadi, B. (2021). Application placement in Fog computing with AI approach: Taxonomy and a state of the art survey. Journal of Network and Computer Applications, 185, 103078.
[13] Yan, S., Jiao, M., Zhou, Y., Peng, M., & Daneshmand, M. (2020). Machine-learning approach for user association and content placement in fog radio access networks. IEEE Internet of Things Journal, 7(10), 9413-9425.
[14] Sami, H., Mourad, A., Otrok, H., & Bentahar, J. (2020, June). Fscaler: Automatic resource scaling of containers in fog clusters using reinforcement learning. In 2020 international wireless communications and mobile computing (IWCMC) (pp. 1824-1829). IEEE.
[15] Suryadevara, N. K. (2021). Energy and latency reductions at the fog gateway using a machine learning classifier. Sustainable Computing: Informatics and Systems, 31, 100582.
[16] Gazori, P., Rahbari, D., & Nickray, M. (2020). Saving time and cost on the scheduling of fog-based IoT applications using deep reinforcement learning approach. Future Generation Computer Systems, 110, 1098-1115.
[17] Guerrero, C., Lera, I., & Juiz, C. (2019). Evaluation and efficiency comparison of evolutionary algorithms for service placement optimization in fog architectures. Future Generation Computer Systems, 97, 131-144.
[18] Aburukba, R. O., AliKarrar, M., Landolsi, T., & El-Fakih, K. (2020). Scheduling Internet of Things requests to minimize latency in hybrid Fog–Cloud computing. Future Generation Computer Systems, 111, 539-551.
[19] Salimian, M., Ghobaei‐Arani, M., & Shahidinejad, A. (2021). Toward an autonomic approach for Internet of Things service placement using gray wolf optimization in the fog computing environment. Software: Practice and Experience, 51(8), 1745-1772.
[20] Skarlat, O., Schulte, S., Borkowski, M., & Leitner, P. (2016, November). Resource provisioning for IoT services in the fog. In 2016 IEEE 9th international conference on service-oriented computing and applications (SOCA) (pp. 32-39). IEEE.
[21] Goudarzi, M., Wu, H., Palaniswami, M., & Buyya, R. (2020). An application placement technique for concurrent IoT applications in edge and fog computing environments. IEEE Transactions on Mobile Computing, 20(4), 1298-1311.
[22] Goudarzi, M., Palaniswami, M. S., & Buyya, R. (2021). A distributed deep reinforcement learning technique for application placement in edge and fog computing environments. IEEE Transactions on Mobile Computing.
[23] Murtaza, F., Akhunzada, A., ul Islam, S., Boudjadar, J., & Buyya, R. (2020). QoS-aware service provisioning in fog computing. Journal of Network and Computer Applications, 165, 102674.
[24] Faraji Mehmandar, M., Jabbehdari, S., & Haj Seyyed Javadi, H. (2020). A dynamic fog service provisioning approach for IoT applications. International Journal of Communication Systems, 33(14), e4541.
[25] Skarlat, O., Nardelli, M., Schulte, S., Borkowski, M., & Leitner, P. (2017). Optimized IoT service placement in the fog. Service Oriented Computing and Applications, 11(4), 427-443.
[26] Khosroabadi, F., Fotouhi-Ghazvini, F., & Fotouhi, H. (2021). SCATTER: Service Placement in Real-Time Fog-Assisted IoT Networks. Journal of Sensor and Actuator Networks, 10(2), 26.
[27] Liu, C., Wang, J., Zhou, L., & Rezaeipanah, A. (2022). Solving the Multi-Objective Problem of IoT Service Placement in Fog Computing Using Cuckoo Search Algorithm. Neural Processing Letters. https://doi.org/10.1007/s11063-021-10708-2.
[28] Zhao, D., Zou, Q., & Boshkani Zadeh, M. (2022). A QoS-Aware IoT Service Placement Mechanism in Fog Computing Based on Open-Source Development Model. Journal of Grid Computing, 20(2), 1-29.
[29] Ghobaei-Arani, M., & Shahidinejad, A. (2022). A cost-efficient IoT service placement approach using whale optimization algorithm in fog computing environment. Expert Systems with Applications, 200, 117012.
[30] Jayadeva, S. M., Al Ayub Ahmed, A., Malik, R., Shaikh, A. A., Siddique, M., & Naved, M. (2023). Roles of Cloud Computing and Internet of Things in Marketing Management: A Critical Review and Future Trends. In Proceedings of Second International Conference in Mechanical and Energy Technology (pp. 165-173). Springer, Singapore.
[31] Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996). Reinforcement learning: A survey. Journal of artificial intelligence research, 4, 237-285.
[32] Natesha, B. V., & Guddeti, R. M. R. (2021). Adopting elitism-based Genetic Algorithm for minimizing multi-objective problems of IoT service placement in fog computing environment. Journal of Network and Computer Applications, 178, 102972.