Design and implementation of distributed systems for big data processing using artificial intelligence algorithms
Subject Areas : New technologies in distributed systems and algorithmic computing
1 - Department of Mathematics Education, Farhangian University, Tehran, Iran
Keywords: Distributed Systems, Big Data processing, Artificial Intelligence Algorithms, Performance Improvement, Systems Efficiency,
Abstract :
In this article, the design and implementation of distributed systems for big data processing using artificial intelligence algorithms have been thoroughly investigated. Due to the rapid and continuous growth of data volume in today's digital world, the utilization of these systems and AI-driven algorithms for efficient data processing has gained increasing importance. The results obtained from this study indicate that these methods can significantly enhance the performance of big data processing, offering substantial advantages for organizations and companies. This paper demonstrates that leveraging distributed systems in combination with artificial intelligence algorithms can lead to remarkable improvements in the performance, scalability, and overall efficiency of various systems, applications, and computational frameworks.
[1] Aminizadeh, S., Heidari, A., Toumaj, S., Darbandi, M., Navimipour, N. J., Rezaei, M., ... & Unal, M. (2023). The applications of machine learning techniques in medical data processing based on distributed computing and the Internet of Things. Computer methods and programs in biomedicine, 107745.
[2] Al-Jumaili, A. H. A., Muniyandi, R. C., Hasan, M. K., Paw, J. K. S., & Singh, M. J. (2023). Big data analytics using cloud computing based frameworks for power management systems: Status, constraints, and future recommendations. Sensors, 23(6), 2952.
[3] Khang, A., Gupta, S. K., Rani, S., & Karras, D. A. (Eds.). (2023). Smart Cities: IoT Technologies, big data solutions, cloud platforms, and cybersecurity techniques. CRC Press.
[4] Manikandan, N., Tadiboina, S. N., Khan, M. S., Singh, R., & Gupta, K. K. (2023, May). Automation of Smart Home for the Wellbeing of Elders Using Empirical Big Data Analysis. In 2023 3rd International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE) (pp. 1164-1168). IEEE.
[6] Hong, S. C. T.-L., S. D'Oca, D. Yan, S. P. (2016). Advances in research and applications of energy-related occupant behavior in buildings. Electronic Library, 116, 694-704.
[7] M. Denil, L. Bazzani, H. Larochelle, and N. de Freitas. Learning where to attend with deep architectures for image tracking. Neural computation, 24(8):2151–2184, 2012
[8] Chandrashekar, G., & Sahin, F. (2014). A survey on feature selection methods. Computers & Electrical Engineering, 40(1), 16-28.
[9] Chunduri, R. K., & Cherukuri, A. K. (2021). Scalable algorithm for generation of attribute implication base using FP-growth and spark. Soft Computing, 1-22.
[10] D’Oca, S., Chen, C. F., Hong, T., & Belafi, Z. . (2017). Synthesizing building physics with social psychology: An interdisciplinary framework for context and occupant behavior in office buildings. Energy research & social science, 34, 240-251.
[11] Fan, S. X., F. (2018). Mining big building operational data for improving building energy efficiency: a case study. Build. Serv. Eng. Res. Technol, 39, 117-128.
[12] Jordan, M. I., & Mitchell, T. M. (2015). Machine learning: Trends, perspectives, and prospects. Science, 349(6245), 255-260.
[13] Laender, A. H., Ribeiro-Neto, B. A., Da Silva, A. S., & Teixeira, J. S. (2002). A brief survey of web data extraction tools. ACM Sigmod Record, 31(2), 84-93.
Loshin, D. (2013). Business Intelligence (Second Edition):
[14] Morgan Kaufmann Mirmozaffari, M., Boskabadi, A., Azeem, G., Massah, R., Boskabadi, E., Dolatsara, H. A., & Liravian, A. (2020). Machine learning clustering algorithms based on the DEA optimization approach for banking system in developing countries. European Journal of Engineering and Technology Research, 5(6), 651-658.
[15] Nabilah, A., Devita, H. P., Van Halen, Y., & Jurizat, A. (2021). Energy Efficiency in Church Building Based on Sefaira Energy Use Intensity Standard. Paper presented at the IOP Conference Series: Earth and Environmental Science.
[16] Poelmans, J., Dedene, G., Verheyden, G., Van der Mussele, H., Viaene, S., & Peters, E. (2010). Combining business process and data discovery techniques for analyzing and improving integrated care pathways. Paper presented at the Industrial Conference on Data Mining.
[17] Qamar Shahbaz Ul Haq. (2016). Data Mapping for Data Warehouse Design: Morgan Kaufmann
[18] Qiu, F. F., Z. Li, G. Yang, P. Xu, Z. Li. (2019). Data mining based framework to identify rule based operation strategies for buildings with power metering system. Build. Simul, 12, 195-205.
[14] Salvador García, J. L., Francisco Herrera. (2014). Data Preprocessing in Data Mining: Springe
[15] Sherman, R. (2015). Business Intelligence Guidebook: Morgan Kaufmann.
Zhang. (2015). A New Data Transformation Method and Its Empirical Research Based on Inverted Cycloidal Kinetic Model. Procedia Computer Science, 55, 485-492.
[16] D. Held, S. Thrun, and S. Savarese. Learning to track at 100 fps with deep regression networks. arXiv preprint arXiv:1604.01802, 2016.
[17] Vatter, J., Mayer, R., & Jacobsen, H. A. (2023). The evolution of distributed systems for graph neural networks and their origin in graph processing and deep learning: A survey. ACM Computing Surveys, 56(1), 1-37.
[18] S. Hong, T. You, S. Kwak, and B. Han. Online tracking by learning discriminative saliency map with convolutional neural network. arXiv preprint arXiv:1502.06796, 2015.
[19] J. Johnson, A. Alahi, and L. Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. arXiv preprint arXiv:1603.08155, 2016.
[20] S. E. Kahou, V. Michalski, and R. Memisevic. Ratm: Recurrent attentive tracking model. arXiv preprint arXiv:1510.08660, 2015.
[21] M. Kristan, J. Matas, A. Leonardis, M. Felsberg, L. Cehovin, G. Fernandez, T. Vojir, G. Hager, G. Nebehay, and R. Pflugfelder. The visual object tracking vot2015 challenge results. In Proceedings of the IEEE International Conference on Computer Vision Workshops, pages 1–23, 2015.
[22] Olaniyi, O., Okunleye, O. J., & Olabanji, S. O. (2023). Advancing data-driven decision-making in smart cities through big data analytics: A comprehensive review of existing literature. Current Journal of Applied Science and Technology, 42(25), 10-18.
[23] Himeur, Y., Elnour, M., Fadli, F., Meskin, N., Petri, I., Rezgui, Y., ... & Amira, A. (2023). AI-big data analytics for building automation and management systems: a survey, actual challenges and future perspectives. Artificial Intelligence Review, 56(6), 4929-5021.
