طراحی شبکه عصبی کانولوشن با وزن‌های موثر با استفاده از الگوریتم ژنتیک برای طبقه¬بندی تصاویر
محورهای موضوعی : مهندسی کامپیوتر و فناوری اطلاعاتسید مجتبی سجادی 1 , محمد باقر توکلی 2 * , فربد ستوده 3 , امیر حسین سالمی 4
1 - گروه مهندسی برق، واحد اراک، دانشگاه آزاد اسلامی، اراک، ایران
2 - گروه مهندس برق، واحد اراک، دانشگاه آزاد اسلامی، اراک، ایران
3 - گروه مهندسی برق، دانشگاه صنعتی اراک، اراک، ایران
4 - گروه مهندسی برق، واحد اراک، دانشگاه آزاد اسلامی، اراک، ایران
کلید واژه: شبکه عصبی کانولوشن , الگوریتم ژنتیک, وزن‌های موثر, طبقه¬بندی تصاویر,
چکیده مقاله :
شبکه‌های عصبی کانولوشن مهمترین شاخه یادگیری عمیق هستند و در سال‌های اخیر، توسعه سریعی را تجربه کرده¬اند. یک چالش عمده در استفاده از این شبکه‌ها، تعداد زیاد پارامترهای آن¬هاست که منجر به هزینه‌های محاسباتی و زمانی بالا در برنامه‌های کاربردی دنیای واقعی می‌شود. در بسیاری از موارد، این افزایش هزینه‌ها به دلیل طراحی شبکه‌های عمیق‌تر با پارامترهای بیشتر برای دستیابی به دقت بالاتر است. مقاله حاضر از الگوریتم‌های تکاملی برای معرفی روشی استفاده کرده که می‌تواند بهترین وزن‌ها را شناسایی کرده و از آنها برای ساخت شبکه‌های دقیق‌تر استفاده کند؛ در نتیجه نیاز به شبکه‌های عمیق‌تر را از بین می‌برد. در پایان مقاله، شبکه¬ی به دست آمده از الگوریتم پیشنهادی با بهترین شبکه‌های موجود مقایسه شده است که نشان می¬دهد شبکه¬ی پیشنهادی دقت طبقه¬بندی را افزایش داده است؛ در حالی‌که تعداد پارامترهای آن بسیار کمتر است و در نتیجه، باعث صرفه¬جویی در منابع محاسباتی و زمان می‌شود.
Convolutional neural networks (CNNs) are the most important branch of deep learning (DL) and have experienced rapid development in recent years. A major challenge in using these networks is their large number of parameters, which result in high computational and time costs in real-world applications. In many cases, this increase in costs is due to the design of deeper networks with more parameters for achieving higher accuracy. The present paper employed evolutionary algorithms (EAs) to introduce a method that can identify the best weights and use them to construct more accurate CNNs, hence eliminating the need for deeper networks. At the end of the article, the CNN obtained from the proposed algorithm is compared with the best existing CNNs; which shows that the proposed CNN has increased the classification accuracy, while the number of its parameters is much less, and as a result, it saves computing resources and time.
[1] Alzubaidi, L., et al.: Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. J Big Data 8, 53 (2021). https://doi.org/10.1186/s40537-021-00444-8.
[2] Dhillon, A., Verma, G.K.: Convolutional neural network: a review of models, methodologies and applications to object detection. Prog Artif Intell 9, 85–112 (2020). https://doi.org/10.1007/s13748-019-00203-0.
[3] Yao, G., Lei, T., Zhong, J.: A review of convolutional-neural-network-based action recognition. Pattern Recognition Letters, 118, 14-22 (2019). https://doi.org/10.1016/j.patrec.2018.05.018.
[4] Andonie, R.: Hyperparameter optimization in learning systems. Journal of Membrane Computing, 1(4), 279–291 (2019). https://doi.org/10.1007/s41965-019-00023-0
[5] Khalid, R., Javaid, N.: A survey on hyperparameters optimization algorithms of forecasting models in smart grid. Sustainable Cities and Society, 61, 102275 (2020). https://doi.org/10.1016/j.egyr.2022.09.188
[6] Pietron, M., Wielgosz, M.: Retrain or not retrain?-Efficient pruning methods of deep CNN networks. In Computational Science–ICCS 2020: 20th International Conference, Amsterdam, The Netherlands, June 3–5, 2020, Proceedings, Part III 20. Springer International Publishing, 452-463 (2020). http://dx.doi.org/10.1007/978-3-030-50420-5_34.
[7] Stojanovic, V., et al.: A service-oriented approach for classifying 3D points clouds by example of office furniture classification. In Proceedings of the 23rd International ACM Conference on 3D Web Technology.1-9 (2018). https://doi.org/10.1145/3208806.3208810.
[8] Chakraborty, S., et al.: Feature map reduction in cnn for handwritten digit recognition. In Recent Developments in Machine Learning and Data Analytics. Springer, 143-148 (2019). http://dx.doi.org/10.1007/978-981-13-1280-9_14.
[9] LeCun, Y., The MNIST database of handwritten digits. http://yann. lecun. com/exdb/mnist/, (1998).
[10] Han, S., H. Mao., Dally, W. J.: Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149, (2015).
[11] Choudhary, T., et al.: Inference-aware convolutional neural network pruning. Future Generation Computer Systems, 135, 44-56 (2022). https://doi.org/10.1016/j.future.2022.04.031.
[12] Molchanov, P., et al.: Pruning convolutional neural networks for resource efficient inference. arXiv preprint arXiv:1611.06440, (2016).
[13] Guo, Y., Yao, A., Chen, Y.: Dynamic network surgery for efficient dnns. Advances in neural information processing systems, 29 (2016). https://doi.org/10.48550/arXiv.1608.04493.
[14] Narkhede, M.V., P.P. Bartakke, and M.S. Sutaone, A review on weight initialization strategies for neural networks. Artificial intelligence review, 2022. 55(1): p. 291-322. https://arxiv.org/abs/2310.08109
[15] Xu, Z., et al., Initializing Models with Larger Ones. arXiv preprint arXiv:2311.18823, 2023.
https://arxiv.org/abs/2404.01383.
[16] Sun, Y., et al., Automatically designing CNN architectures using the genetic algorithm for image classification. IEEE transactions on cybernetics, 2020. 50(9): p. 3840-3854.
[17] Xie, Y., et al., Automated design of CNN architecture based on efficient evolutionary search. Neurocomputing, 2022. 491: p. 160-171.
[18] Ganesan, P., Sajiv, G.| A comprehensive study of edge detection for image processing applications. In 2017 international conference on innovations in information, embedded and communication systems (ICIIECS), IEEE, 1-6 (2017).
https://doi.org/10.1109/ICIIECS.2017.8275968.
[19] Tang, Y., et al.: Principal curvature measures estimation and application to 3D face recognition. Journal of Mathematical Imaging and Vision, 59(2), 211-233 (2017). https://doi.org/10.1007/s10851-017-0728-2.
[20] Possa, P.R., et al.: A multi-resolution FPGA-based architecture for real-time edge and corner detection. IEEE Transactions on Computers, 63(10), 2376-2388 (2013). http://doi.org/10.1109/TC.2013.130.
[21] Lowe, D.G.: Distinctive image features from scale-invariant keypoints. International journal of computer vision, 60(2), 91-110 (2004). https://doi.org/10.1023/B:VISI.0000029664.99615.94.
[22] Krizhevsky, A., Sutskever, I., Hinton, G. E.: Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84-90 (2017). http://dx.doi.org/10.1145/3065386.
[23] Russakovsky, O., et al.: Imagenet large scale visual recognition challenge. International journal of computer vision, 115(3), 211-252 (2015). http://dx.doi.org/10.1007/s11263-015-0816-y.
[24] Lin, T. Y., et al.: Microsoft coco: Common objects in context. In European conference on computer vision, Springer, 740-755 (2014). https://doi.org/10.1007/978-3-319-10602-1_48.
[25] Griffin, G., Holub, A., Perona, P.: Caltech-256 object category dataset, (2007).
[26] Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images. (2009).
[27] Perez, L., Wang, J.: The effectiveness of data augmentation in image classification using deep learning. ArXiv preprint arXiv:1712.04621, (2017).
[28] Everingham, M., et al.: The pascal visual object classes (voc) challenge. International journal of computer vision, 88(2), 303-338 (2010). https://doi.org/10.1007/s11263-009-0275-4.
[29] Miller, B. L., Goldberg, D. E.: Genetic algorithms, tournament selection, and the effects of noise. Complex systems, 9(3), 193-212 (1995). https://doi.org/10.1162/evco.1996.4.2.113.
[30] Blickle, T.: Tournament selection. Evolutionary computation, 1,181-186 (2000). http://dx.doi.org/10.1007/978-3-642-16493-4_19.
[31] Sun, Y., et al.: Automatically designing CNN architectures using the genetic algorithm for image classification. IEEE transactions on cybernetics, 50(9), 3840-3854 (2020). http://dx.doi.org/10.1109/TCYB.2020.2983860.
[32] Huang, G., et al.: Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, 4700-4708 (2017). http://dx.doi.org/10.1109/CVPR.2017.243.
[33] Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. ArXiv preprint arXiv:1409.1556, (2014).
[34] Singh, R.V.: ImageNet Winning CNN Architectures–A Review. Rajat Vikram Singh–Institue of Software Research at Carnegie Mellon University, (2015).
[35] Kumar, N., Kaur, N., Gupta, D.: Major convolutional neural networks in image classification: a survey. In Proceedings of International Conference on IoT Inclusive Life (ICIIL 2019), NITTTR Chandigarh, India, Springer, 243-258 (2020). http://dx.doi.org/10.1007/978-981-15-3020-3_23.
[36] He, K., et al.: Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770-778 (2016). http://dx.doi.org/10.1109/CVPR.2016.90.
[37] Chen, L., et al.: Review of image classification algorithms based on convolutional neural networks. Remote Sensing, 13(22), 4712 (2021). https://doi.org/10.3390/rs13224712.
[38] Xie, L., Yuille, A.: Genetic cnn. In Proceedings of the IEEE international conference on computer vision, (2017).
[39] Shorten, C., Khoshgoftaar, T. T.: A survey on image data augmentation for deep learning. Journal of big data, 6(1), 1-48 (2019). https://doi.org/10.1186/s40537-019-0197-0.
[40] Sun, Y., et al.: Evolving deep convolutional neural networks for image classification. IEEE Transactions on Evolutionary Computation, 24(2), 394-407 (2019). https://doi.org/10.48550/arXiv.1710.10741.