Automatic Location of Carvanserais in satellite Images using Image Processing Techniques based on Deep Learning

Subject Areas : Computer Engineering and IT

Mohammad Hossein Salari ¹ ( Interactive Technologies Lab, School of Computing, Faculty of Science, Forestry and Technology, University of Eastern Finland )
Mohammad Amin Shayegan ² ( Department of Computer Engineering, Shiraz Branch, Islamic Azad University, Shiraz, Iran )
Farnaz Faraji ³ ( Heritage and landscape, Faculty of Humanities, Vrije Universitiet Amsterdam, Research Assistant, School of History, Classics and Archaeology, Newcastle University )

Received: 2024-01-29 Accepted : 2024-02-20 Published : 2024-04-02

Keywords: Deep Learning, Carvansarai, Satellite Images, YOLOv5 Algorithm,

Abstract :

Despite the increasing use of deep learning in image processing, but the use of this technology in the fields related to cultural heritage, due to problems such as the lack of required databases, the cost of creating a new database, the complexity of working with deep neural networks, and etc., remains very limited. In this article, a method for the automatic location of caravanserais in satellite images has been introduced. For this purpose, a database of satellite images of 203 Iranian caravanserais was created and then, using the transfer learning technique, YOLOv5 (YOLOv5) object detecting algorithm was trained to locate caravanserais on the above database. To evaluate the efficiency of this network, 25 new images with dimensions of 2.5*2.5 square kilometers were selected from the surroundings of some caravanserais, and using the sliding window technique and the generated weights in the previous step, the operation of recognizing caravanserais was carried out on these images. In order to reduce the rate of misdiagnosis, the misdiagnosed locations were added to the database as new data and the YOLOv5 algorithm was retrained. The final accuracy of the proposed algorithm in finding the location of the caravanserai was 91.43% mAP_0.5.

References:

[۱] کیانی، م. ی. معماری ایران : دوران اسلامی. سازمان مطالعه و تدوین کتب علوم انسانی دانشگاهها (سمت) ، 1379 .
[۲] زنجانی، ف، مالیان، ع. "استخراج خودکار کاروانسراهای ایرانی از تصاویر ماهواره ای با بهره گیری از الگوریتم تناظریابی الگویی" ، سیزدهمین سمپوزیوم بین المللی پیشرفت های علوم و تکنولوژی:سرزمین پایدار، معماری و شهرسازی ، 1397.
[3] Belhi, Abdelhak, et al. "Deep learning and cultural heritage: the CEPROQHA project case study." 2019 13th International Conference on Software, Knowledge, Information Management and Applications (SKIMA). IEEE, 2019.
[4] Kersten, T. P., and M. Lindstaedt. "Potential of automatic 3D object reconstruction from multiple images for applications in architecture, cultural heritage and archaeology." International Journal of Heritage in the Digital Era 1.3 (2012): 399-420.
[5] Condorelli, Francesca, et al. "A neural networks approach to detecting lost heritage in historical video." ISPRS International Journal of Geo-Information 9.5 (2020): 297.
[۶] کیانی، م، کلایس، و . فهرست کاروانسراهای ایران ، جلد اول ، سازمان میراث فرهنگی کشور، 1363.
[7] Redmon, Joseph, et al. "You only look once: Unified, real-time object detection." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.
[8] Mohamed, Hussam El-Din, et al. "Msr-yolo: Method to enhance fish detection and tracking in fish farms." Procedia Computer Science 170 (2020): 539-546.
[9] Jocher, Glenn, et al. YOLOv5 . Zenodo. https://doi.org/10.5281/zenodo.5563715 (2021)
[10] Zhou, Junchi, et al. "Ship target detection algorithm based on improved YOLOv5." Journal of Marine Science and Engineering 9.8 (2021): 908.
[11] Yao, Jia, et al. "A real-time detection algorithm for Kiwifruit defects based on YOLOv5." Electronics 10.14 (2021): 1711.
[12] Fang, Yiming, et al. "Accurate and automated detection of surface knots on sawn timbers using YOLO-V5 model." BioResources 16.3 (2021): 5390.
[13] Wang, Chien-Yao, et al. "CSPNet: A new backbone that can enhance learning capability of CNN." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops. 2020.
[14] Powers, David MW. "Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation." arXiv preprint arXiv:2010.16061 (2020).
[15] Chen, Sheng, Bernard Mulgrew, and Peter M. Grant. "A clustering technique for digital communications channel equalization using radial basis function networks." IEEE Transactions on neural networks 4.4 (1993): 570-590.
[16] Rezatofighi, Hamid, et al. "Generalized intersection over union: A metric and a loss for bounding box regression." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019.
[۱۷] زنجانی، ف، مالیان، ع. "ملاحظات هندسی و پرتوسنجی برای آشکارسازی و مستندنگاری کاروانسراها" ، نخستین همایش ملی مستندنگاری میراث طبیعی و فرهنگی ، 1396.
[18] Darma, I. Wayan Agus Surya, Nanik Suciati, and Daniel Siahaan. "A performance comparison of balinese carving motif detection and recognition using YOLOv5 and mask R-CNN." 2021 5th International Conference on Informatics and Computational Sciences (ICICoS). IEEE, 2021.
[19] Llamas, Jose, et al. "Classification of architectural heritage images using deep learning techniques." Applied Sciences 7.10 (2017): 992.
[20] Oses, Noelia, Fadi Dornaika, and Abdelmalik Moujahid. "Image-based delineation and classification of built heritage masonry." Remote Sensing 6.3 (2014): 1863-1889.
[21] Groener, Austen, Gary Chern, and Mark Pritt. "A comparison of deep learning object detection models for satellite imagery." 2019 IEEE Applied Imagery Pattern Recognition Workshop (AIPR). IEEE, 2019.
[22] Balaniuk, R., O. Isupova, and S. Reece. "Mining and tailings dam detection in satellite imagery using deep learning. arXiv 2020." arXiv preprint arXiv:2007.01076.
[23] Wang, Tao, et al. "Landslide detection based on improved YOLOv5 and satellite images." 2021 4th International Conference on Pattern Recognition and Artificial Intelligence (PRAI). IEEE, 2021.
[24] Jindal, Manik, et al. "Aircraft Detection from Remote Sensing Images using YOLOV5 Architecture." 2022 6th International Conference on Devices, Circuits and Systems (ICDCS). IEEE, 2022.
[25] Zhanying, Zhang, and Chen Xinyuan. "Research on Forest Fire Detection Algorithm Based on Yolov5." 2021 International Conference on Intelligent Computing, Automation and Systems (ICICAS). IEEE, 2021.
[26] Hu, Mingdi, Yaqian Ren, and Haoxin Chai. "Forest Fire Detection Based on Improved YOLOv5." Proceedings of the 2021 4th International Conference on Artificial Intelligence and Pattern Recognition. 2021.
[۲۷] کیانی ، م، کلایس، ، و . فهرست کاروانسراهای ایران، جلد دوم، سازمان میراث فرهنگی کشور، 1386.
[28] LabelIm. Open Annotation Tool. http://labelme.csail.mit.edu/Release3.0. Last accessed 24 July 2021