Multishape Morphological-based Two-Stage CNN for LiDAR-DSM Classification
Subject Areas : International Journal of Smart Electrical Engineering
1 - Department of Photogrammetry and Remote Sensing, ّFaculty of Geodesy and Geomatics, K. N. Toosi University of Technology, Tehran, Iran
Keywords: LiDAR, Digital surface model (DSM), Morphological Profiles, Deep learning, Smart algorithm,
Abstract :
The classification of Digital Surface Model (DSM) images derived from LiDAR sensors is a challenging task, particularly when distinct ground classes with identical height information must be distinguished. However, DSM images contain valuable spatial information that can be utilized to enhance classification accuracy. This paper proposes a novel strategy, called Multishape Morphological Two-Stage Convolutional Neural Network (MM2CNN), for DSM classification to achieve accurate classified land-cover maps. The proposed method combines the strengths of multishape morphological profiles (MMPs) and a two-stage CNN model as a smart algorithm to effectively discriminate between different land covers from a single-band DSM image. More precisely, the CNN, as a smart method, is used to learn hierarchical rich representations of the data, while the MMPs are used to extract spatial information from the DSM imagery. The approach involves generating MMPs with three structuring elements, training three parallel CNN models, and then stacking and feeding the probability maps to a second-stage CNN to predict the final pixel labels. Experimental results on the Trento benchmark DSM image show that the suggested technique achieves an overall accuracy of 97.32% in a reasonable time, outperforming some other DSM classification methods. The success of the MM2CNN technique demonstrates the potential of integrating MMPs with CNN for precise DSM classification, which has a wide range of applications in environmental investigations.
[1] Michelini, D., Dalponte, M., Carriero, A., Kutchartt, E., Pappalardo, S. E., De Marchi, M., & Pirotti, F, "Hyperspectral and LiDAR data for the prediction via machine learning of tree species, volume and biomass: A contribution for updating forest management plans" Italian Conference on Geomatics and Geospatial Technologies; 2022: Springer. doi: 10.1007/978-3-031-17439-1_17
[2] Liu Y, Ye Z, Xi Y, Liu H, Li W, Bai L, "Multi-Scale and Multi-Direction Feature Extraction Network for Hyperspectral and LiDAR Classification" IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 2024, vol. 17, pp. 9961 - 9973, doi: 10.1109/JSTARS.2024.3400872.
[3] Zhang H, Yao J, Ni L, Gao L, Huang M, "Multimodal attention-aware convolutional neural networks for classification of hyperspectral and LiDAR data" IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 2022, vol.16, pp. 3635-3644, doi: 10.1109/JSTARS.2022.3187730.
[4] Du X, Zheng X, Lu X, Wang X, "Hyperspectral and LiDAR Representation with Spectral-Spatial Graph Network" IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 2023. vol. 16, pp. 9231 - 9245, doi: 10.1109/JSTARS.2023.3321776.
[5] Li J, Liu Y, Song R, Liu W, Li Y, Du Q, "HyperMLP: Superpixel Prior and Feature Aggregated Perceptron Networks for Hyperspectral and Lidar Hybrid Classification" IEEE Transactions on Geoscience and Remote Sensing. 2024, vol. 62, doi: 10.1109/TGRS.2024.3355037.
[6] Roy SK, Sukul A, Jamali A, Haut JM, Ghamisi P. "Cross hyperspectral and LiDAR attention transformer: An extended self-attention for land use and land cover classification" IEEE Transactions on Geoscience and Remote Sensing. 2024, vol. 62, doi: 10.1109/TGRS.2024.3374324.
[7] Hong D, Gao L, Hang R, Zhang B, Chanussot J, "Deep encoder–decoder networks for classification of hyperspectral and LiDAR data" IEEE Geoscience and Remote Sensing Letters. 2020, vol. 19, pp. 1-5, doi: 10.1109/LGRS.2020.3017414.
[8] Wang X, Feng Y, Song R, Mu Z, Song C, "Multi-attentive hierarchical dense fusion net for fusion classification of hyperspectral and LiDAR data" Information Fusion, 2022, vol. 82, pp. 1-18, doi: 10.1016/j.inffus.2021.12.008.
[9] Zhao G, Ye Q, Sun L, Wu Z, Pan C, Jeon B, "Joint classification of hyperspectral and LiDAR data using a hierarchical CNN and transformer" IEEE Transactions on Geoscience and Remote Sensing, 2022, vol.61, pp. 1-16, doi: 10.1109/TGRS.2022.3232498.
[10] Zhang M, Li W, Zhang Y, Tao R, Du Q, "Hyperspectral and LiDAR data classification based on structural optimization transmission" IEEE Transactions on Cybernetics. 2022, vol. 53, no. 5, pp. 3153 - 3164, doi: 10.1109/TCYB.2022.3169773.
[11] Anand R, Veni S, Geetha P, Subramoniam SR, "Extended morphological profiles analysis of airborne hyperspectral image classification using machine learning algorithms" International Journal of Intelligent Networks, 2021, vol. 2, pp. 1-6, doi: 10.1016/j.ijin.2020.12.006.
[12] Kumar B, Dikshit O, "Hyperspectral image classification based on morphological profiles and decision fusion" International Journal of Remote Sensing, 2017, vol.38, no.20, pp. 5830-54, doi: 10.1080/01431161.2017.1348636.
[13] Ghamisi P, Hoefle B, "LiDAR data classification using extinction profiles and a composite kernel support vector machine" IEEE Geoscience and Remote Sensing Letters, 2017, vol.14, no.5, pp. 659-63, doi: 10.1109/LGRS.2017.2669304.
[14] Wang A, He X, Ghamisi P, Chen Y, "LiDAR data classification using morphological profiles and convolutional neural networks" IEEE Geoscience and Remote Sensing Letters, 2018, vol.15, no.5, pp.774-8, doi: 10.1109/LGRS.2018.2810276.
[15] He X, Wang A, Ghamisi P, Li G, Chen Y, "LiDAR data classification using spatial transformation and CNN" IEEE Geoscience and Remote Sensing Letters, 2018, vol.16, no.1, pp.125-9, doi: 10.1109/LGRS.2018.2868378.
[16] Wang A, Wang M, Jiang K, Zhao L, Iwahori Y, "A novel lidar data classification algorithm combined densenet with STN" IEEE IGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Symposium;2019, doi: 10.1109/IGARSS.2019.8900313.
[17] Xie H, Chen Y, "LiDAR data classification based on automatic designed CNN" IEEE Geoscience and Remote Sensing Letters, 2020, vol.18, no.9, pp.1665-9, doi: 10.1109/LGRS.2020.3005209.
[18] Wang A, Wang M, Wu H, Jiang K, Iwahori Y. "A novel LiDAR data classification algorithm combined capsnet with resnet" Sensors, 2020, vol.20, no.4, pp.1151, doi: 10.3390/s20041151.
[19] Wu H, Cao M, Wang A, Wang M. "Classification of LiDAR data combined octave convolution with capsule network" IEEE Access, 2020, vol.8, pp.16155-65, doi: 10.1109/ACCESS.2020.2965278.
[20] Wang A, Xue D, Wu H, Iwahori Y, "LiDAR data classification based on improved conditional generative adversarial networks" IEEE Access. 2020 vol.8, pp.209674-86, doi: 10.1109/ACCESS.2020.3039211.
[21] Hariyono M, Tambunan M, Dewi R, "Support vector machine for land cover classification using lidar data" IOP Conference Series: Earth and Environmental Science; 2021, doi: 10.1088/1755-1315/873/1/012095.
[22] Asghari Beirami B, Mokhtarzade M, "Land Covers Classification from LiDAR-DSM Data Based on Local Kernel Matrix Features of Morphological Profiles" International Journal of Engineering, 2023, vol.36, no. 9, pp.1611-7, doi: 10.5829/IJE.2023.36.09C.04.
[23] Dong J, Liu K, Han J, Zhang M, Zhao X, Li W, X Li, M Rao, "Multi-scale Neighborhood Information Fusion Network for Classification of Remote Sensing LiDAR Images" IEEE Sensors Journal, 2024, vol.24, no.10, pp. 16601 - 16613, doi: 10.1109/JSEN.2024.3386173.
[24] Beirami BA, "Face Recognition based on Multi-shape Morphological Profiles-based Covariance Descriptors and Log-Etuclidean Kernel SVM" IEEE 9th Iranian Joint Congress on Fuzzy and Intelligent Systems (CFIS); 2022, doi: 10.1109/CFIS54774.2022.9756491.
[25] Sharifi O, Mokhtarzadeh M, Asghari Beirami B, "A new deep learning approach for classification of hyperspectral images: Feature and decision level fusion of spectral and spatial features in multiscale CNN" Geocarto International, 2022, vol.37, no.14, pp.4208-33, doi: 10.1080/10106049.2021.1882006.
[26] Beirami BA, Mokhtarzade M, "A new deep learning approach for hyperspectral image classification based on multifeature local kernel descriptors" Advances in Space Research, 2023, vol.72, no.5, pp.1703-20, doi: 10.1016/j.asr.2023.04.025.
[27] Beirami BA, Mokhtarzade M, "SVM classification of hyperspectral images using the combination of spectral bands and Moran's I features" 2017 IEEE 10th Iranian Conference on Machine Vision and Image Processing (MVIP), 2017, doi: 10.1109/IranianMVIP.2017.8342334.