Extraction of Multiple Hybrid Features to Reduce the Semantic Vacuum with the Semi-Supervised Classification
Subject Areas : Electronics Engineering
Mahdi Jalali
1
,
Tohid Sedghi
2
1 - Department of Electrical Engineering, Naghadeh Branch, Islamic Azad University, Naghadeh , Iran
2 - Department of Electrical Engineering, Urmia Branch, Islamic Azad University, Urmia, Iran
Keywords: Image retrieval, Semi-supervised classification, annotation, Feature,
Abstract :
In this paper, for the classification of images, the observed cooperative classification method is proposed with the aim of reducing the semantic vacuum. Because most classification methods are sensitive to cluster centers at initialization, the algorithm converges optimally locally if the quantification is not done correctly. It is also very difficult to combine the results of the classification due to the fact that the labels of the work centers are not clear. Supervised semi-classification is used to solve these problems. To achieve the highest performance, the system classification results are combined with different color space and similarity criteria with multiple features as a semi-supervised cooperative. When the number of features is effective, the relevant feedback is used for its semi-regulatory classification. One of the most important parts of an image retrieval system and a classification algorithm is to determine the appropriate similarity between images. In this study, two methods of classification of cases have been used, which include k-NN and PNN classification, which according to the results in all proposed methods, a better response from k-NN classification than PNN has been observed. The proposed algorithm is very suitable for classifying large image databases due to the reduction of time complexity. Also, in image retrieval in different color spaces and using different similarity criteria, different classifications are obtained. Better results can be achieved if the classification results are combined.
[1] F. Cao , J. Liang and B. Liang, “ A new initialization method for categorical data clustering,” Expert Systems with Applications., vol 36,no.7, pp. 10223–10228, 2009 ,doi:10.1016/j.eswa.2009.01.060.
[2] A. Amato and V. Lecce, “ A knowledge based approach for a fast image retrieval system,” Image and Vision Computing vol 26,no.11, pp. 1466–1480, 2008, doi:10.1016/j.imavis.2008.01.005.
[3] Y. Chen, J. Z. Wang and R. Krovetz, "CLUE: cluster-based retrieval of images by unsupervised learning," in IEEE Transactions on Image Processing, vol. 14, no. 8, pp. 1187-1201, Aug. 2005, doi: 10.1109/TIP.2005.849770.
[4] R. Zhang and Zh. Zhang, “ Empirical Bayesian learning in the relevance feedback for image retrieval,” Image and Vision Computing. vol. 24,no.3, pp.211–223, 2006, doi:10.1016/j.imavis.2005.11.004.
[5] V.Mezaris and I. Kompatsiaris, “ Region Based Image Retrieval Using an Object Ontology and Relevance Feedback,” EURASIP Journal on Applied Signal Processing ,vol.6,pp. 886–901, 2004, doi: 10.1155/S1110865704401188.
[6] Y. Rui, T. S. Huang and S. Mehrotra, "Content-based image retrieval with relevance feedback in MARS," Proceedings of International Conference on Image Processing, 1997, pp. 815-818 vol.2, doi: 10.1109/ICIP.1997.638621.
[7] J.A. Hartigan and M.A. Wong , “Algorithm AS136: A k-means Clustering algorithm,” Applied Statistic, vol.28,no.1, pp. 100-108. 1979, doi:10.2307/2346830.
[8] B. Kimia, “Shape Representation for Image Retrieval”, Image Databases: Search and Retrieval of Digital Imagery, John Wiley & Sons, pp. 345-358, 2001, doi:10.1002/0471224634.ch13.
[9] J.Li, J. Z. Wang and G. Wiederhold , “Integrated Region Matching for Image Retrieval,” ACM Multimedia, p. 147-156,2000 ,doi: 10.1145/354384.354452.
[10] J. Mao and A.K. Jain, “Texture Classification and Segmentation using Multi-Resolution Simultaneous Autoregressive Models,” Pattern Recognition, vol. 25,no.2, pp. 173-188, 2010, doi:10.1016/0031-3203(92)90099-5.
[11] M. Jalali and T. Sedghi, "Semi Supervised Feature Extraction for Filling Semantic Gap in Image Retrieval," Iranian Conference on Machine Vision and Image Processing, 2011, pp. 1-4, doi: 10.1109/IranianMVIP.2011.6121537.
[12] A. Pentland, R. Picard and S. Sclaroff “Photobook: Contentbased manipulation of image databases,” International Journal of Computer Vision, vol.18,no.3, pp.233–254 ,1996, doi:10.1007/BF00123143.
[13] T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” 5th Int. Conf. Learn. Represent. ICLR 2017 - Conf. Track Proc., pp. 1–14, 2017.
[14] R. Zhang and Z. Zhang " BALAS: Empirical Bayesian learning in the relevance feedback for image retrieval," Image and Vision Computing , vol.24, no.3, pp. 211–223,2006, doi:10.1016/j.imavis.2005.11.004.
[15] J. Smith, “Color for Image Retrieval”, Image Databases: Search and Retrieval of Digital Imagery, John Wiley & Sons, New York, pp.285-311,2001.
[16] J. R. Smith and Shih-Fu Chang, "Single color extraction and image query," Proceedings., International Conference on Image Processing, 1995, pp. 528-531 vol.3, doi: 10.1109/ICIP.1995.537688.
[17] M. Stricker and M. Swain, "The capacity of color histogram indexing," 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1994, pp. 704-708, doi: 10.1109/CVPR.1994.323774.
[18] M. Unser, "Texture classification and segmentation using wavelet frames," in IEEE Transactions on Image Processing, vol. 4, no. 11, pp. 1549-1560, Nov. 1995, doi: 10.1109/83.469936.
[19] K. P. Yip, D. W. Cheung and M. K. Ng, "On discovery of extremely low-dimensional clusters using semi-supervised projected clustering," 21st International Conference on Data Engineering (ICDE'05), 2005, pp. 329-340, doi: 10.1109/ICDE.2005.96.
[20] L. Nanni, A. Rigo, A. Lumini, and S. Brahnam, “Spectrogram classification using dissimilarity space,” Appl. Sci., vol. 10, no. 12, pp. 1–17, 2020, doi:10.3390/app10124176.
[21] Y. Chen, J. Z. Wang and R. Krovetz, "CLUE: cluster-based retrieval of images by unsupervised learning," in IEEE Transactions on Image Processing, vol. 14, no. 8, pp. 1187-1201, Aug. 2005, doi: 10.1109/TIP.2005.849770.
[22] K. Z. Mao, K. -C. Tan and W. Ser, "Probabilistic neural-network structure determination for pattern classification," in IEEE Transactions on Neural Networks, vol. 11, no. 4, pp. 1009-1016, July 2000, doi: 10.1109/72.857781.
[23] M. Flickner et al., "Query by image and video content: the QBIC system," in Computer, vol. 28, no. 9, pp. 23-32, Sept. 1995, doi: 10.1109/2.410146.
[24] X. Zhang, M. Lei, D. Yang, Y. Wang and L. Ma, “Multi-scale curvature product for robust image corner detection in curvature scale space,” Pattern Recognition Letters, vol.28,no.5,Apr. 2007, doi:10.1016/j.patrec.2006.10.006.
[25] N. C. Shirazi, R. Hamzehyan, and A. Masoomi, " The Comparison of Classification Algorithms for Remote Sensing Images," Journal of Communication Engineering., vol. 5,no.17, pp. 31-38, 2015(in persian).
[26] A. Soltani-Farani, H. R. Rabiee, and S. A. Hosseini, "Spatial-aware dictionary learning for hyperspectral image classification," IEEE Trans. Geosci. Remote Sens., vol. 53,no.1, pp. 527-541, 2015, doi: 10.1109/TGRS.2014.2325067.
_||_
[1] F. Cao , J. Liang and B. Liang, “ A new initialization method for categorical data clustering,” Expert Systems with Applications., vol 36,no.7, pp. 10223–10228, 2009 ,doi:10.1016/j.eswa.2009.01.060.
[2] A. Amato and V. Lecce, “ A knowledge based approach for a fast image retrieval system,” Image and Vision Computing vol 26,no.11, pp. 1466–1480, 2008, doi:10.1016/j.imavis.2008.01.005.
[3] Y. Chen, J. Z. Wang and R. Krovetz, "CLUE: cluster-based retrieval of images by unsupervised learning," in IEEE Transactions on Image Processing, vol. 14, no. 8, pp. 1187-1201, Aug. 2005, doi: 10.1109/TIP.2005.849770.
[4] R. Zhang and Zh. Zhang, “ Empirical Bayesian learning in the relevance feedback for image retrieval,” Image and Vision Computing. vol. 24,no.3, pp.211–223, 2006, doi:10.1016/j.imavis.2005.11.004.
[5] V.Mezaris and I. Kompatsiaris, “ Region Based Image Retrieval Using an Object Ontology and Relevance Feedback,” EURASIP Journal on Applied Signal Processing ,vol.6,pp. 886–901, 2004, doi: 10.1155/S1110865704401188.
[6] Y. Rui, T. S. Huang and S. Mehrotra, "Content-based image retrieval with relevance feedback in MARS," Proceedings of International Conference on Image Processing, 1997, pp. 815-818 vol.2, doi: 10.1109/ICIP.1997.638621.
[7] J.A. Hartigan and M.A. Wong , “Algorithm AS136: A k-means Clustering algorithm,” Applied Statistic, vol.28,no.1, pp. 100-108. 1979, doi:10.2307/2346830.
[8] B. Kimia, “Shape Representation for Image Retrieval”, Image Databases: Search and Retrieval of Digital Imagery, John Wiley & Sons, pp. 345-358, 2001, doi:10.1002/0471224634.ch13.
[9] J.Li, J. Z. Wang and G. Wiederhold , “Integrated Region Matching for Image Retrieval,” ACM Multimedia, p. 147-156,2000 ,doi: 10.1145/354384.354452.
[10] J. Mao and A.K. Jain, “Texture Classification and Segmentation using Multi-Resolution Simultaneous Autoregressive Models,” Pattern Recognition, vol. 25,no.2, pp. 173-188, 2010, doi:10.1016/0031-3203(92)90099-5.
[11] M. Jalali and T. Sedghi, "Semi Supervised Feature Extraction for Filling Semantic Gap in Image Retrieval," Iranian Conference on Machine Vision and Image Processing, 2011, pp. 1-4, doi: 10.1109/IranianMVIP.2011.6121537.
[12] A. Pentland, R. Picard and S. Sclaroff “Photobook: Contentbased manipulation of image databases,” International Journal of Computer Vision, vol.18,no.3, pp.233–254 ,1996, doi:10.1007/BF00123143.
[13] T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” 5th Int. Conf. Learn. Represent. ICLR 2017 - Conf. Track Proc., pp. 1–14, 2017.
[14] R. Zhang and Z. Zhang " BALAS: Empirical Bayesian learning in the relevance feedback for image retrieval," Image and Vision Computing , vol.24, no.3, pp. 211–223,2006, doi:10.1016/j.imavis.2005.11.004.
[15] J. Smith, “Color for Image Retrieval”, Image Databases: Search and Retrieval of Digital Imagery, John Wiley & Sons, New York, pp.285-311,2001.
[16] J. R. Smith and Shih-Fu Chang, "Single color extraction and image query," Proceedings., International Conference on Image Processing, 1995, pp. 528-531 vol.3, doi: 10.1109/ICIP.1995.537688.
[17] M. Stricker and M. Swain, "The capacity of color histogram indexing," 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1994, pp. 704-708, doi: 10.1109/CVPR.1994.323774.
[18] M. Unser, "Texture classification and segmentation using wavelet frames," in IEEE Transactions on Image Processing, vol. 4, no. 11, pp. 1549-1560, Nov. 1995, doi: 10.1109/83.469936.
[19] K. P. Yip, D. W. Cheung and M. K. Ng, "On discovery of extremely low-dimensional clusters using semi-supervised projected clustering," 21st International Conference on Data Engineering (ICDE'05), 2005, pp. 329-340, doi: 10.1109/ICDE.2005.96.
[20] L. Nanni, A. Rigo, A. Lumini, and S. Brahnam, “Spectrogram classification using dissimilarity space,” Appl. Sci., vol. 10, no. 12, pp. 1–17, 2020, doi:10.3390/app10124176.
[21] Y. Chen, J. Z. Wang and R. Krovetz, "CLUE: cluster-based retrieval of images by unsupervised learning," in IEEE Transactions on Image Processing, vol. 14, no. 8, pp. 1187-1201, Aug. 2005, doi: 10.1109/TIP.2005.849770.
[22] K. Z. Mao, K. -C. Tan and W. Ser, "Probabilistic neural-network structure determination for pattern classification," in IEEE Transactions on Neural Networks, vol. 11, no. 4, pp. 1009-1016, July 2000, doi: 10.1109/72.857781.
[23] M. Flickner et al., "Query by image and video content: the QBIC system," in Computer, vol. 28, no. 9, pp. 23-32, Sept. 1995, doi: 10.1109/2.410146.
[24] X. Zhang, M. Lei, D. Yang, Y. Wang and L. Ma, “Multi-scale curvature product for robust image corner detection in curvature scale space,” Pattern Recognition Letters, vol.28,no.5,Apr. 2007, doi:10.1016/j.patrec.2006.10.006.
[25] N. C. Shirazi, R. Hamzehyan, and A. Masoomi, " The Comparison of Classification Algorithms for Remote Sensing Images," Journal of Communication Engineering., vol. 5,no.17, pp. 31-38, 2015(in persian).
[26] A. Soltani-Farani, H. R. Rabiee, and S. A. Hosseini, "Spatial-aware dictionary learning for hyperspectral image classification," IEEE Trans. Geosci. Remote Sens., vol. 53,no.1, pp. 527-541, 2015, doi: 10.1109/TGRS.2014.2325067.