Increasing the Accuracy of Convolutional Neural Networks Based on 4-Stream Model with Image Processing Filters and Linear Mapping of Dissimilarity Space

Subject Areas : Image and video processing

Zahra Heydaran Daroogheh Amnyieh ¹ , Seyed Mohammad Jalal Rastegar Fatemi ² , Maryam Rastgarpour ³ , Golnaz Aghaee ghazvini ⁴

1 - Department of Electrical Engineering- Saveh Branch, Islamic Azad University, Saveh, Iran
2 - Department of Electrical Engineering- Saveh Branch, Islamic Azad University, Saveh, Iran
3 - Department of Computer Engineering- Saveh Branch, Islamic Azad University, Saveh, Iran
4 - Department of Computer Engineering- Dolatabad Branch, Islamic Azad University, Dolatabad, Iran

Received: 2022-08-26 Accepted : 2022-12-14 Published : 2022-12-14

Keywords: prototype, deep learning, Representational Dissimilarity Matrix, convolutional neural network, Dissimilarity Vector Space,

Abstract :

With the expansion and success of convolutional networks, the topic of deep learning has attracted increasing attention in recent years; Since convolutional networks include many layers, optimal learning of network layers is of great importance. In this paper, a new model, called the 4-stream model, is presented with the aim of helping to linearize the data space using representational dissimilarity transformation, and the effects of this transformation on standard classifications for artificial data and Cifar10 images are investigated. Then, two models based on data preprocessing with dissimilarity transform representation and Sobel and Edge Detector filters are analyzed. The 4-stream model increased the accuracy by 3.2% due to the increase in the number of model parameters, and hence the capacity of the network. Besides, adding the dissimilarity representation wherever the classifier cannot perform a high-resolution classification by merely using the main features, can help to increase the discriminability of classes by adding linear features.

References:

[1] M. Zhang, W. Li, Q. Du, "Diverse region-based CNN for hyper spectral image classification", IEEE Trans. on Image Processing, vol. 27, no. 6, pp. 2623-2634, June 2018 (doi: 10.1109/TIP.2018.2809606).

[2] Z. Gong, P. Zhong, Y. Yu, W. Hu, S. Li, "A CNN with multi scale convolution and diversified metric for hyper spectral image classification", IEEE Trans. on Geoscience and Remote Sensing, vol. 57, no. 6, pp. 3599-3618, June 2019 (doi: 10.1109/TGRS.2018.2886022).

[3] Y. Pei, Y. Huang, Q. Zou, X. Zhang, S. Wang, "Effects of image degradation and degradation removal to cnn-based image classification", IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 43, no. 4, pp. 1239-1253, April 2021 (doi: 10.1109/TPAMI.2019.2950923).

[4] M. Gour, S. Jain, T.S. Kumar, "Residual learning based CNN for breast cancer histopathological image classification", International Journal of Imaging Systems and Technology, vol. 30, no. 3, pp. 621-635, Sept. 2020 (doi:10.1002/ima.22403).

[5] Y.M. Costa, D. Bertolini, A.S. Britto, G.D. Cavalcanti, L.E. Oliveira, "The dissimilarity approach: A review", Artificial Intelligence Review, vol. 53, no. 4, pp. 2783-2808, April 2020 (doi: 10.1007/s10462-019-09746-z).

[6] E. Pękalska, R.P. Duin, "Dissimilarity representations allow for building good classifiers", Pattern Recognition Letters, vol. 23, no. 8, pp. 943-956, June 2002 (doi: 10.1016/S0167-8655(02)00024-7).

[7] E. Pekalska, P. Paclik, R. P. Duin, "A generalized kernel approach to dissimilarity-based classification", Journal of Machine Learning Eesearch, vol. 2, pp. 175-211, Dec. 2001 (doi: 10.1.1.16.3363).

[8] E. Pekalska, R.P. Duin, "Dissimilarity-based classification for vectorial representations", Proceding of the IEEE/ ICPR, pp. 137-140, Hong Kong, China, Aug. 2006 (doi: 10.1109/ICPR.2006.457).

[9] R.P. Duin, M. Loog, E. Pȩkalska, D.M. Tax, "Feature-based dissimilarity space classification", Proceding of ICPR, vol. 6388, pp. 46-55, Springer, Berlin, Heidelberg, Aug. 2010 (doi: 10.1007/978-3-642-17711-8_5).

[10] R.P. Duin, E. Pękalska, "The dissimilarity space: Bridging structural and statistical pattern recognition", Pattern Recognition Letters, vol. 33, no. 7, pp. 826-832, May 2012 (doi: 10.1016/j.patrec.2011.04.019).

[11] I. Theodorakopoulos, D. Kastaniotis, G. Economou, S. Fotopoulos, "Pose-based human action recognition via sparse representation in dissimilarity space", Journal of Visual Communication and Image Represent­at­i­o­n, vol. 25, no. 1, pp. 12-23, Jan. 2014 (doi: 10.1016/j.jvcir.2013.03.008).

[12] H. Bunke, K. Riesen, "Graph classification based on dissimilarity space embedding", Proceding of  IAPR-SPR-SSPR, vol 5342, pp. 996-1007, Springer, Berlin, Heidelberg, Dec. 2008 (doi: 10.1007/978-3-540-89689-0_103). 

[13] R.H. Pinheiro, G.D. Cavalcanti, R. Tsang, "Combining dissimilarity spaces for text categorizati­on",  Information Sciences, vol. 406-407, pp. 87-101, Sept. 2017 (doi: 10.1016/j.ins.2017.04.025).

[14] C. Santos, E.J. Justino, F. Bortolozzi, R. Sabourin, "An off-line signature verification method based on the questioned document expert's approach and a neural network classifier", Proceding of the IEEE/IWFHR, pp. 498-502, Kokubunji, Japan, Oct. 2004 (doi: 10.1109/IWFHR.2004.17).

[15] S.H. Cha, S.N. Srihari, "Writer identification: statistical analysis and dichotomizer", Proceding of  IAPR-SPR-SSPR, vol 1876, pp. 123-132, Springer, Berlin, Heidelberg, Dec. 2000 (doi:10.1007/3-540-44522-6_13).

[16] D. Bertolini, L. S. Oliveira, E. Justino, R. Sabourin, "Texture-based descriptors for writer identification and verification", Expert Systems with Applications, vol. 40, no. 6, pp. 2069-2080, May 2013 (doi: 10.1016/j.esw­a.20­12.10.016).

[17] L.S. Oliveira, E. Justino, R, Sabourin, "Off-line signature verification using writer-independent approach", Proceding of the IEEE/IJCNN, pp. 2539-2544, Orlando, FL, USA, Aug. 2007 (doi: 10.1109/IJCNN.200­7.437­13­58).

[18] J.G. Martins, L.S. Oliveira, A.S. Britto, R Sabourin, "Forest species recognition based on dynamic classifier selection and dissimilarity feature vector representation", Machine Vision and Applications, vol. 26, no. 2, pp. 279-293, Apr. 2015 (doi: 10.1007/s00138-015-0659-0).

[19] R.H. Zottesso, Y.M. Costa, D. Bertolini, L.E. Oliveira, "Bird species identification using spectrogram and dissimilarity approach", Ecological Informatics, vol. 48, pp. 187-197, Nov. 2018 (doi: 10.1016/j.ecoinf.­201­8.0­8.007).

[20] N. Kriegeskorte, M. Mur, P.A. Bandettini, "Representational similarity analysis-connecting the branches of systems neuroscience", Frontiers in Systems Neuroscience, vol. 2, no. 4, Nov. 2008 (doi: 10.3389/neuro­.06.­00­4.2008).

[21] H. Popal, Y. Wang, I.R. Olson, "A guide to representational similarity analysis for social neuroscience", Social Cognitive and Affective Neuroscience, vol. 14, no. 11, pp. 1243-1253, Nov. 2019 (doi: 10.31234/osf.i­o/n­d8fh). 

[22] T.K. Pegors, S. Tompson, M.B. O’Donnell, E. B. Falk, "Predicting behavior change from persuasive messages using neural representational similarity and social network analyses", NeuroImage, vol. 157, pp. 118-128, Aug. 2017 (doi: 10.1016/j.neuroimage.2017.05.063). 

[23] R.M. Visser, H.S. Scholte, T. Beemsterboer, M. Kindt, "Neural pattern similarity predicts long-term fear memory", Nature neuroscience, vol. 16, no. 4, pp. 388-390, April 2013 (doi: 10.1038/nn.3345).

[24] Z.H.D. Amnyieh, S.M.J.R. Fatemi, M. Rastgarpour, G.A. Ghazvini, "CNN-RDM: A new image processing model for improving the structure of deep learning based on representational dissimilarity matrix", Journal of Supercomputing, vol. 9, pp. 1-25, Sept. 2022 (doi: 10.1007/s11227-022-04661-7). 

[25] J.B. Ritchie, H.L. Masson, S. Bracci, H.P.O. Beeck, "The unreliable influence of multivariate noise normalization on the reliability of neural dissimilarity", NeuroImage, vol. 245, Article Number: 118686, Dec. 2021 (doi: 10.1016/j.neuroimage.2021.118686).

[26] I. Muukkonen, K. Ölander, J. Numminen, V. R Salmela, "Spatio-temporal dynamics of face percep­ti­o­n",  NeuroImage, vol. 209, Article Number: 116531,  April 2020 (doi: 10.1101/550038). 

[27] J. Diedrichsen, E. Berlot, M. Mur, H. H. Schütt, M. Shahbazi, N. Kriegeskorte, "Comparing representational geometries using whitened unbiased-distance-matrix similarity", Neural Data Science/Analysis, vol. 5, no. 3,  Aug. 2021 (doi: 10.51628/001c.27664).

[28] H. Wu, B. Xiao, N. Codella, M. L iu, X. Dai, L. Yuan, L. Zhang, "Cvt: Introducing convolutions to vision transformers", Proceding of  the IEEE/ICCV, pp. 22-3, Montreal, QC, Canada, Oct. 2021 (doi: 10.1109/ic­cv­48­922.2021.00009). 

[29] F. Fooladgar, S. Kasaei, "Lightweight residual densely connected convolutional neural network", Multimedia Tools and Applications, vol. 79, no. 35, pp. 25571-25588, Sept. 2020 (doi: 10.1007/s11042-020-09223-8).

[30] N. Gruzling, "Linear separability of the vertices of an n-dimensional hypercube", Ph.D. Thesisi, University of Northern British Columbia, 2007 (doi: 10.24124/2007/bpgub464).

[31] R. Elshawi, A. Wahab, A. Barnawi, S. Sakr, "DLBench: a comprehensive experimental evaluation of deep learning frameworks", Cluster Computing, vol. 24, no. 3, pp. 2017-2038. Sept. 2021 (doi: 10.1007/s10586-021-03240-4).