Improving the Structure of Deep Learning Algorithm in Image Processing Inspired by Representational Brain Dissimilarity Matrix
Subject Areas : Renewable energy
Zahra Heydaran Daroogheh
1
(
Department of Electrical, College of Engineering, Saveh Branch, Islamic Azad University, Saveh, Iran
)
Mohammad Jalal Rastegar Fatemi
2
(
Department of Electrical, College of Engineering, Saveh Branch, Islamic Azad University, Saveh, Iran
)
Maryam Rastgarpour
3
(
Department of Computer, College of Engineering, Saveh Branch, Islamic Azad University, Saveh, Iran
)
Keywords: deep learning, Convolutional system, Recognition rate, Gaussian differential filter, Network Energy level, Sparsity, Representational Dissimilarity Matrix,
Abstract :
Deep learning algorithms achieves some results at human level or even better in pattern recognition problems. Meanwhile they apply a different mechanism other than human brain. This paper describes a human-inspired segmentation and interpolation algorithm, which applies the retinal layer in the proposed model after the input layer. Following this retina, this layer encrypts the input image and transmits the input image to the second space, which try to change deep network structure inspired of the brain's visual path. Network feedback, recognition rate, and network energy level or the comprehensiveness of the trained network examined in subsets of the Caltech data set. In similar examples, deep learning algorithms require more data to learn other than human. In the difference between deep learning and human, there is a difference in the representation of information. In deep learning, weights improve in a way that optimizes the result in a particular experiment, but in millions of years of human evolution, the human brain has evolved optimally and effectively representation. Another point of contention is the deepening of deep learning layers. The number of these layers has multiplied compared to the brain that lead to more complexity and energy expenditure. However, in the brain it can make a diagnosis with less energy. The maximum recognition rate of the proposed model is 93% and the base model is close to 91%. Also, the proposed model is thinner and the rate of fire of neurons in the initial layers is lower and has a high stability to changes in light intensity. The Dissimilarity of the model layers has been higher and it has been able to show a better response in the face of noise images and record less recognition loss.
[1] M. Riesenhuber, T. Poggio, "Hierarchical models of object recognition in cortex", Nature Neuroscience, vol. 2, no. 11, pp. 1019–25, Nov. 1999 (doi: 10.1038/14819).
[2] K. Fukushima, "Artificial vision by multi-layered neural networks: neocognitron and its advances", Neural Network, vol. 37, no. 2013, pp. 103–19, Jan. 2013 (doi: 10.1016/j.neunet.2012.09.016)
[3] Y. Li, W. Wu, B. Zhang, F. Li, "Enhanced HMAX model with feedforward feature learning for multiclass categorization", Frontiers in Computational Neuroscience, vol. 9, p. 123, Oct. 2015 (doi: 10.3389/fncom.2015.00123).
[4] B. Yang, L. Zhou, Z. Deng, "C-HMAX: Artificial cognitive model inspired by the color vision mechanism of the human brain", Tsinghua Science Technology, vol. 18, no. 1, pp. 51–56, Feb. 2013 (doi: 10.1109/TST.2013.6449407).
[5] S. Zabbah, K. Rajaei, A. Mirzaei, R. Ebrahimpour, S. M. Khaligh-Razavi, "The impact of the lateral geniculate nucleus and corticogeniculate interactions on efficient coding and higher-order visual object processing", Vision Research, vol. 101, pp. 82–93, Aug. 2014 (doi: 10.1016/j.visres.2014.05.006).
[6] A. Krizhevsky, I. Sutskever, G. E. Hinton, "Imagenet classification with deep convolutional neural networks", in Advances in neural information processing systems, pp. 1097–1105, 2012.
[7] D. Ciregan, U. Meier, J. Schmidhuber, "Multi-column deep neural networks for image classification", Proceeding of the IEEE/CVPR, pp. 3642–3649, Providence, RI, USA, June 2012 (doi: 10.1109/CVPR.2012.6248110).
[8] D. Berman, A. Buczak, J. Chavis, C. Corbett, "A survey of deep learning methods for cyber security", Information, vol. 10, no. 4, Article Number: 122, April 2019 (doi:10.3390/info10040122).
[9] A. Gupta, H. K. Thakur, R. Shrivastava, P. Kumar, S. Nag, "A big data analysis framework using apache spark and deep learning", Proceeding of the IEEE/ICDMW, pp. 9–16, New Orleans, LA, USA, Nov. 2017 (doi:10.1109/ICDMW.2017.9)
[10] L. Deng, "A tutorial survey of architectures, algorithms, and applications for deep learning", APSIPA Trans. Signal Inf. Process., vol. 3, 2014. (doi: 10.1017/atsip.2013.9)
[11] S. Helie, F. G. Ashby, "Learning and transfer of category knowledge in an indirect categorization task", Psychological Research, vol. 76, no. 3, pp. 292–303, 2012 (doi: 10.1007/s00426-011-0348-1).
[12] M. Stettler, G. Francis, "Using a model of human visual perception to improve deep learning", Neural Networks, vol. 104, pp. 40–49, 2018 (doi: 10.1016/j.neunet.2018.04.005)
[13] K. He, X. Zhang, S. Ren, J. Sun, "Deep residual learning for image recognition", Proceedings of the IEEE/CVPR, pp. 770–778, Las Vegas, NV, USA, June 2016 (doi: 10.1109/CVPR.2016.90)
[14] R. C. Fong, W. J. Scheirer, D. D. Cox, "Using human brain activity to guide machine learning", Scientific Reports, vol. 8, no. 1,Article Number: 5397, 2018 (doi: 10.1038/s41598-018-23618-6)
[15] H. Mahdavi-Nasab, M. S. Memarzadeh, P. Moallem, "Automatic Persian license plate recognition by edge detection using hopfield neural network", Journal of Intelligent Procedures in Electrical Technology, vol. 1, no. 4, pp. 41-48, Winter 2011.
[16] S. Chatterjee, G. D. Field, G. D. Horwitz, E. N. Johnson, K. Koida, K. Mancuso, "Advances in color science: from retina to behavior", Journal Neuroscience, vol. 30, no. 45, pp. 14955–63, Nov. 2010 (doi: 10.1523/JNEUROSCI.4348-10.2010).
[17] M. E. Rudd, “Edge integration in achromatic color perception and the lightness– darkness asymmetry through retinex theory", Journal of Vision, vol. 13, pp. 1–30, Dec. 2013 (doi: 10.1167/13.14.18)
[18] A. Lu, G. Xu, H. Jin, L. Mo, J. Zhang, J. X. Zhang, "Electrophysiological evidence for effects of color knowledge in object recognition", Neuroscience Letters, vol. 469, no. 3, pp. 405–10, Jan. 2010 (doi: 10.1016/j.neulet.2009.12.039).
[19] R. Kozik, M. Chora´s, M. Ficco, F. Palmieri, "A scalable distributed machine learning approach for attack detection in edge computing environments", Journal of Parallel and Distributed Computing, vol. 119, pp. 18 – 26, Sept. 2018 (doi: 10.1016/j.jpdc.2018.03.006).
[20] W. Fu, M. Johnston, M. Zhang, "Automatic construction of Gaussian-based edge detectors using genetic programmings", Applications of Evolutionary Computation, pp. 336-375, 2011 (doi: 10.1016/j.patrec.2005.07.024).
[21] Nili, Hamed, et al. "A toolbox for representational similarity analysis", PLoS Computational Biology, vol. 10, no. 4, April 2014 (doi: 10.1371/journal.pcbi.1003553).
[22] A. R. Karimian, M. Torabian, M. R. Yazdchi, "Improvement of industrial radiography for defect detection of oil and gas pipelines in weld regions by image processing", Journal of Intelligent Procedures in Electrical Technology, vol. 1, no. 2, pp. 23-30, Summer 2010.
[23] I. F. Salazar-reque, S. G. Huamán, G. Kemper, J. Telles, D. Diaz, "An algorithm for plant disease visual symptom detection in digital images based on superpixels", International Journal on Advanced Science, Engineering and Information Technology, vol. 9, no. 1, pp. 194–203, 2019 (doi: 10.18517/ijaseit.9.1.5322)
[24] J. G. A. Barbedo, "A new automatic method for disease symptom segmentation in digital photographs of plant leaves", European Journal of Plant Pathology, vol. 147, no. 2, pp. 349–364, 2017 (doi: 10.1007/s10658-016-1007-6).
[25] G. Marcus, “Deep Learning: A Critical Appraisal”, arXiv Prepr. arXiv:1801.00631, 2018.
[26] J. Ngiam, Z. Chen, P. W. Koh, A. Y. Ng, "Learning deep energy models", Proceedings of the ICML pp. 1105–1112, June 2011.
[27] K. He, X. Zhang, S. Ren, J. Sun, "Identity mappings in deep residual networks", Computer Vision and Pattern Recognition, pp. 630–645, March 2016.
[28] A. Ng, "Sparse autoencoder", CS294A Lecture Notes, vol. 72, no. 2011, pp. 1–19, 2011.
[29] M. S. Goli, A. Nsghsh, "Robust digital image watermarking against cropping and salt & pepper noise using two-step sudoku", Journal of Intelligent Procedures in Electrical Technology, vol. 8, no. 31, pp. 21-32, Autumn 2017.
_||_[1] M. Riesenhuber, T. Poggio, "Hierarchical models of object recognition in cortex", Nature Neuroscience, vol. 2, no. 11, pp. 1019–25, Nov. 1999 (doi: 10.1038/14819).
[2] K. Fukushima, "Artificial vision by multi-layered neural networks: neocognitron and its advances", Neural Network, vol. 37, no. 2013, pp. 103–19, Jan. 2013 (doi: 10.1016/j.neunet.2012.09.016)
[3] Y. Li, W. Wu, B. Zhang, F. Li, "Enhanced HMAX model with feedforward feature learning for multiclass categorization", Frontiers in Computational Neuroscience, vol. 9, p. 123, Oct. 2015 (doi: 10.3389/fncom.2015.00123).
[4] B. Yang, L. Zhou, Z. Deng, "C-HMAX: Artificial cognitive model inspired by the color vision mechanism of the human brain", Tsinghua Science Technology, vol. 18, no. 1, pp. 51–56, Feb. 2013 (doi: 10.1109/TST.2013.6449407).
[5] S. Zabbah, K. Rajaei, A. Mirzaei, R. Ebrahimpour, S. M. Khaligh-Razavi, "The impact of the lateral geniculate nucleus and corticogeniculate interactions on efficient coding and higher-order visual object processing", Vision Research, vol. 101, pp. 82–93, Aug. 2014 (doi: 10.1016/j.visres.2014.05.006).
[6] A. Krizhevsky, I. Sutskever, G. E. Hinton, "Imagenet classification with deep convolutional neural networks", in Advances in neural information processing systems, pp. 1097–1105, 2012.
[7] D. Ciregan, U. Meier, J. Schmidhuber, "Multi-column deep neural networks for image classification", Proceeding of the IEEE/CVPR, pp. 3642–3649, Providence, RI, USA, June 2012 (doi: 10.1109/CVPR.2012.6248110).
[8] D. Berman, A. Buczak, J. Chavis, C. Corbett, "A survey of deep learning methods for cyber security", Information, vol. 10, no. 4, Article Number: 122, April 2019 (doi:10.3390/info10040122).
[9] A. Gupta, H. K. Thakur, R. Shrivastava, P. Kumar, S. Nag, "A big data analysis framework using apache spark and deep learning", Proceeding of the IEEE/ICDMW, pp. 9–16, New Orleans, LA, USA, Nov. 2017 (doi:10.1109/ICDMW.2017.9)
[10] L. Deng, "A tutorial survey of architectures, algorithms, and applications for deep learning", APSIPA Trans. Signal Inf. Process., vol. 3, 2014. (doi: 10.1017/atsip.2013.9)
[11] S. Helie, F. G. Ashby, "Learning and transfer of category knowledge in an indirect categorization task", Psychological Research, vol. 76, no. 3, pp. 292–303, 2012 (doi: 10.1007/s00426-011-0348-1).
[12] M. Stettler, G. Francis, "Using a model of human visual perception to improve deep learning", Neural Networks, vol. 104, pp. 40–49, 2018 (doi: 10.1016/j.neunet.2018.04.005)
[13] K. He, X. Zhang, S. Ren, J. Sun, "Deep residual learning for image recognition", Proceedings of the IEEE/CVPR, pp. 770–778, Las Vegas, NV, USA, June 2016 (doi: 10.1109/CVPR.2016.90)
[14] R. C. Fong, W. J. Scheirer, D. D. Cox, "Using human brain activity to guide machine learning", Scientific Reports, vol. 8, no. 1,Article Number: 5397, 2018 (doi: 10.1038/s41598-018-23618-6)
[15] H. Mahdavi-Nasab, M. S. Memarzadeh, P. Moallem, "Automatic Persian license plate recognition by edge detection using hopfield neural network", Journal of Intelligent Procedures in Electrical Technology, vol. 1, no. 4, pp. 41-48, Winter 2011.
[16] S. Chatterjee, G. D. Field, G. D. Horwitz, E. N. Johnson, K. Koida, K. Mancuso, "Advances in color science: from retina to behavior", Journal Neuroscience, vol. 30, no. 45, pp. 14955–63, Nov. 2010 (doi: 10.1523/JNEUROSCI.4348-10.2010).
[17] M. E. Rudd, “Edge integration in achromatic color perception and the lightness– darkness asymmetry through retinex theory", Journal of Vision, vol. 13, pp. 1–30, Dec. 2013 (doi: 10.1167/13.14.18)
[18] A. Lu, G. Xu, H. Jin, L. Mo, J. Zhang, J. X. Zhang, "Electrophysiological evidence for effects of color knowledge in object recognition", Neuroscience Letters, vol. 469, no. 3, pp. 405–10, Jan. 2010 (doi: 10.1016/j.neulet.2009.12.039).
[19] R. Kozik, M. Chora´s, M. Ficco, F. Palmieri, "A scalable distributed machine learning approach for attack detection in edge computing environments", Journal of Parallel and Distributed Computing, vol. 119, pp. 18 – 26, Sept. 2018 (doi: 10.1016/j.jpdc.2018.03.006).
[20] W. Fu, M. Johnston, M. Zhang, "Automatic construction of Gaussian-based edge detectors using genetic programmings", Applications of Evolutionary Computation, pp. 336-375, 2011 (doi: 10.1016/j.patrec.2005.07.024).
[21] Nili, Hamed, et al. "A toolbox for representational similarity analysis", PLoS Computational Biology, vol. 10, no. 4, April 2014 (doi: 10.1371/journal.pcbi.1003553).
[22] A. R. Karimian, M. Torabian, M. R. Yazdchi, "Improvement of industrial radiography for defect detection of oil and gas pipelines in weld regions by image processing", Journal of Intelligent Procedures in Electrical Technology, vol. 1, no. 2, pp. 23-30, Summer 2010.
[23] I. F. Salazar-reque, S. G. Huamán, G. Kemper, J. Telles, D. Diaz, "An algorithm for plant disease visual symptom detection in digital images based on superpixels", International Journal on Advanced Science, Engineering and Information Technology, vol. 9, no. 1, pp. 194–203, 2019 (doi: 10.18517/ijaseit.9.1.5322)
[24] J. G. A. Barbedo, "A new automatic method for disease symptom segmentation in digital photographs of plant leaves", European Journal of Plant Pathology, vol. 147, no. 2, pp. 349–364, 2017 (doi: 10.1007/s10658-016-1007-6).
[25] G. Marcus, “Deep Learning: A Critical Appraisal”, arXiv Prepr. arXiv:1801.00631, 2018.
[26] J. Ngiam, Z. Chen, P. W. Koh, A. Y. Ng, "Learning deep energy models", Proceedings of the ICML pp. 1105–1112, June 2011.
[27] K. He, X. Zhang, S. Ren, J. Sun, "Identity mappings in deep residual networks", Computer Vision and Pattern Recognition, pp. 630–645, March 2016.
[28] A. Ng, "Sparse autoencoder", CS294A Lecture Notes, vol. 72, no. 2011, pp. 1–19, 2011.
[29] M. S. Goli, A. Nsghsh, "Robust digital image watermarking against cropping and salt & pepper noise using two-step sudoku", Journal of Intelligent Procedures in Electrical Technology, vol. 8, no. 31, pp. 21-32, Autumn 2017.