Depth Image Resolution Enhancement Using Discrete Wavelet Transform and Convolution Neural Networks

Subject Areas : Majlesi Journal of Telecommunication Devices

Mohsen Ashourian ¹ , Seyed Mehrdad Mahdavi ²

1 - Department of Electrical Engineering, Majlesi Branch, Islamic Azad University, Majlesi, Iran
2 - Najafabad Branch, Islamic Azad University, Najafabad, Iran

Received: 2020-02-10 Accepted : 2020-02-13 Published : 2021-03-01

Keywords: Image Enhancement, Super resolution, convolution neural networks, Depth Camera Images,

Abstract :

The depth image plays an increasingly important role in fundamental research and daily applications, with the reducing the price and increasing the number of affordable and portable depth cameras. Infrared sensors or depth sensors are widely used to control dynamic and static 3D scenes. However, the depth image quality is limited to low-quality images, as the infrared sensor does not have high resolution. Therefore, given the problems and the importance of using 3-D images, the quality of these images should be improved in order to provide accurate images from depth cameras. In this paper a resolution enhancement method of depth images using convolutional neural networks is considered. A convolutional neural network with a depth of 20 and three layers and a pre-trained neural network is used. We developed the system and tested its performance for two datasets, Middlebury and EURECOM Kinect Face. Results show for EURECOM Kinect Face images, PSNR improvement is approximately 7 to 16 dB and for Middlebury images the PSNR improvement is about 6 to 12 dB.

References:

[1] S. Foix, G. Alenya, and C. Torras, "Lock-in time-of-flight (ToF) cameras: A survey," IEEE Sensors Journal, vol. 11, no. 9, pp. 1917-1926, 2011.

[2] E Eichhardt, Ivan, Dmitry Chetverikov, and Zsolt Janko. "Image-guided ToF depth upsampling: a survey." Machine Vision and Applications 28.3-4 (2017): 267-282.

[3] Park, Sung Cheol, Min Kyu Park, and Moon Gi Kang. "Super-resolution image reconstruction: a technical overview." IEEE signal processing magazine 20.3 (2003): 21-36.

[4] W. K. Carey, D. B. Chuang, and S. S. Hemami, “Regularity-preserving image interpolation,” IEEE Trans. Image Process., vol. 8, no. 9, pp. 1295–1297, Sep. 1999.

[5] Y. Piao, I. Shin, and H. W. Park, “Image resolution enhancement using inter-subband correlation in wavelet domain,” in Proc. Int. Conf. Image Process., 2007, vol. 1, pp. I-445–448.

[6] A. Temizel and T. Vlachos, “Image resolution upscaling in the wavelet domain using directional cycle spinning,” J. Electron. Imag., vol. 14, no. 4, 2005.

[7] Demirel, Hasan, and Gholamreza Anbarjafari. "Image resolution enhancement by using discrete and stationary wavelet decomposition." IEEE transactions on image processing 20.5 (2010): 1458-1460.

[8] Yao, Guangle, Tao Lei, and Jiandan Zhong. "A review of Convolutional-Neural-Network-based action recognition." Pattern Recognition Letters 118 (2019): 14-22.

[9] K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, "Beyond a gaussian denoiser: Residual learning of deep CNN for image denoising," IEEE Transactions on Image Processing, vol. 26, no. 7, pp. 3142-3155, 2017.

[10] A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Advances in neural information processing systems, 2012, pp. 1097-1105.

[11] Min, Rui, Neslihan Kose, and Jean-Luc Dugelay. "Kinectfacedb: A kinect database for face recognition." IEEE Transactions on Systems, Man, and Cybernetics: Systems 44.11 (2014): 1534-1548.

[12] Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M., Szeliski, R.: A database and evaluation methodology for optical flow. IJCV 92, 1–31 (2011)