Improving Super-Resolution in Face Images by Modeling Image Degradation Using Pairs of High-Quality and Low-Quality Images
Subject Areas : Electronics Engineering
1 - Department of information and Communication, Amin Police University, Tehran, Iran
Keywords: Quality of face image, Adversarial generative network, Super-resolution, Deep learning.,
Abstract :
Improving image quality for identification and authentication in security and surveillance systems is of particular importance, and today, using artificial intelligence, the quality of images can be significantly improved. In this regard, the present paper, focusing on the details of face images, has improved the image failure detection model in the adversarial generator network, which led to a suitable performance in the meta-dissolving of face images. Most of the CNN networks that have been presented in recent years require a large set of images with appropriate annotations for proper performance, and they usually perform poorly in the case of degradation that have not been trained, which is addressed in this research to improve this challenge. In this work, pairs of high-quality and low-quality images are used to train the image degradation detection model; This information is then transferred to real images. The naturalness of the output images is one of the most important challenges in this field. The obtained results show that the criterion of perceptual similarity of the obtained image is equal to 38.4%, which is comparable to recent researches. As a result, using the proposed model, more natural images were produced
Improving super-resolution in face images by modeling image degradation using pairs of high-quality and low-quality images
Improving image quality for identification and authentication in security and surveillance systems.
Using SynNet and DegNet networks, the image damage detection model was improved and the image details were preserved.
[1] P. Kaur and H. S. Pannu, “Comparative analysis of continuous and discrete orthogonal moments for face recognition,” Proc. Int. Conf. Electron. Commun. Aerosp. Technol. ICECA, 2017, pp. 449–453, 2017, doi: 10.1109/ICECA.2017.8203724.
[2] N. Aloysius and M. Geetha, "A review on deep convolutional neural networks," International Conference on Communication and Signal Processing (ICCSP), Chennai, India, 2017, pp. 0588-0592, doi: 10.1109/ICCSP.2017.8286426.
[3] A. Khan, A. Sohail, U. Zahoora and A. S. Qureshi, “A survey of the recent architectures of deep convolutional neural networks,” Artif. Intell. Rev., vol. 53, no. 8, pp. 5455–5516, 2020, doi: 10.1007/s10462-020-09825-6.
[4] J. Kim, J. K. Lee and K. M. Lee, “Accurate image super-resolution using very deep convolutional networks,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., 2016, pp. 1646–1654, doi: 10.1109/CVPR.2016.182.
[5] S. Guo, Z. Yan, K. Zhang, W. Zuo and L. Zhang, “Toward convolutional blind denoising of real photographs,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., 2019, pp. 1712–1722, doi: 10.1109/CVPR.2019.00181.
[6] X. Ji, Y. Cao, Y. Tai, C. Wang, J. Li and F. Huang, “Real-world super-resolution via kernel estimation and noise injection,” IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. Work., 2020, pp. 1914–1923, doi: 10.1109/CVPRW50498.2020.00241.
[7] Z. Luo, Y. Huang, S. Li, L. Wang and T. Tan, “Unfolding the alternating optimization for blind super resolution,” Adv. Neural Inf. Process. Syst., vol. 33, pp. 5632-5643 , 2020.
[8] J. Gu, H. Lu, W. Zuo, and C. Dong, “Blind super-resolution with iterative kernel correction,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., 2019, pp. 1604–1613, doi: 10.1109/CVPR.2019.00170.
[9] X. Wang, L. Xie, C. Dong and Y. Shan, “Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data,” Proc. IEEE Int. Conf. Comput. Vis., 2021, pp. 1905–1914, doi: 10.1109/ICCVW54120.2021.00217.
[10] K. Zhang, J. Liang, L. Van Gool and R. Timofte, “Designing a Practical Degradation Model for Deep Blind Image Super-Resolution,” Proc. IEEE Int. Conf. Comput. Vis., 2021, pp. 4771–4780, doi: 10.1109/ICCV48922.2021.00475.
[11] L. Liu and S. Liu, “Remote detection of human vital sign with stepped-frequency continuous wave radar,” IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., vol. 7, no. 3, pp. 775–782, 2014, doi: 10.1109/JSTARS.2014.2306995.
[12] M. Elad and A. Feuer, “Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images,” IEEE Trans. Image Process., vol. 6, no. 12, pp. 1646–1658, 1997, doi: 10.1109/83.650118.
[13] C. Liu and D. Sun, "On Bayesian Adaptive Video Super Resolution," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 2, pp. 346-360, Feb. 2014, doi: 10.1109/TPAMI.2013.127.
[14] K. Zhang, W. Zuo and L. Zhang, "Deep Plug-And-Play Super-Resolution for Arbitrary Blur Kernels," IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, pp. 1671-1681, doi: 10.1109/CVPR.2019.00177.
[15] X. Li, C. Chen, S. Zhou, X. Lin, W. Zuo and L. Zhang, “Blind Face Restoration via Deep Multi-scale Component Dictionaries,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), 2020, pp. 399–415, doi: 10.1007/978-3-030-58545-7_23.
[16] Z. Wei, Y. Huang, Y. Chen, C. Zheng and J. Gao, “A-ESRGAN: Training Real-World Blind Super-Resolution with Attention U-Net Discriminators,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), 2023, pp. 16–27, doi: 10.1007/978-981-99-7025-4_2.
[17] M. Zhang and Q. Ling, “Supervised Pixel-Wise GAN for Face Super-Resolution,” IEEE Trans. Multimed., vol. 23, pp. 1938–1950, 2021, doi: 10.1109/TMM.2020.3006414.
[18] C. Saharia, J. Ho, W. Chan, T. Salimans, D. J. Fleet and M. Norouzi, "Image Super-Resolution via Iterative Refinement," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 4, pp. 4713-4726, 1 April 2023, doi: 10.1109/TPAMI.2022.3204461.
[19] J. Jiang, Y. Yu, S. Tang, J. Ma, A. Aizawa and K. Aizawa, “Context-Patch Face Hallucination Based on Thresholding Locality-Constrained Representation and Reproducing Learning,” IEEE Trans. Cybern., vol. 50, no. 1, pp. 324–337, 2020, doi: 10.1109/TCYB.2018.2868891.
[20] Y. Yin, J. P. Robinson, Y. Zhang and Y. Fu, “Joint super-resolution and alignment of tiny faces,” AAAI 2020 - 34th AAAI Conf. Artif. Intell., pp. 12693–12700, 2020, doi: 10.1609/aaai.v34i07.6962.
[21] A. Lugmayr, M. Danelljan, L. Van Gool and R. Timofte, “SRFlow: Learning the Super-Resolution Space with Normalizing Flow,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), 2020, pp. 715–732, doi: 10.1007/978-3-030-58558-7_42.
[22] C. Chen, D. Gong, H. Wang, Z. Li, and K. Y. K. Wong, “Learning Spatial Attention for Face Super-Resolution,” IEEE Trans. Image Process., vol. 30, pp. 1219–1231, 2021, doi: 10.1109/TIP.2020.3043093.
[23] Y. Wang, Y. Hu and J. Zhang, “Panini-Net: GAN Prior Based Degradation-Aware Feature Interpolation for Face Restoration,” Proc. AAAI Conf. Artif. Intell. AAAI 2022, vol. 36, pp. 2576–2584, 2022, doi: 10.1609/aaai.v36i3.20159.
[24] T. Wang et al., “A Survey of Deep Face Restoration: Denoise, Super-Resolution, Deblur, Artifact Removal,” 2022, [Online]. Available: http://arxiv.org/abs/2211.02831, doi: 10.48550/arXiv.2211.02831.
[25] Y. Wang, Y. Hu, J. Yu and J. Zhang, “GAN Prior Based Null-Space Learning for Consistent Super-resolution,” Proc. AAAI Conf. Artif. Intell. AAAI 2023, vol. 37, pp. 2724–2732, 2023, doi: 10.1609/aaai.v37i3.25372.
[26] Y. Liu, Z. Dong, K. Pang Lim and N. Ling, "A Densely Connected Face Super-Resolution Network Based on Attention Mechanism," in IEEE Conference on Industrial Electronics and Applications (ICIEA), Kristiansand, Norway, 2020, pp. 148-152, doi: 10.1109/ICIEA48937.2020.9248111.
[27] R. Ghorbandoost and F. Razzazi, "High Fidelity Reversible information Steganography in Images using Difference Expansion and Appropriate Region Selection," Journal of Information and Communication Technology in Policing, vol. 3, no. 9, pp. 1-16, 2022, doi: 10.22034/pitc.2022.1265729.1101[in persian].
[28] I. Hadinejad, M. A. Amiri and M. H. Fahimifar, "An Optimum Method for Noise Reduction and Quality Improvement of the Passive Millimeter Wave Images Based on Nonsubsampled Shearlet Transform and Improved Adaptive Median Filter," Journal of Information and Communication Technology in Policing, vol. 3, no. 12, pp. 30-43, 2022, doi: 10.22034/pitc.2023.1271283.1179 [in Persian].
[29] M. S. Kalami Yazdi, M. Nezhadshahbodaghi and M. R. Mosavi, "INS/Image Integrated Navigation System based on Deep Learning in order to Monitor the Places Traveled by Drivers," Journal of Information and Communication Technology in Policing, vol. 3, no. 11, pp. 35-46, 2022, doi: 10.22034/pitc.2022.1270483.1174 [in persian].
[30] P. Samadinia, , K. Rahbar and A. Broumandnia, "Efficient Multi-Focus Image Fusion via Depthmap," Journal of Information and Communication Technology in Policing, vol. 3, no. 10, pp. 59-70, 2022, doi: 10.22034/pitc.2022.1269046.1148 [in persian].
[31] M. Shokoohi, "Introducing an intelligent system for detecting traffic signs with deep learning to reduce road accidents," Journal of Information and Communication Technology in Policing, vol. 3, no. 10, pp. 47-58, 2022, doi: 10.22034/pitc.2022.1268864.1145 [in persian].
[32] Z. Liu et al., “Fine-Grained Face Swapping Via Regional GAN Inversion,” IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada, 2023, pp. 8578-8587, doi: 10.1109/CVPR52729.2023.00829.
[33] A. Sargsyan, S. Navasardyan, X. Xu and H. Shi, “MI-GAN: A Simple Baseline for Image Inpainting on Mobile Devices,” Proc. IEEE Int. Conf. Comput. Vis., 2023, pp. 7301–7311, doi: 10.1109/ICCV51070.2023.00674.