Unsupervised Domain Adaptation for image classification based on Deep Neural Networks

Subject Areas : Intelligent Multimedia Processing and Communication Systems (IMPCS)

Amirfarhad  Farhadi ¹ (Ph.D. Student, Department of Computer Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran)
Mitra  Mirzarezaee ² (Assistant professor, Department of Computer Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran)
Arash  Sharifi ³ (Assistant professor, Department of Computer Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran)
Mohammad  Teshnehlab ⁴ (Professor, Department of Control Engineering, K.N. Toosi University of Technology, Tehran, Iran)

Received: 2023-01-17 Accepted : 2023-09-15 Published : 2023-03-21

Keywords: WMMD, fuzzy clustering, CNN, GaN, Domain adaptation,

Abstract :

Introduction: Domain adaptation has become an important issue today. A high percentage of data processing domain adaptation is done with a significant percentage of studies related to deep learning. Traditional methods often ignore the distance between the intra-class in source domain and target domain. As a result, models can be sensitive to outliers and noisy data, additionally increasing the negative transfer in the model. This method applied GAN to extract appropriate features and then used Fuzzy c-means to cluster train datasets in the target domain. Finally, based on the WMMD metric and CNN, the model estimates the final label data. Five real datasets are selected to generate eight transfer tasks. The results show that the superiority of the proposed model lies in transferring more knowledge from the source domain to the target domain.Method: In this approach, firstly based on GAN extracting features from source domains and the target domain (without labels), then label estimation by Fuzzy c-means clustering, finding the center of Fuzzy c-means on target domain data, new data points with labels in target domain as a new input to feature extraction module and regenerate features by GAN based on new pseudo labels. Afterward, we apply WMMD metrics based on CNN to ultimately assign labels for the target domain. Consequently, classification tasks have been done.Results: Empirical results on various benchmark datasets showcase the exceptional performance of the proposed method compared to state-of-the-art DA approaches, validating the proposed Deep-Learning Unsupervised Domain Adaptation approach efficacy. Overall, the approach shows potential for advancing domain adaptation research by offering an efficient and resilient approach for addressing domain shifts in real-world applications. Experimental results on visual object recognition and a digit dataset reveal that the proposed algorithm is robust, flexible, and significantly superior regarding accuracy compared to the baseline DA approaches. Based on the three and combined digit datasets, 1.7% and 2.4% accuracy improvement are achieved, respectively, compared to the best baseline DA approach results.Discussion: In this research, we addressed the challenging issues of outlier and negative transfer in the context of domain adaptation. Despite significant progress in domain adaptation techniques, outliers and negative transfer instances continue to hinder models' generalization performance across different domains. Based on DNNs and the WMMD metric, our proposed method was designed to mitigate these issues and effectively enhance knowledge transfer between domains. 

References:

• P.-A. I. Good fellow, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. and a. Y. B. Courville, "Generative adversarial nets," presented at the 27th Advances in Neural Information Processing Systems, 2014.


• C. Yoshua Bengio, Pascal Vincent, "Representation Learning: A Review and New Perspectives," IEEE transactions on pattern analysis and machine intelligence, vol. 35, no. 8, pp. 1798-1828, 2013.


• B. Yann LeCun, Geoffrey Hinton, "Deep learning," Nature, vol. 521, pp. 436–444, 2015.


• Z. Rongrong Peng, and Peiming Shi "Multi-Representation Domain Adaptation Network with Duplex Adversarial Learning for Hot-Rolling Mill Fault Diagnosis," Entopy, vol. 25, no. 1, p. 83, 2023.


• Z. Peiyi Wei, Yanping Tang, Zhixin Li, Zhiwen Wang, "Reinforced domain adaptation with attention and adversarial learning for unsupervised person Re-ID," Applied Intelligence, vol. 53, pp. pages4109–4123, 2023.


• S. Alex Krizhevsky, Geoffrey E. Hinton, "ImageNet Classification with Deep Convolutional Neural Networks," presented at the 25th Advances in Neural Information Processing Systems 2012.


• W. PEEYUSH SINGHAL, SHEELA RAMANNA, and A. K. KOTECHA, "Domain Adaptation: Challenges, Methods, Datasets, and Applications," IEEE Access, vol. 11, 2023.


• C. Mingsheng Long, Jianmin Wang, Michael I. Jordan, "Learning Transferable Features with Deep Adaptation Networks," presented at the 32nd International Conference on Machine Learning, Lille, France, 2015.


• U. Yaroslav Ganin, Hana Ajakan and P. Germain, "Domain-Adversarial Training of Neural Networks," Journal of Machine Learning Research, vol. 17, pp. 1-35, 2016.


• F. Jindong Wang, Yiqiang Chen,Han Yu, Philip S. Yu, "Visual Domain Adaptation with Manifold Embedded Distribution Alignment," in 26th ACM international conference on Multimedia, 2018.


• D. Hongliang Yan, Peihua Li, Qilong Wang, Yong Xu, Wangmeng Zuo, "Mind the Class Weight Bias: Weighted Maximum Mean Discrepancy for Unsupervised Domain Adaptation," presented at the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.


• Z. Yiju Yang , Guanyu Li , Taejoon Kim , Guanghui Wang, "An unsupervised domain adaptation model based on dual-module adversarial training," Neurocomputing, vol. 475, pp. 102-111, 2022.


• M. B. Arthur Gretton, Malte J. Rasch, Bernhard Scho ̈lkopf, Alexander Smola, "A Kernel Two-Sample Test," Journal of Machine Learning Research, vol. 13, pp. 723-773, 2012.


• Zadeh, "Fuzzy Sets," nformation and Control, vol. 8, pp. 338-353, 1965.


• E. James C. Bezdek, William Full, "FCM: The fuzzy c-means clustering algorithm," Computers & Geosciences, vol. 10, no. 2-3, pp. 191-203, 1984.


• T. S. Lida Naderlou, "Presenting a model for Multi-layer Dynamic Social Networks to discover Influential Groups based on a combination of Developing Frog Leaping Algorithm and C-means Clustering," Intelligent Multimedia processing and Communication Systems (IMPCS), vol. 3, no. 3, pp. 29-39, 2022.


• C. Martin Arjovsky, and L ́eon Bottou, "Wasserstein GAN," arXiv:1701.07875v3, 2017.


• L. B. Diederik P. Kingma, "ADAM: A METHOD FOR STOCHASTIC OPTIMIZATION," arXiv:1412.6980v9, 2015.


• N. Mahdi Hariri, "Improve the Quality of Mammogram Images by Image Processing Techniques," Intelligent Multimedia processing and Communication Systems (IMPCS), vol. 3, no. 1, pp. 57-69, 2022.