International Journal of Information Engineering and Electronic Business (IJIEEB)
IJIEEB Vol. 16, No. 2, 8 Apr. 2024
Cover page and Table of Contents: PDF (size: 1110KB)
Unified Domain Adaptation with Discriminative Features and Similarity Preservation
PDF (1110KB), PP.39-53
Views: 0 Downloads: 0
Author(s)
Obsa Gilo
1,*
Jimson Mathew
1
Samrat Mondal
1
Index Terms
Distribution shift, Discriminative features, Entropy regularization, Domain adaptation
Abstract
In visual domain adaptation, the goal is to train effective classifiers for the target domain by leveraging information from the source domain. In unsupervised domain adaptation, the source domain provides labeled data while the target domain lacks labels. However, it is crucial to recognize that the source and target domains have different underlying distributions despite sharing the same label space. Directly applying source domain information to the target domain often leads to poor performance due to the distribution gap between the two domains. Unsupervised do- main adaptation aims to bridge this gap and improve performance. We introduce a comprehensive UDADFSP (Unified Domain Adaptation with Discriminative Features and Similarity Preservation) de- signed explicitly for unsupervised domain adaptation to tackle these challenges. Our framework focuses on incorporating discriminative and invariant features. We employ clustering with entropy regularization on the unlabeled target domain to refine the neighbor relationships. This step significantly enhances the alignment between the target and source domains, facilitating a more effective adaptation. Furthermore, we seamlessly incorporate discriminative features while preserving similarity in the source and target domains. We carefully balance the discrimination and similarity aspects by considering linear and non-linear data representations. Extensive testing demonstrates that learning discriminative and similarity features in the same feature space yields significant improvements over several state-of-the-art domain adaptation techniques. In a comparative evaluation, our approach surpasses several existing methods across four diverse cross-domain visual tasks and the Amazon re- view sentiment analysis task.
Cite This Paper
Obsa Gilo, Jimson Mathew, Samrat Mondal, "Unified Domain Adaptation with Discriminative Features and Similarity Preservation", International Journal of Information Engineering and Electronic Business(IJIEEB), Vol.16, No.2, pp. 39-53, 2024. DOI:10.5815/ijieeb.2024.02.04
Reference
[1]Sinno Jialin Pan and Qiang Yang. A survey on transfer learning. IEEE Transactions on knowledge and data engineering, 22(10):1345–1359, 2009.
[2]Hal Daumé III. Frustratingly easy domain adaptation. arXiv preprint arXiv:0907.1815, 2009.
[3]Mei Wang and Weihong Deng. Deep visual domain adaptation: A survey. Neurocomputing, 312:135–153, 2018.
[4]Gabriela Csurka. A comprehensive survey on domain adaptation for visual applications. Domain adaptation in computer vision applications, pages 1–35, 2017.
[5]Arash Saboori and Hassan Ghassemian. Adversarial discriminative active deep learning for domain adaptation in hyperspectral images classification. Interna- tional Journal of Remote Sensing, 42(10):3981–4003, 2021.
[6]A. Gretton K. M. Borgwardt J. Huang, A. J. Smola and B. Scholkopf. Correcting sample selection bias by un- labeled data. 2006.
[7]J. Wang J. Sun Y. Guo M. Long, G. Ding and P. S. Yu. Transfer sparse coding for robust image representation. In CV-PR, 2013.
[8]Obsa Gilo, Jimson Mathew, and Samrat Mondal. Integration of discriminate features and similarity preserving for unsupervised domain adaptation. In 2022 IEEE 19th India Council International Conference (IN- DICON), pages 1–6. IEEE, 2022.
[9]J. T. Kwok S. J. Pan, I. W. Tsang and Q. Yang. Domain adaptation via transfer component analysis. IEEE transactions on neural networks, (22(2):):199–210, 2011.
[10]Domain adaptation problems: A dasvm classification technique and a circular validation strategy. 2009.
[11]F. De la Torre W.-S. Chu and J. F. Cohn. Selective transfer machine for personalized facial action unit dete tion. In CVPR, 2013.
[12]Hoffman J. Saenko K. Tzeng, E. and T. Darrell. Adversarial discriminative domain adaptation. In IEEE CV PR, 2017.
[13]Jianfei Yang, Han Zou, Yuxun Zhou, and Lihua Xie. Robust adversarial discriminative domain adaptation for real-world cross-domain visual recognition. Neurocomputing, 433:28–36, 2021.
[14]Jingyu Wang, Zhenyu Ma, Feiping Nie, and Xuelong Li. Entropy regularization for unsupervised clustering with adaptive neighbors. Pattern Recognition, page 108517, 2022.
[15]Rui Zhang, Xuelong Li, Hongyuan Zhang, and Feiping Nie. Deep fuzzy k-means with adaptive loss and entropy regularization. IEEE Transactions on Fuzzy Sys tems, 28(11):2814–2824, 2019.
[16]Li W. Zhang, J. and P. Ogunbona. Joint geometrical and statistical alignment for visual domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2017.
[17]Wang J. Ding G. Sun J. Long, M. and P. S. Yu. Transfer feature learning with joint distribution adaptation. Proceedings of the IEEE international conference on com- puter vision recognition, 2013.
[18]Wang J. Ding G. Sun J. Long, M. and P. S. Yu. Trans- fer joint matching for unsupervised domain adaptation. In Proceedings of the IEEE international conference on computer vision recognition, 2014.
[19]Wu D. A. (2020 July). Zhang, W. Discriminative joint probability maximum mean discrepancy (djp-mmd) for domain adaptation. In 2020 International Joint Conference on Neural Networks (IJCNN), pages 1–8.
[20]Baochen Sun, Jiashi Feng, and Kate Saenko. Return of frustratingly easy domain adaptation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 30, 2016.
[21]Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. In 9ECCV 2016 Workshops, 2016.
[22]Baochen Sun, Jiashi Feng, and Kate Saenko. Correlation alignment for unsupervised domain adaptation. In Domain Adaptation in Computer Vision Applications, pages 153–171. Springer, 2017.
[23]Przemysław Spurek, Krzysztof Byrski, and Jacek Tabor.Online updating of active function cross- entropy clustering. Pattern Analysis and Applications, 22(4):1409–1425, 2019.
[24]Wenming Cao, Zhongfan Zhang, Cheng Liu, Rui Li, Qianfen Jiao, Zhiwen Yu, and Hau-San Wong. Unsupervised discriminative feature learning via finding a clustering-friendly embedding space. Pattern Recognition, 129:108768, 2022.
[25]Aleix M Martinez and Avinash C Kak. Pca versus lda. IEEE transactions on pattern analysis and machine intelligence, 23(2):228–233, 2001.
[26]Uri Lipowezky. Selection of the optimal prototype sub- set for 1-nn classification. Pattern Recognition Letters, 19(10):907–918, 1998.
[27]Richard O Duda, Peter E Hart, et al. Pattern classifi- cation and scene analysis, volume 3. Wiley New York, 1973.
[28]Arash Abdi, Mohammad Rahmati, and Mohammad M Ebadzadeh. Entropy based dictionary learning for im- age classification. Pattern Recognition, 110:107634, 2021.
[29]Basura Fernando, Amaury Habrard, Marc Sebban, and Tinne Tuytelaars. Unsupervised visual domain adaptation using subspace alignment. In Proceedings of the IEEE international conference on computer vision, pages 2960–2967, 2013.
[30]Baochen Sun and Kate Saenko.Subspace distribution alignment for unsupervised domain adaptation. In BMVC, volume 4, pages 24–1, 2015.
[31]Boqing Gong, Yuan Shi, Fei Sha, and Kristen Grauman. Geodesic flow kernel for unsupervised domain adaptation. In 2012 IEEE conference on computer vision and pattern recognition, pages 2066–2073. IEEE, 2012.
[32]Muhammad Ghifary, David Balduzzi, W Bastiaan Kleijn, and Mengjie Zhang. Scatter component anal- ysis: A unified framework for domain adaptation and domain generalization. IEEE transactions on pattern analysis and machine intelligence, 39(7):1414–1430, 2016.
[33]Nicolas Courty, Rémi Flamary, and Devis Tuia. Do- main adaptation with regularized optimal transport. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 274–289. Springer, 2014.
[34]Devis Tuia and Gustau Camps-Valls.Kernel manifold alignment for domain adaptation.PloS one, 11(2):e0148655, 2016.
[35]MS Rizal Samsudin, Syed AR Abu-Bakar, and Musa M Mokji.Balanced weight joint geometrical and statistical alignment for unsupervised domain adaptation. Journal of Advances in Information Technology Vol, 13(1), 2022.
[36]Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models to new do- mains. In European conference on computer vision, pages 213–226. Springer, 2010.
[37]Iain Matthews. Fast and accurate active appearance models. In Proceedings of the HCSNet workshop on Use of vision in human-computer interaction-Volume 56, pages 3–3, 2006.
[38]Sameer A Nene, Shree K Nayar, Hiroshi Murase, et al.Columbia object image library (coil-100). 1996.
[39]John Blitzer, Mark Dredze, and Fernando Pereira. Biographies, bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification. In Pro- ceedings of the 45th annual meeting of the association of computational linguistics, pages 440–447, 2007.
[40]Zixuan Cao, Yongmei Zhou, Aimin Yang, and Sancheng Peng. Deep transfer learning mechanism for fine-grained cross-domain sentiment classification. Connection Science, 33(4):911–928, 2021.
[41]Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.