中科院计算所视觉信息处理与学习组
中科院计算所视觉信息处理与学习组


News / Events

VIPL's paper on person re-identification is accepted by IEEE TIP

Date of publication:2021-03-16      Number of clicks: 0


  Recently, one paper on language-based person search is accepted by the journal IEEE TIP. The full name of IEEE TIP is Transactions on Image Processing, which is an international journal on computer vision and image processing with an impact factor of 6.79 announced in 2020. The paper information is as follow:

 

  Yucheng Chen, Rui Huang, Hong Chang, Chuanqi Tan, Tao Xue and Bingpeng Ma. “Cross-Modal Knowledge Adaptation for Language-Based Person Search”, IEEE Transactions on Image Processing (TIP), 2021. (Accepted)

 

  Language-based person search is a challenging retrieval task. The inconsistent representation of different modalities makes it difficult to directly measure the similarity between visual images and textual descriptions. The common representation learning methods have achieved certain success on this task. Most of the common representation learning methods project image features and text features into a shared space in an equal manner. However, the information contained in an image and a text is not equal. Since text provides a description of the person in an image, it summarizes partial image information. In other words, images contain image-specific information that is rarely described by texts. Unequal amount of information of image and text will result in redundancy of image information and difficulty in aligning features across different modalities. Image-specific information may also be detrimental to the learning of image representation. Considering that text can be used to guide image features to enrich them with important person details while avoiding the interference of image-specific information, in this paper, we propose a method named Cross-Modal Knowledge Adaptation (CMKA). Specially, text-to-image guidance is obtained at different levels: individuals, lists, and classes. By combining these levels of knowledge adaptation, the image-specific information is suppressed, and the common space of image and text is better constructed. The overall framework of CMKA is as follows: 



  In summary, the main contributions of this work include: 1) A cross-modal knowledge adaptation method is proposed. By combining different levels of knowledge adaptation, the information between modalities is balanced and more textual-visual correspondences are learned; 2) The effectiveness of CMKA is verified on the language-based person search dataset; 3) The effectiveness of CMKA is verified on the different retrieval tasks (including image-text bi-directional retrieval and image-to-text re-ID).


中科院计算所视觉信息处理与学习组
  • 单位地址:北京海淀区中关村科学院南路6号
  • 邮编:100190
  • 联系电话:010-62600514
  • Email:yi.cheng@vipl.ict.ac.cn
  • Valse

  • 深度学习大讲堂

版权所有 @ 中科院计算所视觉信息处理与学习组 京ICP备05002829号-1 京公网安备1101080060