Multimodal Biometric
Leader: Hu Han (Associate Professor)
Email: hanhu [at] ict.ac.cn
Introduction of research group

Bio-perception oriented intelligent computing (BPOIC) is an interdisciplinary field spanning computer science, psychology, cognitive science, and medical science. BPOIC is essential for deep understanding of human by machines covering visible attributes (e.g., demographic expression) to latent characteristics (e.g., psychological and physiological). BPOIC is challenging because of data source diversity, modality heterogeneity, data or label missing, data imbalance, etc. We aim at the research and development of methods for modal image/video enhancement, visual attribute learning, intelligent healthcare, and representation learning, etc. The goal is to establish the capability of robust bio-perception under unconstrained scenarios.

Research

1. BPOIC problem: RGB-D face recognition, 3D face modeling, biometrics and soft-biometrics, visual attribute learning, presentation attack detection, emotion recognition; rPPG based physiological measurement, medial image analysis (XRay, CT, MRI, Echocardiography, etc.).

2. BPOIC methodology: Multimodal data enhancement and processing based on self-supervised learning, cross-modality correlation, and domain knowledge; multi-modality complementary feature representation learning and cross-modality correlation modeling; spatial-temporal representation learning; multi-task leaning via task correlation and heterogeneity modeling; learning with missing and noisy labels.


Papers

Journal Papers

  • Xuesong Niu, Shiguang Shan*, Hu Han, and Xilin Chen, \"RhythmNet: End-to-end Heart Rate Estimation from Face via Spatial-temporal Representation,\" IEEE Transactions on Image Processing (T-IP), vol. 29, pp. 2409-2423, 2020.
  • Jiancheng Cai, Hu Han, Shiguang Shan, and Xilin Chen, \"FCSR-GAN: Joint Face Completion and Super-resolution via Multi-task Learning,\" IEEE Transactions on Biometrics, Behavior, and Identity Science, vol. 2, no. 2, pp. 109-121, April 2020.
  • Haomiao Sun, Hongyu Pan, Hu Han and Shiguang Shan. Deep Conditional Distribution Learning for Age Estimation. IEEE Transactions on Information Forensics and Security (TIFS), 16:4679-4690, 2021.
  • Guoqing Wang, Hu Han, Shiguang Shan and Xilin Chen. Unsupervised Adversarial Domain Adaptation for Cross-domain Face Presentation Attack Detection. IEEE Transactions on Information Forensics and Security (TIFS), 16:56-69, 2021.
  • Xuesong Niu, Hu Han, and Shiguang Shan, "Remote Photoplethysmography-based Physiological Measurement: A Survey," Journal of Image and Graphics (CIG), vol.25, no.11, pp.2321-2336, 2020.
  • Abhijit Das, Xuesong Niu, Antitza Dantcheva, S L Happy, Hu Han, Radia Zeghari, Philippe Robert, Shiguang Shan, Francois Bremond and Xilin Chen. A Spatio-temporal Approach for Apathy Classification. IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2021. (Accepted)
  • Keyurkumar Patel, Hu Han, Anil K.Jain, \"Secure Face Unlock:Spoof Detection on Smartphones,\" IEEE Transactions on Information Forensics and Security, vol. 11, no. 10, pp. 2268-2283, 2016.
  • Di Wen, Hu Han, Anil K.Jain, "Face Spoof Detection with Image Distortion Analysis," IEEE Transactions on Information Forensics and Security, vol. 10, no. 4, pp. 746-761, 2015.
  • Hu Han, Charles Otto, Xiaoming Liu, Anil K.Jain, \"Demographic Estimation from Face Images:Human vs. Machine Performance,\" IEEE Transactions on Pattern Analysis and Machine Intelligence(TPAMI2015), vol. 37, no. 6, pp. 1148-1161, 2015.
  • Hu Han, Anil K.Jain, Fang Wang, Shiguang Shan, Xilin Chen, \"Heterogeneous Face Attribute Estimation:A Deep Multi-Task Learning Approach,\" IEEE Transactions on Pattern Analysis and Machine Intelligence(TPAMI), vol. 40, no. 11, pp. 2597-2609, Nov. 2018.
  • Hu Han, Jie Li, Anil Jain, Shiguang Shan, Xilin Chen, \"Tattoo Image Search at Scale: Joint Detection and Compact Representation Learning,\" IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), vol. 41, no. 10, pp. 2333-2348, Oct. 1, 2019.

Conference Papers

  • Xuesong Niu, Hu Han, Shiguang Shan, and Xilin Chen, \"Multi-label Co-regularization for Semi-supervised Facial Action Unit Recognition,\" The Thirty-third Annual Conference on Neural Information Processing Systems (NeurIPS), Vancouver, Canada, Dec. 8-14, 2019.
  • Guoqing Wang, Hu Han, Shiguang Shan, Xilin Chen, \"Cross-domain Face Presentation Attack Detection via Multi-domain Disentangled Representation Learning,\" IEEE Conference on Computer Vision and Pattern Recognition(CVPR 2020), pp. 6678-6687, 2020.
  • Xiaobai Li, Hu Han, Hao Lu, Xuesong Niu, Zitong Yu, Antitza Dantcheva, Guoying Zhao, Shiguang Shan, "The 1st Challenge on Remote Physiological Signal Sensing (RePSS)," in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp.1-8, Seattle, Washington, USA, Jun. 14-19, 2020.
  • Xuesong Niu, Zitong Yu, Hu Han, Xiaobai Li, Shiguang Shan, Guoying Zhao, "Video-based Remote Physiological Measurement via Cross-verified Feature Disentangling," Proceedings of the 16th European Conference on Computer Vision (ECCV), Glasgow, UK / Cyberspace, August 23-28, 2020.
  • Abhijit Das, Hao Lu, Hu Han, Antitza Dantcheva, Shiguang Shan and Xilin Chen. BVPNet: Video-to-BVP Signal Prediction for Remote Heart Rate Estimation. International Conference on Automatic Face and Gesture Recognition (FG), 2021. (Accepted)
  • Jinsong Chen, Hu Han and Shiguang Shan. Towards High-Fidelity Face Self-occlusion Recovery via Multi-view Residual-based GAN Inversion. AAAI Conference on Artificial Intelligence (AAAI), 2022. (Accepted)
  • Guoqing Wang, Hu Han, Shiguang Shan, Xilin Chen, \"Improving Cross-database Face Presentation Attack Detection via Adversarial Domain Adaptation,\" 12th IAPR International Conference on Biometrics (ICB 2019), pp. 1-8, Crete, Greece, Jun 4-7, 2019.
  • Mengying Hu, Hu Han, Shiguang Shan, Xilin Chen, “Weakly Supervised Image Classification through Noise Regularization,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR2019), pp. 11517-11525,  Long Beach, California, USA, June 16-20, 2019.
  • Xuesong Niu, Xingyuan Zhao, Hu Han, Abhijit Das, Antitza Dantcheva, Shiguang Shan, Xilin Chen,"Robust Remote Heart Rate Estimation from Face Utilizing Spatial-temporal Attention," Proceedings of the 14th IEEE International Conference on Automatic Face and Gesture Recognition (FG), pp. 1-8, Lille, France, May 14-18, 2019.
  • Shikang Yu, Hu Han, Shiguang Shan, Antitza Dantcheva, and Xilin Chen, "Improving Face Sketch Recognition via Adversarial Sketch-Photo Transformation," Proceedings of the 14th IEEE International Conference on Automatic Face and Gesture Recognition (FG), pp. 1-8, Lille, France, May 14-18, 2019.
  • Xuesong Niu, Hu Han, Songfan Yang, Yan Huang, Shiguang Shan, \" Local Relationship Learning With Person-Specific Shape Regularization for Facial Action Unit Detection,\" IEEE Conference on Computer Vision and Pattern Recognition(CVPR), pp. 11917-11926, Long Beach, California, USA, June 16-20, 2019.
  • Guoqing Wang, Chuanxin Lan, Hu Han, Shiguang Shan, and Xilin Chen, \"Multi-modal Face Presentation Attack Detection via Spatial and Channel Attentions,\" IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 1-6, Long Beach, CA, USA, Jun. 16-20, 2019.
  • Jiancheng Cai, Hu Han, Shiguang Shan, and Xilin Chen, "FCSR-GAN: End-to-end Learning for Joint Face Completion and Super-resolution," Proceedings of the 14th IEEE International Conference on Automatic Face and Gesture Recognition (FG), pp. 1-8, Lille, France, May 14-18, 2019.
  • Jiyun Cui, Hu Han, Shiguang Shan, Xilin Chen. \"RGB-D Face Recognition: A Comparative Study of Representative Fusion Schemes,\" The 13th Chinese Conference on Biometric Recognition (CCBR2018), Xinjiang, China, Aug. 11-12, 2018.
  • Mengying Hu, Hu Han, Shiguang Shan, Xilin Chen, \"Multi-label Learning from Noisy Labels with Non-linear Feature Transformation,\" Asian Conference on Computer Vision 2018(ACCV2018), 2-6 Dec. 2018, Perth Western Australia.
  • Xuesong Niu, Hu Han, Shiguang Shan, Xilin Chen, \"VIPL-HR: A Multi-modal Database for Pulse Estimation from Less-constrained Face Video,\" Asian Conference on Computer Vision 2018(ACCV2018), 2-6 Dec. 2018, Perth Western Australia.
  • Hongyu Pan, Hu Han, Shiguang Shan, Xilin Chen, \"Mean-Variance Loss for Deep Age Estimation from A Face,\" IEEE Conference on Computer Vision and Pattern Recognition(CVPR2018), pp. 5285-5294, Salt Lake City, USA, Jun. 18-22, 2018.
  • Hao Zhang, Hu Han, Jiyun Cui, Shiguang Shan, Xilin Chen. \"RGB-D Face Recognition via Deep Complementary and Common Feature Learning,\" The 13th IEEE International Conference on Automatic Face and Gesture Recognition (FG2018), pp. 1-8, Xi\'an, China, May 15-19, 2018.
  • Yuanshun Cui, Jie Li, Hu Han, Shiguang Shan, Xilin Chen, \"TKDN: Scene Text Detection via Keypoints Detection,\" Asian Conference on Computer Vision 2018(ACCV2018), 2-6 Dec. 2018, Perth Western Australia
  • Gang Chen, Xufen Cai, Hu Han, Shiguang Shan, Xingguang Song, Xilin Chen, \"HeadNet: Pedestrian Head Detection Utilizing Body in Context,\" The 13th IEEE Conference on Automatic Face and Gesture Recognition (FG2018), pp. 1-8, Xi\'an, China, 2018.