Affective Computing
Leader: Jiabei Zeng / Shiguang Shan (Professor)
Email: jiabei.zeng [at] vipl.ict.ac.cn; sgshan [at] ict.ac.cn
Introduction of research group

  Affective Computing is computing that relates to, arises from, or deliberately influences emotion or other affective phenomena (Picard, MIT Press 1997). Affective computing group focus on perceiving and analyzing peoples’ facial expressions, emotions or other affective phenomena mainly according to the visual information.

Research

The main research topics focus on:

A. Algorithms and methodologies that address the basic issues in recognizing emotions, detection facial action unit, estimating affect valence and arousal. For example,

    1) How to train facial expression recognition systems from ill-annotated data (e.g., incorrect labels, inconsistent labels)?

    2) How to recognize facial expression under the open scenarios (e.g., partially occluded faces, with multiple modality)?

B. Applications

     1) Students’ engagement estimation.

     2) Credit risk assessment from facial expressions

Papers

Journal Papers

  • Yong Li, Jiabei Zeng and Shiguang Shan. Learning Representations for Facial Actions from Unlabeled Videos. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol.44, no.1, pp. 302-317, 2022.
  • Yunjia Sun, Jiabei Zeng, and Shiguang Shan. Gaze estimation with semi-supervised eye landmark detection as an auxiliary task. Pattern Recognition (PR), 146: 109980, 2024.
  • Mengyi Liu, Shaoxin Li, Shiguang Shan, Xilin Chen, "AU-inspired Deep Networks for Facial Expression Feature Learning," Neurocomputing, vol. 159, pp. 126-136, 2015.
  • Mengyi Liu, Ruiping Wang, Shaoxin Li, Zhiwu Huang, Shiguang Shan, Xilin Chen, "Video Modeling and Learning on Riemannian Manifold for Emotion Recognition in the Wild," Journal on Multimodal User Interfaces, vol. 10, no. 2, pp. 113-124, 2016.
  • Mengyi Liu, Ruiping Wang, Shiguang Shan, Xilin Chen, "Learning Prototypes and Similes on Grassmann Manifold for Spontaneous Expression Recognition," Computer Vision and Image Understanding, vol. 147, pp. 95-101, 2016.
  • Mengyi Liu, Shiguang Shan, Ruiping Wang, Xilin Chen, "Learning Expressionlets via Universal Manifold Model for Dynamic Facial Expression Recognition," IEEE Transactions on Image Processing, vol. 25, no. 12, pp. 5920-5932, 2016.

Conference Papers

  • Yujian Yuan, Jiabei Zeng, Shiguang Shan. Describe Your Facial Expressions by Linking Image Encoders and Large Language Models. British Machine Vision Conference, Aberdeen, UK, Nov. 20-24, 2023.
  • Fei Chang, Jiabei Zeng, Qiaoyun Liu, Shiguang Shan. Gaze Pattern Recognition in Dyadic Communication. Proceedings of the 2023 Symposium on Eye Tracking Research and Applications(ETRA), Tübingen, Germany, May. 30-Jun. 2, 2023.
  • Xin Cai, Jiabei Zeng, Shiguang Shan, Xilin Chen. Source-Free Adaptive Gaze Estimation by Uncertainty Reduction. IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, Canada, Jun. 18-22, 2023.
  • Yuanyuan Liu, Wei Dai, Chuanxu Feng, Wenbin Wang, Guanghao Yin, Jiabei Zeng, Shiguang Shan. MAFW: A Large-scale, Multi-modal, Compound Affective Database for Dynamic Facial Expression Recognition in the Wild. ACM Conference on Multimedia, 2022.
  • Yong Li, Jiabei Zeng, Shiguang Shan, Xilin Chen, "Self-supervised Representation Learning from Videos for Facial Action Unit Detection", IEEE Conference on Computer Vision and Pattern Recognition(CVPR), pp.10924-10933, Long Beach, California, USA, June 16-20, 2019.
  • Yunjia Sun, Jiabei Zeng, Shiguang Shan, Xilin Chen. Cross-Encoder for Unsupervised Gaze Representation Learning. IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3702-3711, Montreal, Canada, Oct. 11-17, 2021.
  • Xin Cai, Jiabei Zeng and Shiguang Shan. Landmark-aware Self-supervised Eye Semantic Segmentation. Unknown Aware Feature Learning for Face Forgery Detection. International Conference on Automatic Face and Gesture Recognition (FG), 2021. (Accepted)
  • Xuran Sun, Jiabei Zeng and Shiguang Shan. Emotion-aware Contrastive Learning for Facial Action Unit Detection. International Conference on Automatic Face and Gesture Recognition (FG), 2021. (Accepted)
  • Mengyi Liu, Xin Liu, Yan Li, Xilin Chen, Alexander Hauptmann, Shiguang Shan, "Exploiting Feature Hierarchies With Convolutional Neural Networks for Cultural Event Recognition," IEEE International Conference on Computer Vision(ICCV2015) Workshops, pp. 32-37, 2015.
  • Jiabei Zeng, Shiguang Shan, Xilin Chen, "Facial Expression Recognition with Inconsistently Annotated Datasets," To appear in Proc. European Conference on Computer Vision (ECCV2018), pp. 1-16, Munich, Germany, 2018.
  • Zijia Lu, Jiabei Zeng, Shiguang Shan, Xilin Chen, "Zero-Shot Facial Expression Recognition with Multi-Label Label Propagation," Asian Conference on Computer Vision 2018(ACCV2018), 2-6 Dec. 2018, Perth Western Australia