Visual Information Processing and Learning

Affective Computing

Leader：Jiabei Zeng / Shiguang Shan (Professor)

Email：jiabei.zeng [at] vipl.ict.ac.cn; sgshan [at] ict.ac.cn

Affective Computing is computing that relates to, arises from, or deliberately influences emotion or other affective phenomena (Picard, MIT Press 1997). Affective computing group focus on perceiving and analyzing peoples’ facial expressions, emotions or other affective phenomena mainly according to the visual information.

Research

The main research topics focus on:

A. Algorithms and methodologies that address the basic issues in recognizing emotions, detection facial action unit, estimating affect valence and arousal. For example,
1) How to train facial expression recognition systems from ill-annotated data (e.g., incorrect labels, inconsistent labels)?
2) How to recognize facial expression under the open scenarios (e.g., partially occluded faces, with multiple modality)?

B. Applications
1) Students’ engagement estimation.
2) Credit risk assessment from facial expressions

Papers

Journal Papers

1. Mengyi Liu, Ruiping Wang, Shiguang Shan, Xilin Chen, "Learning Prototypes and Similes on Grassmann Manifold for Spontaneous Expression Recognition," Computer Vision and Image Understanding, vol. 147, pp. 95-101, 2016. 【pdf】

2. Mengyi Liu, Shiguang Shan, Ruiping Wang, Xilin Chen, "Learning Expressionlets via Universal Manifold Model for Dynamic Facial Expression Recognition," IEEE Transactions on Image Processing, vol. 25, no. 12, pp. 5920-5932, 2016. 【pdf】

3. Mengyi Liu, Ruiping Wang, Shaoxin Li, Zhiwu Huang, Shiguang Shan, Xilin Chen, "Video Modeling and Learning on Riemannian Manifold for Emotion Recognition in the Wild," Journal on Multimodal User Interfaces, vol. 10, no. 2, pp. 113-124, 2016. 【pdf】

4. Mengyi Liu, Shaoxin Li, Shiguang Shan, Xilin Chen, "AU-inspired Deep Networks for Facial Expression Feature Learning," Neurocomputing, vol. 159, pp. 126-136, 2015.

Conference Papers

1. Yong Li, Jiabei Zeng, Shiguang Shan, Xilin Chen, "Self-supervised Representation Learning from Videos for Facial Action Unit Detection", IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Feb. 2019.(Accepted)

2. Zijia Lu, Jiabei Zeng, Shiguang Shan, Xilin Chen, "Zero-Shot Facial Expression Recognition with Multi-Label Label Propagation," Asian Conference on Computer Vision 2018(ACCV2018), 2-6 Dec. 2018, Perth Western Australia 【pdf】

3. Jiabei Zeng, Shiguang Shan, Xilin Chen, "Facial Expression Recognition with Inconsistently Annotated Datasets," To appear in Proc. European Conference on Computer Vision (ECCV2018), pp. 1-16, Munich, Germany, 2018. 【pdf】

4. Mengyi Liu, Xin Liu, Yan Li, Xilin Chen, Alexander Hauptmann, Shiguang Shan, "Exploiting Feature Hierarchies With Convolutional Neural Networks for Cultural Event Recognition," IEEE International Conference on Computer Vision(ICCV2015) Workshops, pp. 32-37, 2015. 【pdf】

5. Mengyi Liu, Shaoxin Li, Shiguang Shan, Ruiping Wang, Xilin Chen, "Deeply Learning Deformable Facial Action Parts Model for Dynamic Expression Analysis," Asian Conference on Computer Vision(ACCV2014), pp. 143-157, 2014.

6. Mengyi Liu, Shiguang Shan, Ruiping Wang, Xilin Chen, "Learning Expressionlets on Spatio-Temporal Manifold for Dynamic Facial Expression Recognition," IEEE Conference on Computer Vision and Pattern Recognition(CVPR2014), pp. 1749-1756, 2014.

7. Mengyi Liu, Ruiping Wang, Shaoxin Li, Zhiwu Huang, Shiguang Shan, Xilin Chen, "Combining Multiple Kernel Methods on Riemannian Manifold for Emotion Recognition in the Wild," ACM International Conference on Multimodal Interaction(ICMI2014), pp. 494-501, 2014.

8. Mengyi Liu, Shaoxin Li, Shiguang Shan, Xilin Chen, "AU-Aware Deep Networks for Facial Expression Recognition," IEEE Conference on Automatic Face and Gesture Recognition(FG2013), pp. 1-6, 2013.

9. Mengyi Liu, Ruiping Wang, Zhiwu Huang, Shiguang Shan, Xilin Chen, "Partial Least Squares Regression on Grassmannian Manifold for Emotion Recognition," ACM International Conference on Multimodal Interaction(ICMI2013), pp. 525-530, 2013.

10. Mengyi Liu, Shaoxin Li, Shiguang Shan, Xilin Chen. Enhancing Expression Recognition in the Wild with Unlabeled Reference Data, Asian Conference on Computer Vision (ACCV), pp. 577-588, 2012.