AI Safety and Trustworthiness Group----Visual Information Processing and Learning (VIPL)

Location：

Home > Research>AI Safety and Trustworthiness Group

AI Safety and Trustworthiness Group

Leader： Jie Zhang, Associate Professor; Shiguang Shan, Professor

Email： zhangjie at ict dot ac dot cn, sgshan at ict dot ac dot cn

Introduction of research group

With the support of projects such as Strategic Priority Research Program of the Chinese Academy of Sciences, National Key R&D Program of China, and the National Natural Science Foundation, fundamental and applied researches are conducted around the intrinsic and derivative security issues of artificial intelligence. By analyzing the inherent mechanisms of algorithm defects, a comprehensive security evaluation system on artificial intelligence algorithm is established, exploring defect and risk mitigation mechanisms, and breaking through the theoretical and technical bottlenecks of "trustworthy, manageable, and controllable" intelligent algorithms, to ensure the safe application of intelligent algorithms.

Research

The research group conducts the following studies centering on the endogenous security issues and derivative security issues of intelligent algorithms:

1. Adversarial Attacks and Defenses:

a) Adversarial Attacks: Explore how to improve the transferability of adversarial samples.

b) Adversarial Defenses: Improve the adversarial robustness of models from the perspectives of robust structure design, efficient adversarial training, etc.

2. Backdoor Attacks and Defenses:

a) Backdoor Attacks: Explore how to enhance the attack success rate, concealment and stability of backdoors.

b) Backdoor Defenses: Investigate methods on backdoor detection, trigger localization and backdoor removal.

3. Out-of-Distribution Generalization and Detection:

Research on theoretical analysis for trustworthy AI, effective domain shift measurement and domain generalization methods, etc.

4. Security Assessment of Multimodal Large Models:

Evaluate the fundamental capabilities, fairness, privacy leakage risks, hallucinations, and value misalignment of multimodal large models.

5. Deepfakes and forgery detection, liveness detection:

a) Digital world: Forgery methods such as generating specific individuals' voices, voice-driven synthesis, expression transfer, and image/video forgery detection methods.

Expression transfer

b) Physical World: Face anti-spoofing.

（a) Distribution differences between fake faces and real faces.

（b) Single-side domain generalization framework.

Papers

Journal Papers

Jie Zhang, Zhifan Wan, Lanqing Hu, Stephen Lin, Shuzhe Wu, Shiguang Shan. Collaboratively Self-supervised Video Representation Learning for Action Recognition. IEEE Transactions on Information Forensics & Security (TIFS), 2025. (Accepted)
Zheng Yuan, Jie Zhang, Zhaoyan Jiang, Liangliang Li, Shiguang Shan. Adaptive Perturbation for Adversarial Attack. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Vol. 46, No. 8, pp. 5663-5676, 2024.
Zheng Yuan, Jie Zhang, Yude Wang, Shiguang Shan, Xilin Chen. Towards Robust Semantic Segmentation against Patch-based Attack via Attention Refinement. International Journal of Computer Vision (IJCV), Vol. 132, No. 11, pp. 5270-5292, 2024.
Xingming Long, Jie Zhang, Shuzhe Wu, Xin Jin, Shiguang Shan. Dual Sampling Based Causal Intervention for Face Anti-Spoofing with Identity Debiasing. IEEE Transactions on Information Forensics and Security (TIFS), Vol. 19, pp. 851-862, 2023.
Yunpei Jia, Jie Zhang*, Shiguang Shan. Dual-Branch Meta-learning Network with Distribution Alignment for Face Anti-spoofing. IEEE Transactions on Information Forensics and Security, Vol.17, pp 138-151, 2022.
Yunpei Jia, Jie Zhang, Shiguang Shan and Xilin Chen. Unified Unsupervised and Semi-supervised Domain Adaptation Network for Cross-scenario Face Anti-spoofing. Pattern Recognition, 115: 107888, 2021.

Conference Papers

Zonghui Guo, Xinyu Han, Jie Zhang, Shiguang Shan, Haiyong Zheng. Video Harmonization with Triplet Spatio-Temporal Variation Patterns. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 19177-19186, Seattle WA, USA, Jun. 17-21, 2024.
Sibo Wang, Jie Zhang, Zheng Yuan, Shiguang Shan. Pre-trained Model Guided Fine-Tuning for Zero-Shot Adversarial Robustness. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 24502-24511, Seattle WA, USA, Jun. 17-21, 2024.
Zhongqi Wang, Jie Zhang, Shiguang Shan, Xilin Chen. T2IShield: Defending Against Backdoors on Text-to-Image Diffusion Models. European Conference on Computer Vision (ECCV), pp. 107-124, Mico Milano, Italy, Sep 29-Oct 4, 2024.
Zheng Yuan, Jie Zhang, Shiguang Shan. Adaptive Image Transformations for Transfer-based Adversarial Attack. Proceedings of the 17th European Conference on Computer Vision (ECCV), Oct. 23-27, 2022, Tel Aviv, Isreal / Cyberspace.
Zheng Yuan, Jie Zhang, Yunpei Jia, Chuanqi Tan, Tao Xue and Shiguang Shan. Meta Gradient Adversarial Attack. IEEE/CVF International Conference on Computer Vision (ICCV), pp. 7748-7757, Montreal, Canada, Oct.11-17, 2021.
Yunpei Jia, Jie Zhang, Shiguang Shan, Xilin Chen, "Single-Side Domain Generalization for Face Anti-Spoofing," IEEE Conference on Computer Vision and Pattern Recognition(CVPR), pp. 8484-8493, 2020.