Congratulations! VIPL’s paper on Text-to-Image Diffusion Models and privacy protection, "Anonymization Prompt Learning for Facial Privacy-Preserving Text-to-Image Generation" (authors: Liang Shi, Jie Zhang, Shiguang Shan), was accepted by International Journal of Computer Vision (IJCV). IJCV is a CCF-ranked-A top-tier Computer Vision journal.
Recent text-to-image diffusion models can generate high-resolution, high-fidelity images from textual descriptions and have been widely deployed in public image generation services. However, their ability to realistically synthesize human faces also raises privacy and security concerns, particularly the risk of generating highly convincing fake facial images that may lead to privacy violations and misinformation. To address this problem from a proactive defense perspective, we propose Anonymization Prompt Learning (APL), a prompt-learning-based method for preventing identity-specific face synthesis.
Specifically, we introduce a learnable prompt prefix, referred to as the Anonymization Prompt (AP), into the text input of text-to-image diffusion models. With the diffusion model parameters kept frozen, APL optimizes only the anonymization prompt so that the model generates identity-mismatched facial images when given prompts containing specific person identities, while preserving alignment with corresponding facial attribute descriptions. Furthermore, a regularization mechanism ensures that the anonymization prompt remains ineffective for prompts without identity-specific information, thereby preserving the model’s original image generation capability.
Extensive quantitative and qualitative experiments demonstrate that APL effectively reduces identity consistency between generated images and identity-specific prompts, while generalizing to unseen identities. Meanwhile, image quality and text alignment remain largely unchanged for prompts without identity references. In addition, we show the plug-and-play property of the anonymization prompt: a prompt prefix trained on one diffusion model can be directly transferred to other pretrained text-to-image models while maintaining strong anonymization performance, providing a lightweight and deployable solution for cross-model privacy and security protection.
Download: