← ML Research Wiki / 2301.13188

Extracting Training Data from Diffusion Models Equal contribution + Equal contribution † Equal contribution

Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramèr, Borja Balle, Daphne Ippolito, Eric Wallace, Google, Deepmind, Princeton, U C Berkeley (2023)

Paper Information
arXiv ID
Venue
USENIX Security Symposium
Domain
Artificial Intelligence, Machine Learning
SOTA Claim
Yes
Reproducibility
7/10

Abstract

Image diffusion models such as DALL-E 2, Imagen, and Stable Diffusion have attracted significant attention due to their ability to generate high-quality synthetic images. In this work, we show that diffusion models memorize individual images from their training data and emit them at generation time. With a generate-and-filter pipeline, we extract over a thousand training examples from stateof-the-art models, ranging from photographs of individual people to trademarked company logos. We also train hundreds of diffusion models in various settings to analyze how different modeling and data decisions affect privacy. Overall, our results show that diffusion models are much less private than prior generative models such as GANs, and that mitigating these vulnerabilities may require new advances in privacy-preserving training.

Summary

This paper investigates the privacy implications of image diffusion models like DALL-E 2, Imagen, and Stable Diffusion, demonstrating that these models memorize individual training images and can regenerate them. The authors extract over a thousand examples from these models using a generate-and-filter pipeline, showing that diffusion models are less private than previous generative models like GANs. They propose new definitions of memorization and conduct various attacks to highlight the privacy risks associated with diffusion models. The study emphasizes that harmful data extraction is feasible with current models, and existing privacy-enhancing techniques don't adequately mitigate these risks. Their findings raise ethical concerns regarding the use of generative models in privacy-sensitive contexts and call for responsible deployment and new privacy-preserving methods.

Methods

This paper employs the following methods:

  • generate-and-filter pipeline
  • membership inference
  • data extraction attack
  • inpainting attack

Models Used

  • DALL-E 2
  • Imagen
  • Stable Diffusion

Datasets

The following datasets were used in this research:

  • CIFAR-10
  • LAION

Evaluation Metrics

  • F1-score
  • FID

Results

  • Extracted over a thousand training examples from diffusion models
  • Diffusion models leak more training data than GANs
  • Existing privacy techniques were found inadequate
  • The study suggests that better models lead to more privacy leakage.

Limitations

The authors identified the following limitations:

  • Existing privacy-preserving techniques do not provide an acceptable tradeoff between privacy and utility.

Technical Requirements

  • Number of GPUs: None specified
  • GPU Type: None specified

Keywords

diffusion models privacy memorization data extraction generative models

Papers Using Similar Methods

External Resources