Towards Safe Concept Transfer of Multi-Modal Diffusion via Causal Representation Editing
Authors: Peiran Dong, Bingjie WANG, Song Guo, Junxiao Wang, Jie ZHANG, Zicong Hong
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we empirically evaluate the effectiveness of our proposed Causal Representation Editing (CRE). |
| Researcher Affiliation | Academia | 1Hong Kong Polytechnic University 2Hong Kong University of Science and Technology 3Guangzhou University 4King Abdullah University of Science and Technology |
| Pseudocode | Yes | Pseudocode of Algorithm 1; Pseudocode of Algorithm 2 |
| Open Source Code | No | We will open source all datasets, pre-training parameters and code files in the camera-ready version. |
| Open Datasets | Yes | We select one class from the Image Net dataset as an unsafe concept and generate 500 images using the prompt an image of a [class name] with Stable Diffusion 2.1 [36]. ... We create our dataset and train a Res Net-50 classifier and a Vi T-base classifier based on the dreambench dataset [2] for unsafe style transfer. |
| Dataset Splits | No | The paper mentions evaluating performance on generated images and training classifiers, but it does not specify explicit train/validation/test splits for the main concept transfer experiments. |
| Hardware Specification | Yes | We conduct all experiments on an RTX 3090 and an A100-80G. |
| Software Dependencies | No | The paper mentions using Kosmos-G and Stable Diffusion 2.1, but does not provide specific version numbers for any software libraries or dependencies, such as Python or PyTorch versions. |
| Experiment Setup | Yes | For determining the causal period, we set δk to 0 for all types of unsafe concepts. We set the guidance scale to 9.0. ... We set the guidance scale to 7.5. ... We employ stochastic gradient descent with an initial learning rate of 0.001 and momentum of 0.9. The training process lasts for 50 epochs. |