Concept Algebra for (Score-Based) Text-Controlled Generative Models
Authors: Zihao Wang, Lin Gui, Jeffrey Negrea, Victor Veitch
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the idea with examples using Stable Diffusion. ... We now work through some examples testing if this subspace structure does indeed exist, and whether it enables manipulation of concepts. ... We compare this with directly prompting the model, and with concept composition [e.g., Du+21; Liu+21]. ... Raters consistently favored images produced by concept algebra, as highlighted in table 1. |
| Researcher Affiliation | Collaboration | Zihao Wang1, Lin Gui1, Jeffrey Negrea2, and Victor Veitch1,2,3 1Department of Statistics, University of Chicago 2Data Science Institute, University of Chicago 3Google Research |
| Pseudocode | Yes | Algorithm 1 Concept Manipulation through Projection ... Algorithm 2 Find Subspace Basis ... Algorithm 3 Find Subspace Mask |
| Open Source Code | No | The paper does not provide a direct link or explicit statement about the open-source release of their own code for the described methodology. |
| Open Datasets | No | The paper refers to 'training data' in a general sense (e.g., 'training data for large-scale generative models') and mentions fine-tuning a diffusion model, but it does not specify any publicly available datasets used for their experiments with concrete access information (e.g., links, DOIs, or formal citations). |
| Dataset Splits | No | The paper does not provide specific details on training, validation, and test dataset splits, such as percentages or sample counts. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used to run the experiments (e.g., GPU models, CPU types, or memory specifications). |
| Software Dependencies | No | The paper mentions using Stable Diffusion and Dreambooth, and references external software in [Liu+22], but does not provide specific version numbers for any software dependencies. |
| Experiment Setup | Yes | First, we fine-tune the diffusion model using Dreambooth, applying a learning rate of 5e 6 and setting the number of steps to 800. ... During sampling, the original prompt is utilized in the first 20% of timesteps to better retain the content. ... To ensure image fidelity, we exclusively employ the score function sdreambooth( a sks toy ) for guiding the denoising process for the last 6% of the denoising steps. |