On gauge freedom, conservativity and intrinsic dimensionality estimation in diffusion models
Authors: Christian Horvat, Jean-Pascal Pfister
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In section 5, we present a method for estimating the intrinsic dimensionality of the data manifold by analyzing the local variability when approaching the data manifold. Additionally, we provide empirical evidence that with a nonconservative vector field, the ID is not estimated correctly while using the derived method we can estimate the ID correctly if sθ is constrained to be conservative (not necessarily matching s exactly). |
| Researcher Affiliation | Academia | Christian Horvat Department of Physiology University of Bern christian.horvat@unibe.ch Jean-Pascal Pfister Department of Physiology Bern, Switzerland jeanpascal.pfister@unibe.ch |
| Pseudocode | No | No explicit pseudocode or algorithm blocks were found in the paper. |
| Open Source Code | No | The paper does not contain an explicit statement about the release of source code or a link to a code repository. |
| Open Datasets | No | The paper uses synthetic data (2-dimensional Gaussian, spheres, tori, swiss rolls) for its experiments, but does not provide access information for a publicly available or open dataset. |
| Dataset Splits | No | The paper does not specify dataset splits (e.g., train, validation, test percentages or counts) or reference predefined splits for reproducibility. |
| Hardware Specification | No | No specific hardware details (e.g., GPU/CPU models, memory, or cloud resources) used for running experiments are mentioned in the paper. |
| Software Dependencies | No | No specific software dependencies with version numbers (e.g., library names like PyTorch, TensorFlow, or specific solvers with versions) are mentioned in the paper. |
| Experiment Setup | Yes | We train a conservative and non-conservative diffusion model using f(x, t) = 0 and g(t) = 25t as drift and diffusion coefficient, respectively. The non-conservative diffusion model is simply an unconstrained neural network sθ(x, t) = ψθ where ψθ : R5 R>0 R5. The conservative version is sθ(x, t) = ||ψθ(x, t)||2 2 as suggested by Du et al. (2023). For the non-conservative diffusion model, we simply use a standard feed-forward neural network where we first embed the data into 100 dimensions and linearly transform it followed by a nonlinearity (first step). Further, we embed the resulting features into 200 dimensions, again linearly transform it followed by a non-linearity, and finally project back into the data dimensions (second step). We embed the time into 100 dimensions using a Gaussian-Fourier projection and add these embeddings to the features after the first step. |