A Fixed-Point Approach for Causal Generative Modeling
Authors: Meyer Scetbon, Joel Jennings, Agrin Hilmkil, Cheng Zhang, Chao Ma
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, we conduct an extensive evaluation of each method individually, and show that when combined, our model outperforms various baselines on generated out-of-distribution problems. |
| Researcher Affiliation | Industry | Work done when Joel Jennings and Cheng Zhang were affiliated with Microsoft Research. 1Microsoft Research 2Google Deep Mind. Correspondence to: Meyer Scetbon <tmscetbon@microsoft.com>. |
| Pseudocode | Yes | Algorithm 1 d-TOE(M, (Dtr, Gtr)) |
| Open Source Code | No | The paper does not provide an explicit statement or link to its own open-source code for the described methodology. It only references implementations for baselines and other related works. |
| Open Datasets | Yes | We reproduce the procedure proposed in (Lorch et al., 2022) to generate synthetic datasets and their associated DAGs using randomly sampled SCMs. |
| Dataset Splits | Yes | Given a dataset D Rntot d where ntot is the total number of samples, we split it into three datasets w.r.t the sample size, with ratio 0.8, 0.1, 0.1 for training, validation and testing respectively. |
| Hardware Specification | Yes | We use 4 A100 GPUs with a total of 320 Gi B of memory and 85 CPUs to train our architecture M. |
| Software Dependencies | No | The paper mentions "the Adam implementation of Pytorch (Paszke et al., 2017)" but does not provide specific version numbers for Pytorch or any other critical software dependencies. |
| Experiment Setup | Yes | We consider TANM with an embedding dimension of D = 128, and L = 2 layers. The causal attention mechanism uses 8 heads with an embedding dimension of dhead = 32. We do not hyper-tune the model on each specific instance, but use the same training configuration for all experiments. See appendix E for details. |