CoPhy: Counterfactual Learning of Physical Dynamics
Authors: Fabien Baradel, Natalia Neverova, Julien Mille, Greg Mori, Christian Wolf
ICLR 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | This work poses a new problem of counterfactual learning of object mechanics from visual input. We develop the Co Phy benchmark to assess the capacity of the state-of-the-art models for causal physical reasoning in a synthetic 3D environment and propose a model for learning the physical dynamics in a counterfactual setting. Having observed a mechanical experiment that involves, for example, a falling tower of blocks, a set of bouncing balls or colliding objects, we learn to predict how its outcome is affected by an arbitrary intervention on its initial conditions, such as displacing one of the objects in the scene. The alternative future is predicted given the altered past and a latent representation of the confounders learned by the model in an end-to-end fashion with no supervision of confounders. We compare against feedforward video prediction baselines and show how observing alternative experiences allows the network to capture latent physical properties of the environment, which results in significantly more accurate predictions at the level of super human performance. |
| Researcher Affiliation | Collaboration | Fabien Baradel1 Natalia Neverova2 Julien Mille3 Greg Mori4 Christian Wolf1,5 1Universit e Lyon, INSA Lyon, CNRS, LIRIS, Villeurbanne, France 2Facebook AI Research, Paris, France 3Laboratoire d Informatique de l Univ. de Tours, INSA Centre Val de Loire, Blois, France 4Simon Fraser University and Borealis AI, Vancouver, Canada 5Inria, Chroma group, CITI Laboratory, Villeurbanne, France |
| Pseudocode | No | The paper describes the model architecture and training process using text and mathematical equations, but does not include explicit pseudocode or algorithm blocks. |
| Open Source Code | No | The code will be made publicly available. |
| Open Datasets | Yes | a large-scale Co Phy benchmark with three physical scenarios and 300k synthetic experiments including rendered sequences of frames, metadata (object positions, angles, sizes) and values of confounders (masses, frictions, gravity). This benchmark was specifically designed in bias-free fashion to make the counter-factual reasoning task challenging by optimizing the impact of the confounders on the outcome of the experiment. The dataset is publicly available1. ... 1Project page: http://projet.liris.cnrs.fr/cophy |
| Dataset Splits | Yes | The training / validation / test split is defined as 0.7 : 0.2 : 0.1 for each of the three scenarios. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for the experiments, such as GPU or CPU models. It only mentions that models were implemented in PyTorch. |
| Software Dependencies | No | All models were implemented in Py Torch. |
| Experiment Setup | Yes | Training details. All models were implemented in Py Torch. We used the Adam optimizer (Kingma & Ba, 2015) and a learning rate of 0.001. ... f, g are implemented as MLPs with 4 and 2 layers respectively, with hidden layers of size 32 and Re Lu activations. φ, ψ are implemented as GRU modules with 2 layers and a hidden state of dimension 32. |