MotionCraft: Physics-Based Zero-Shot Video Generation
Authors: Antonio Montanaro, Luca Savant Aira, Emanuele Aiello, Diego Valsesia, Enrico Magli
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We present quantitative and qualitative experimental results where we show that our zero-shot MOTIONCRAFT is capable of synthesising realistic videos with finely controlled temporal evolution governed by fluid-dynamics equations, rigid body physics, and multi-agent interaction models, while zero-shot state-of-art techniques cannot. |
| Researcher Affiliation | Academia | Politecnico di Torino |
| Pseudocode | Yes | Algorithm 1: Pseudocode of MOTIONCRAFT |
| Open Source Code | Yes | Project page: https://mezzelfo.github.io/MotionCraft/. |
| Open Datasets | Yes | We conducted a quantitative experiment using the MSU Video Frame Interpolation Benchmark dataset [12], considering only real videos. |
| Dataset Splits | No | The paper does not explicitly mention using a 'validation set' or 'validation split' for its experiments. |
| Hardware Specification | Yes | All our experiments are done on a single NVIDIA A6000 (48GB); |
| Software Dependencies | No | The paper mentions 'runwayml/stable-diffusion-v1-5' as the model and 'OpenCV', 'Φ-flow', and 'agentpy' libraries, but only the Stable Diffusion model is specified with a version number (v1-5). Other key software components lack specific version numbers. |
| Experiment Setup | Yes | We set τ = 400, the number of inference steps (both for DDIM inversion and for inverse diffusion) is set to 200 and the used model is runwayml/stable-diffusion-v1-5 (license Creative ML Open RAIL-M). |