Motion Graph Unleashed: A Novel Approach to Video Prediction
Authors: Yiqi Zhong, Luming Liang, Bohan Tang, Ilya Zharkov, Ulrich Neumann
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on various datasets, including UCF Sports, KITTI and Cityscapes, highlight the strong representative ability of motion graph. Especially on UCF Sports, our method matches and outperforms the SOTA methods with a significant reduction in model size by 78% and a substantial decrease in GPU memory utilization by 47%. |
| Researcher Affiliation | Collaboration | Yiqi Zhong13 Luming Liang1 Bohan Tang2 Ilya Zharkov1 Ulrich Neumann3 1Microsoft 2University of Oxford 3University of Southern California 1{yiqizhong,lulian,zharkov}@microsoft.com 2bohan.tang@eng.ox.ac.uk 3{yiqizhon,uneumann}@usc.edu |
| Pseudocode | No | The paper contains figures depicting network architectures (e.g., Figure 7, 8, 9) but does not include any explicitly labeled “Pseudocode” or “Algorithm” blocks. |
| Open Source Code | Yes | Please refer to this link for the official code. |
| Open Datasets | Yes | For evaluation, we trained our video prediction pipeline in an end-to-end fashion on three public datasets: i) UCF Sports [36] (...); ii) KITTI [37] (...); and iii) Cityscapes [39] (i.e. 3,475 driving videos with 2,945 in the training set and 500 in the validation set). |
| Dataset Splits | Yes | On the UCF Sports MMVP split, the validation dataset has been divided into three categories: the easy (SSIM 0.9), intermediate (0.6 SSIM < 0.9), and hard subsets (SSIM < 0.6), which take up 66%, 26%, and 8% of the full set respectively. (...) iii) Cityscapes [39] (i.e. 3,475 driving videos with 2,945 in the training set and 500 in the validation set). |
| Hardware Specification | Yes | We implement the video prediction system using Py Torch [54] and conduct end-to-end training on a single NVIDIA A100 GPU. |
| Software Dependencies | No | The paper mentions software like “PyTorch [54]”, “AdamW optimizer [55]”, and “cosine decay scheduler [56]” but does not provide specific version numbers for these software components. |
| Experiment Setup | Yes | In Table 9, we demonstrate the hyper-parameter setting for each dataset. Table 9 includes specific values for Image feat., Tendency feat., Location feat., Number of Graph views, k, Epoch, and Loss for UCF Sports, Cityscapes, and KITTI. |