E-Motion: Future Motion Simulation via Event Sequence Diffusion
Authors: Song Wu, Zhiyu Zhu, Junhui Hou, GUANGMING Shi, Jinjian Wu
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Through extensive testing and validation, we demonstrate the effectiveness of our method in various complex scenarios, showcasing its potential to revolutionize motion flow prediction in computer vision applications such as autonomous vehicle guidance, robotic navigation, and interactive media. Our findings suggest a promising direction for future research in enhancing the interpretative power and predictive accuracy of computer vision systems. |
| Researcher Affiliation | Academia | Song Wu 1, Zhiyu Zhu 2, Junhui Hou 2, Guangming Shi 1 , Jinjian Wu 1 1 Xidian University, 2 City University of Hong Kong |
| Pseudocode | Yes | Algorithm 1 Motion Alignment Process |
| Open Source Code | Yes | The source code is publicly available at https://github.com/p4r4mount/E-Motion. |
| Open Datasets | Yes | In our study, we utilize two large-scale event datasets, i.e., Vis Event [55] and Event VOT dataset [56]. |
| Dataset Splits | Yes | The dataset is meticulously annotated and divided into training (841 videos), validation (18 videos), and testing (282 videos) subsets, ensuring a comprehensive framework for robust algorithm testing and benchmarking. |
| Hardware Specification | Yes | All experiments are conducted on machines with 8 Ge Force RTX 3090 GPUs, Intel(R) Core(TM) i7-10700 CPU of 2.90GHz, and 64-GB RAM. |
| Software Dependencies | No | The paper mentions optimizers like ADAM and PPO, but it does not specify software dependencies with version numbers (e.g., Python, PyTorch, CUDA versions). |
| Experiment Setup | Yes | In the pre-training stage, we employed the ADAM optimizer with the exponential decay rates β1 = 0.9 and β2 = 0.999. The total training process was 20000 iterations for both kinds of noise experiments. We initialized the learning rate as 1e-5. We set the batch size to 128 (with 8 gradient accumulation steps). For the alignment process... The updating episode of the reinforcement learning process is set at 100 optimization steps... We also employed the ADAM optimizer with the exponential decay rates β1 = 0.9 and β2 = 0.999. We initialized the learning rate as 2e 6 and set the batch size to 16(with 2 gradient accumulation steps) in all experiments. |