STREAM: Spatio-TempoRal Evaluation and Analysis Metric for Video Generative Models
Authors: Pum Jun Kim, Seojun Kim, Jaejun Yoo
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We provide analytical and experimental evidence demonstrating that STREAM provides an effective evaluation tool for both visual and temporal quality of videos, offering insights into area of improvement for video generative models. To the best of our knowledge, STREAM is the first evaluation metric that can separately assess the temporal and spatial aspects of videos. Our code is available at STREAM. |
| Researcher Affiliation | Academia | Pum Jun Kim, Seojun Kim, Jaejun Yoo Ulsan National Institute of Science and Technology {pumjun.kim,seojun.kim,jaejun.yoo}@unist.ac.kr |
| Pseudocode | Yes | Algorithm 1 STREAM-T |
| Open Source Code | No | Our code is available at STREAM. The paper mentions 'Our code is available at STREAM' but does not provide a direct link to a repository or explicitly state its inclusion in supplementary materials. 'STREAM' is the name of the proposed metric, not a URL or access point. |
| Open Datasets | Yes | We use the synthetic CATER dataset (Girdhar & Ramanan, 2020) in our experiments. [...] using real dataset: Kinetics-600 (Carreira et al., 2018) and UCF-101 (Soomro et al., 2012) datasets |
| Dataset Splits | No | The paper mentions datasets used for training (e.g., 'All the models are trained on the UCF-101 dataset') but does not explicitly provide specific percentages or counts for training, validation, or test splits, nor does it cite predefined splits with detailed specifications for reproducibility. |
| Hardware Specification | No | The paper does not provide any specific hardware specifications (e.g., GPU models, CPU types, memory amounts) used for running the experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow versions or library versions). |
| Experiment Setup | Yes | In all experiments, we consider a total of 2,048 real and fake data. The results for all metrics are the average of five repeated measurements. We set the hyperparameter for k-NN to k = 5. We have configured the histogram s bin size to 50. |