Learning Truncated Causal History Model for Video Restoration
Authors: Amirhosein Ghasemabadi, Muhammad Janjua, Mohammad Salameh, Di Niu
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We report new state-of-the-art results on a multitude of video restoration benchmark tasks, including video desnowing, nighttime video deraining, video raindrops and rain streak removal, video super-resolution, real-world and synthetic video deblurring, and blind video denoising while reducing the computational cost compared to existing best contextual methods on all these tasks. |
| Researcher Affiliation | Collaboration | Amirhosein Ghasemabadi ECE Department, University of Alberta ghasemab@ualberta.ca Muhammad Kamran Janjua Huawei Technologies, Canada kamran.janjua@huawei.com Mohammad Salameh Huawei Technologies, Canada mohammad.salameh@huawei.com Di Niu ECE Department, University of Alberta dniu@ualberta.ca |
| Pseudocode | No | The paper describes the methodology using text and architectural diagrams (Figure 1, Figure 2) but does not include explicit pseudocode or algorithm blocks. |
| Open Source Code | Yes | https://kjanjua26.github.io/turtle/ and from NeurIPS checklist: "Yes] Justification: We provide a readme file, along with the code, that lists the steps to run the experiments." |
| Open Datasets | Yes | All of the experiments presented in this manuscript employ publicly available datasets that are disseminated for the purpose of scientific research on video/image restoration. All of the datasets employed are cited wherever they are referred to in the manuscript, and we summarize the details here. Video Desnowing: We utilize the video desnowing dataset introduced in [10]. The dataset is made available by the authors at the link: Video Desnowing |
| Dataset Splits | Yes | We follow the train/test protocol outlined in [47, 35], and train TURTLE on 10 videos from scratch, and evaluate on a held-out test set of 20 videos. In total, the dataset includes 110 videos, of which 80 are used for training, while 30 are held-out test set to measure desnowing performance. We follow the proposed train/test split in the original work [10] and train TURTLE on the video desnowing dataset. |
| Hardware Specification | Yes | All of our models are implemented in the PyTorch library, and are trained on 8 NVIDIA Tesla v100 PCIe 32 GB GPUs for 250k iterations. |
| Software Dependencies | No | The paper mentions implementation in 'PyTorch library' but does not specify a version number for PyTorch or any other software dependencies. |
| Experiment Setup | Yes | We follow the standard training setting of architectures in the restoration literature [29, 79, 15] with Adam optimizer [27] (β1 = 0.9, β2 = 0.999). The initial learning rate is set to 4e-4, and is decayed to 1e-7 throughout training following the cosine annealing strategy [40]. All of our models are implemented in the Py Torch library, and are trained on 8 NVIDIA Tesla v100 PCIe 32 GB GPUs for 250k iterations. Each training video is sampled into clips of γ = 5 frames, and TURTLE restores frames of each clip with recurrence. The training videos are cropped to 192 × 192 sized patches at random locations, maintaining temporal consistency, while the evaluation is done on the full frames during inference. We assume no prior knowledge of the degradation process for all the tasks. Further, we apply basic data augmentation techniques, including horizontal-vertical flips and 90-degree rotations. |