SpikeReveal: Unlocking Temporal Sequences from Real Blurry Inputs with Spike Streams
Authors: Kang Chen, Shiyan Chen, Jiyuan Zhang, Baoyue Zhang, Yajing Zheng, Tiejun Huang, Zhaofei Yu
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Quantitative and qualitative experiments conducted on our real-world and synthetic datasets with spikes validate the superior generalization of the proposed framework. |
| Researcher Affiliation | Academia | 1 School of Computer Science, Peking University 2 State Key Laboratory for Multimedia Information Processing, Peking University 3 Institute for Artificial Intelligence, Peking University |
| Pseudocode | No | The paper provides network diagrams but does not include explicit pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code, data and trained models are available at https://github.com/chenkang455/S-SDM. |
| Open Datasets | Yes | We construct the synthetic dataset based on the widely employed GOPRO [17] dataset. ... We build an RGB-Spike binocular system and propose the first spatially-temporally calibrated Real-world Spike Blur (RSB) dataset in this community. Our code, data and trained models are available at https://github.com/chenkang455/S-SDM. |
| Dataset Splits | No | The paper states that the GOPRO dataset is used for training and evaluation under different spike thresholds (Vth = 1, 2, 4), and a new RSB dataset is used for evaluation, but it does not specify explicit train/validation/test splits (e.g., percentages or counts) for reproduction. |
| Hardware Specification | Yes | We use Py Torch to build and train our S-SDM using an NVIDIA Ge Force GTX 4090 GPU and AMD EPYC 7742 64-Core Processor. |
| Software Dependencies | No | The paper mentions using 'Py Torch' but does not specify its version or other software dependencies with version numbers. |
| Experiment Setup | Yes | To augment the dataset and accelerate the training process, we randomly crop 512 512 image from each blurry frame, along with the 128 128 spike stream. ... We complete the training of BSN on the GOPRO dataset, employing an initial learning rate of 3e 4 and spanning 1000 epochs. The training uses the Adam optimizer with a cosine scheduler and sets the batch size to 8 for each epoch. Adopting the same settings as BSN, EDSR is trained on the blur-spike paired data for 70 epochs. Subsequently, LDN undergoes the training of 100 epochs with the learning rate adjusted to 1e 3. |