Joint Demosaicing and Denoising for Spike Camera
Authors: Yanchen Dong, Ruiqin Xiong, Jing Zhao, Jian Zhang, Xiaopeng Fan, Shuyuan Zhu, Tiejun Huang
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | For training and evaluation, we designed a spike camera simulator to generate Bayer-pattern spike streams with synthesized noise. Besides, we captured some Bayer-pattern spike streams, building the first realworld captured dataset to our knowledge. Experimental results show that our method can restore clean images from Bayer-pattern spike streams. The source codes and dataset are available at https://github.com/csycdong/SJDD-Net. |
| Researcher Affiliation | Academia | 1 National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University 2 National Computer Network Emergency Response Technical Team 3 School of Electronic and Computer Engineering, Peking University 4 School of Computer Science and Technology, Harbin Institute of Technology 5 School of Information and Communication Engineering, University of Electronic Science and Technology of China |
| Pseudocode | No | The paper describes the proposed method and its modules in detail using text and diagrams (Figure 1, Figure 2, Figure 3), but it does not include any formal pseudocode or algorithm blocks. |
| Open Source Code | Yes | The source codes and dataset are available at https://github.com/csycdong/SJDD-Net. |
| Open Datasets | Yes | For training and evaluation, we designed a spike camera simulator to generate Bayer-pattern spike streams with synthesized noise... With the simulator, we use videos from REDS (Nah et al. 2019) and DAVIS (Pont-Tuset et al. 2017) to generate data. As a result, we generated 1,950 spike sequences for training. To further evaluate the blind JDD performance of our method, we build a Bayerpattern spike stream (BSS) dataset captured by a spike camera with CFA, which is the first real-world dataset to our knowledge. |
| Dataset Splits | No | The paper states: "We also generated a DAVIS-based dataset (DSPK) with 120 sequences and a REDS-based dataset (RSPK) with 60 sequences for evaluation." and "We randomly crop the spike frames into 64 64 patches and set the batch size to 8." It does not explicitly specify train/validation/test splits by percentage or count for the training dataset beyond generating 1,950 sequences for training and separate datasets for evaluation. It uses a simulator and also captures real-world data, but explicit split details like 80/10/10 are not present. |
| Hardware Specification | Yes | All the models are trained using one NVIDIA GTX 1080Ti GPU. |
| Software Dependencies | No | The paper mentions "Adam optimizer (Kingma and Ba 2014)" but does not specify software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | In our implementation, the stage number K of iterations is set to 3... The number of input spike frames N is set to 39. We randomly crop the spike frames into 64 64 patches and set the batch size to 8. For data augmentation, we randomly perform horizontal and vertical flips on the input frames. During training, our models are optimized using Adam optimizer (Kingma and Ba 2014) with a learning rate initially set as 10 4. The learning rate is scaled by 0.8 every 50 epochs. Besides, we use L2 loss for training. |