Self-Supervised Mutual Learning for Dynamic Scene Reconstruction of Spiking Camera
Authors: Shiyan Chen, Chaoteng Duan, Zhaofei Yu, Ruiqin Xiong, Tiejun Huang
IJCAI 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The experimental results demonstrate that our methods evidently outperform previous unsupervised spiking camera reconstruction methods and achieve desirable results compared with supervised methods. We evaluate the performance of our method on both synthetic and real-world datasets. |
| Researcher Affiliation | Academia | 1School of Electronic and Computer Engineering, Peking University 2School of Computer Science, Peking University 3Institute for Artificial Intelligence, Peking University |
| Pseudocode | No | The paper describes the network architecture and modules using descriptive text and diagrams, but it does not contain any pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | Yes | The code is available at https://github.com/hnmizuho/ SSML-Spiking-Camera-Reconstruction. |
| Open Datasets | Yes | For quantitative evaluation, we use the synthetic dataset with ground truth to train our network, which is obtained from Spk Img Net [Zhao et al., 2021]. This synthetic dataset is generated by converting videos from REDS [Nah et al., 2019] to spike stream, with frames in the videos as the ground truth. |
| Dataset Splits | No | The paper specifies the training set (800 pairs) and testing set (40 pairs) but does not explicitly mention or detail a validation set split. |
| Hardware Specification | Yes | ...and train the network with TFP (w = 7) as pseudo labels on Nvidia RTX 2080 GPU for 100k iterations. |
| Software Dependencies | No | The paper mentions using the "Adam optimizer" but does not provide specific version numbers for any software dependencies or frameworks (e.g., Python, PyTorch, TensorFlow). |
| Experiment Setup | Yes | The parameter λ of the loss function is set to 0.01, and the input spike stream is cropped into 40 × 40 patches with a batch size of 4. Besides, we set the short-term temporal window size and long-term temporal window size to 41 and 27, respectively, and remove all additive terms from the convolutional layer as in [Sheth et al., 2021] for better generalization. Moreover, we use Adam optimizer with the default setting to optimize our network, and train the network with TFP (w = 7) as pseudo labels on Nvidia RTX 2080 GPU for 100k iterations. The BSN path is first trained for 15k iterations before mutual learning. |