From voxels to pixels and back: Self-supervision in natural-image reconstruction from fMRI
Authors: Roman Beliy, Guy Gaziv, Assaf Hoogi, Francesca Strappini, Tal Golan, Michal Irani
NeurIPS 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We achieve competitive results in image reconstruction on both datasets (f MRI on Image Net [16], and vim-1 [1]). |
| Researcher Affiliation | Academia | Roman Beliy Dept. of Computer Science and Applied Math The Weizmann Institute of Science 76100 Rehovot, Israel roman.beliy@weizmann.ac.il Guy Gaziv Dept. of Computer Science and Applied Math The Weizmann Institute of Science 76100 Rehovot, Israel guy.gaziv@weizmann.ac.il Assaf Hoogi Dept. of Computer Science and Applied Math The Weizmann Institute of Science 76100 Rehovot, Israel assaf.hoogi@weizmann.ac.il Francesca Strappini Dept. of Neurobiology The Weizmann Institute of Science 76100 Rehovot, Israel francescastrappini@gmail.com Tal Golan Zuckerman Institute Columbia University 10027 New York, NY, USA tal.golan@columbia.edu Michal Irani Dept. of Computer Science and Applied Math The Weizmann Institute of Science 76100 Rehovot, Israel michal.irani@weizmann.ac.il |
| Pseudocode | No | The paper describes the architecture and training phases using text and diagrams, but does not include structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper provides a 'Project Website' link but does not explicitly state in the text that the source code for the method is available there or elsewhere. |
| Open Datasets | Yes | We experimented with two publicly available (and very different) benchmark f MRI datasets, using the same architectures and hyperparameters: (i) f MRI on Image Net [16], and (ii) vim-1 [1]. These datasets provide f MRI recordings paired with their corresponding underlying images. ... We used additional 50K unlabeled natural images from Image Net [32] validation data for the unsupervised training on unlabeled images (Encoder-Decoder objective, Fig. 1d). |
| Dataset Splits | Yes | f MRI on Image Net comprises 1250 distinct Image Net images drawn from 200 selected categories. The train- and test-f MRI data consist of 1 and 35 (repeated recordings) per presented stimulus, respectively. Fifty image categories provided the fifty test images, one from each category. The remaining 1200 were defined as train set (with only one f MRI recording). ... vim-1 comprises 1870 distinct grayscale images. f MRI was recorded (i) twice for 1750 images and defined the training data, and (ii) 13 times for the remaining 120 images, defining the test data. |
| Hardware Specification | Yes | Our system completes the two-stage training within approximately 15 min using a single Tesla V100 GPU |
| Software Dependencies | No | The paper mentions optimizers (SGD, Adam) and pretrained models (Alex Net, VGG19) but does not provide specific version numbers for any software dependencies. |
| Experiment Setup | Yes | We trained the Encoder with α = 0.9 using SGD optimizer for 80 epochs with an initial learning rate of 0.1, with a predefined learning rate scheduler. During Decoder training with supervised and unsupervised objectives, each training batch consisted of 60% paired data (supervised training), 20% unlabeled natural images (without f MRI), and 20% unlabeled test-f MRI (without images). We trained the Decoder for 150 epochs using Adam optimizer with an initial learning rate of 1e-3, and 80% learning rate drop after every 30 epochs. |