Conditional Generative Neural Decoding with Structured CNN Feature Prediction
Authors: Changde Du, Changying Du, Lijie Huang, Huiguang He2629-2636
AAAI 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results show that our approach yields state-of-the-art visual reconstructions from brain activities. |
| Researcher Affiliation | Collaboration | Changde Du,1,2,3 Changying Du,4 Lijie Huang,1 Huiguang He1,2,5, 1Research Center for Brain-Inspired Intelligence & NLPR, CASIA, Beijing 100190, China 2University of Chinese Academy of Sciences, Beijing 100049, China 3Huawei Cloud BU EI Innovation Lab, China 4Huawei Noah s Ark Lab, Beijing 100085, China 5Center for Excellence in Brain Science and Intelligence Technology, CAS, Beijing, China |
| Pseudocode | No | The paper describes the optimization process in narrative text and mathematical equations, but does not include a structured pseudocode or algorithm block. |
| Open Source Code | No | The paper states 'Our f MRI data will be shared online.' but does not make an explicit statement about the release of their source code for the methodology described. |
| Open Datasets | Yes | 1) Vim-1: a publicly available f MRI dataset, which contains the blood-oxygen-level dependent (BOLD) responses of two subjects when they are presented with grayscale natural images (Kay et al. 2008). ... For our ICG model, we use the gray scale Image Net-1k (Deng et al. 2009) and Celeb A (Liu et al. 2015) datasets to augment the training sets of Vim-1 and Face Bold, respectively. |
| Dataset Splits | No | For Vim-1: 'The dataset is partitioned into distinct training and test sets which consist of 1750 and 120 instances, respectively.' For Face Bold: 'In total, 720 faces were presented once for the training set, and 80 faces were presented twice for the test set.' A specific, distinct validation set is not mentioned for the overall dataset split, though five-fold cross-validation is used for hyperparameter selection of λ1. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU/CPU models, memory) used to run the experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers (e.g., library names like PyTorch or TensorFlow with their respective versions). |
| Experiment Setup | Yes | For SMR model, we experiment with its two variants. One without the sparsity assumptions on the inverse covariance matrices, and the other with the sparsity assumptions on the inverse covariance matrices. We fix the hyperparameter λ as 0.001 for both cases. For non-sparse case, we fix λ2 = λ3 = 10 6, and λ1 is selected using five-fold cross-validation within the range [10 5, 103]. For sparse case, we use the same value of λ1 that was selected for non-sparse case, and only λ2 and λ3 are selected by crossvalidation. For ICG model, we treat the top 5000 decodable CNN units (according to the rank of each unit s decodability) as condition, and set {α = 0.5, β = 1} to combine the advantages of both CVAE and CGAN. We set {α = 0, β = 1} and {α = 1, β = 0} in ICG to implement CVAE and CGAN, respectively. The latent variable z is randomly drawn from a N(0, I) distribution, with the dimension set to 512 and 256 on the Vim-1 and Face Bold datasets, respectively. |