Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Finding Shared Decodable Concepts and their Negations in the Brain
Authors: Cory Efird, Alex Murphy, Joel Zylberberg, Alona Fyshe
ICLR 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We train a highly accurate contrastive model that maps brain responses during naturalistic image viewing to CLIP embeddings. We then use a novel adaptation of the DBSCAN clustering algorithm to cluster the parameters of these participant-specific contrastive models. This reveals what we call Shared Decodable Concepts (SDCs)... We evaluate our models using top-k accuracy. Figure 12 shows the results of this evaluation. The contrastive decoder outperforms ridge regression across all values of k. |
| Researcher Affiliation | Academia | Cory Efird Computing Science University of Alberta Alberta Machine Intelligence Institute (Amii) EMAIL; Alex Murphy Computing Science University of Alberta Alberta Machine Intelligence Institute (Amii) EMAIL; Joel Zylberberg Physics and Astronomy York University EMAIL; Alona Fyshe Computing Science and Psychology University of Alberta Alberta Machine Intelligence Institute (Amii) EMAIL |
| Pseudocode | Yes | To discover shared brain-decodable concepts we apply a novel clustering method to the per-voxel model parameter vectors w(k) i across all participants. We base our clustering method on the Density Based Spatial Clustering of Applications with Noise (DBSCAN) (Ester et al., 1996) algorithm, which is parameterized by a neighborhood size ε and a point threshold min Pts. The algorithm can be summarized using the following steps: (1) Points that have at least min Pts points in their ε-neighborhood are marked as core points; (2) construct a graph G where the vertices are core points, and there are edges between core points that have a distance less than ε; (3) clusters are formed by finding the connected components of G and (4) the remaining non-core points are added to clusters if they are within the ε-neighborhood of a core point, otherwise, they are marked as outliers not belonging to any cluster. In our modified cross-participant DBSCAN, we redefine the core point threshold min Pts as min Neighbors. Then, in step (1) a point w(k) i is a core point if there are points from at least min Neighbors other participants within its ε neighborhood. ... Our second modification is to apply a within-participant expansion of clusters. We define a new hyperparameter εexpansion and apply a final expansion step: (5) All points inside the εexpansion neigh-borhood of a cluster become members of that cluster with the constraint that they must be from the same participant. |
| Open Source Code | Yes | Code to reproduce this work can be found at the following URL: https://github.com/ fyshelab/contrastive-decoding-iclr2025 |
| Open Datasets | Yes | The natural scenes dataset (NSD) is a massive f MRI dataset acquired to study the underpinnings of natural human vision (Allen et al., 2022). Eight participants were presented with 30,000 images (10,000 unique images over 3 repetitions) from the Common Objects in Context (COCO) naturalistic image dataset (Lin et al., 2014). |
| Dataset Splits | Yes | We split the per-image brain responses X and CLIP embeddings Y into training (XTrain, YTrain), validation (XVal, YVal), and test (XTest, YTest) folds for each of the 8 NSD participants. For each participant, the validation and test folds were chosen to have exactly 1,000 images with three presentations. Of the shared 1,000 images that all participants saw, 413 images were shown three times to every participant across the sessions released by NSD. These 413 images appear in each participant s testing fold. |
| Hardware Specification | Yes | The decoder models were trained on an NVIDIA Ge Force RTX 2060. |
| Software Dependencies | No | The paper mentions using 'GLMSingle toolbox (Prince et al., 2022)', 'CLIP (Radford et al., 2021)', and the 'wordcloud python package', but does not specify version numbers for any of these software components. |
| Experiment Setup | Yes | The decoding model is trained for 5000 iterations (29 to 45 epochs depending on the participant s training set size) with the Adam optimizer, a batch size of 128, and a fixed learning rate of 1e 4. Data augmentation is applied to help slow overfitting by adding random noise to training samples xi xi + z sampled from a normal distribution z N(0, σ2) where the noise standard deviation σ is a hyper-parameter. We set τ = 0.03 and σ = 0.1 in our implementation. Hyper-parameters were selected based on performance on the validation set. |