Separating common from salient patterns with Contrastive Representation Learning
Authors: Robin Louiset, Edouard Duchesnay, Antoine Grigis, Pietro Gori
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We validate our method, called Sep CLR, on three visual datasets and three medical datasets, specifically conceived to assess the pattern separation capability in Contrastive Analysis. We compute (Balanced) Accuracy scores (=(B-)ACC), or Area-under Curve scores (=AUC) for categorical variables, Mean Average Error (=MAE) for continuous variables, and the sum of the differences (δtot) between the obtained results and the expected ones. Hyper-parameters. We empirically choose τ = 0.5 for all experiments and losses. The other hyper-parameters are λC and λS, which weigh the common terms and salient terms, respectively, and λ, which weighs the independence regularization. |
| Researcher Affiliation | Academia | 1 Neuro Spin, University Paris Saclay, France. 2 LTCI, T el ecom Paris, IPParis, France |
| Pseudocode | No | The paper describes its methods using text and mathematical formulations but does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code available at https: //github.com/neurospin-projects/2024_rlouiset_sep_clr |
| Open Datasets | Yes | We validate our method, called Sep CLR, on three visual datasets and three medical datasets... Digits superimposed on natural backgrounds. In this experiment particularly suited to CA and inspired from Zou et al. (2013), we consider CIFAR-10 images as the background dataset (y = 0) and CIFAR-10 images with an overlaid digit as the target dataset (y = 1). Celeb A accessories dataset. We consider a subset of Celeb A Liu et al.. Neuroimaging: parsing schizophrenia s heterogeneity. ... As in Louiset et al. (2021; 2023), we gather T1w VBM Ashburner (2000) warped MRIs (1283 voxels)... Chest and eye pathologies subtyping. We propose two experiments using subsets of Che Xpert Irvin (2019) and ODIR dataset (Ocular Disease Intelligent Recognition dataset) 7... 7https://www.kaggle.com/datasets/andrewmvd/ocular-disease-recognition-odir5k |
| Dataset Splits | Yes | In practice, we used a train set of 50000 images (25000 Cifar-10 images, 25000 Cifar-10 images with random MNIST digits overlaid) and an independent test set of 1000 images (500, 500). We used a train set of 20000 images, (10000 no accessories, 5000 glasses, 5000 hats) and an independent test set of 4000 images (2000 no accessories, 1000 glasses, 1000 hats). Experiments were run 5 times with a different train/val split (respectively 75% and 25% of the dataset) to account for initialization and data uncertainty. Train dataset contains 1890 healthy images, 363 diabetes images, 278 glaucoma images, 281 cataract images, 242 age-related macular degeneration images, and 227 pathological myopia images. On the other hand, TEST dataset contains respectively 210 healthy, 37 diabetes, 26 glaucoma, 39 cataract, 23 macular degeneration, 30 myopia images. |
| Hardware Specification | No | The paper refers to general computing resources (e.g., 'on a cluster') and mentions GPUs in the context of cited works, but it does not provide specific details such as GPU models, CPU types, or memory specifications for its own experimental setup. |
| Software Dependencies | No | The paper mentions using specific neural network architectures (e.g., ResNet-18) and optimizers (Adam), but it does not specify versions for any core software components or libraries (e.g., Python, PyTorch, TensorFlow). |
| Experiment Setup | Yes | Hyper-parameters. We empirically choose τ = 0.5 for all experiments and losses. The other hyper-parameters are λC and λS, which weigh the common terms and salient terms, respectively, and λ, which weighs the independence regularization. The learning rate was set to 0.001 with an Adam optimizer. The models were trained during 250 epochs with batch size equal to 512. As for the Sep CLR s hyper-parameters, we chose λC = 1, λS = β = 1000, and λ = 10. |