Function Contrastive Learning of Transferable Meta-Representations
Authors: Muhammad Waleed Gondal, Shruti Joshi, Nasim Rahaman, Stefan Bauer, Manuel Wuthrich, Bernhard Schölkopf
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments on a number of synthetic and real-world datasets show that the representations we obtain outperform strong baselines in terms of downstream performance and noise robustness, even when these baselines are trained in an end-to-end manner. |
| Researcher Affiliation | Academia | 1Max Planck Institute for Intelligent Systems, T ubingen, Germany 2Mila, University of Montreal, Montreal, Canada 3CIFAR Azrieli Global Scholar. |
| Pseudocode | No | The paper describes the method using mathematical equations and textual explanations, but no structured pseudocode or algorithm blocks are present. |
| Open Source Code | No | No explicit statement or direct link to the source code for the proposed FCRL method is provided. A link to 'trifinger_simulation' is in the references, but this appears to be a dataset/simulation used, not the FCRL code itself. |
| Open Datasets | Yes | We consider images of MNIST digits (Le Cun et al., 1998)... For the first task, our goal is to determine whether the representation rk contains enough information to infer the underlying factors of variation (Bengio et al., 2013) of a given scene. To this end, we use MPI3D (Gondal et al., 2019)... |
| Dataset Splits | Yes | We consider a dataset of 20, 000 training, 1000 validation and 1000 test sinusoidal functions. The training and validation datasets consists of 60, 000 MNIST training and 10, 000 test samples, respectively. |
| Hardware Specification | No | No specific hardware details (such as GPU models, CPU models, or cloud computing instances with specifications) are provided for the experimental setup. |
| Software Dependencies | No | No specific software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow versions, or specific library versions) are listed for reproducibility. |
| Experiment Setup | Yes | For sinusoidal functions, we fix the maximum number of context points to 20 and the number of examples N is chosen randomly in [2, 20] for each k. For MNIST digits as 2D functions, we allow a maximum of 200 samples per context set, and N is sampled randomly from [2, 200] for each k. The encoder g(φ,Φ) is then trained by splitting each context set Ok into J disjoint views. We set J = 2 for the sinusoidal functions and J = 10 for the 2D functions. |