How Learning by Reconstruction Produces Uninformative Features For Perception
Authors: Randall Balestriero, Yann Lecun
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | For example, the supervised Tiny Imagenet task with images projected onto the top subspace explaining 90% of the pixel variance can be solved with 45% test accuracy. Using the bottom subspace instead, accounting for only 20% of the pixel variance, reaches 55% test accuracy. |
| Researcher Affiliation | Academia | Randall Balestriero 1 Yann Le Cun 2 1Brown University 2NYU. Correspondence to: Randall Balestriero <rbalestr@brown.edu>. |
| Pseudocode | No | The paper includes Python code snippets for `fast_gram_eigh` and `alignment_sweep` in sections 5.7 and 5.8, but these are implementation details for specific computations rather than a general pseudocode or algorithm block for the overall research methodology. |
| Open Source Code | Yes | Codebase provided at github.com/Randall Balestriero/Learning By Reconstruction. |
| Open Datasets | Yes | For example, the supervised Tiny Imagenet task with images projected onto the top subspace explaining 90% of the pixel variance can be solved with 45% test accuracy. |
| Dataset Splits | No | Figure 1. Features for reconstruction are uninformative for perception (top): Tiny Imagenet Res Net9 top-1 accuracy when trained and validated on images projected on the top-subspace (red) or bottom subspace (blue) of explained variance... |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU or CPU models, or cloud computing instance types used for running the experiments. |
| Software Dependencies | No | The paper includes Python code snippets, but it does not specify version numbers for Python itself or any other software libraries or dependencies used in the experiments (e.g., PyTorch, TensorFlow, scikit-learn). |
| Experiment Setup | No | Training was done with cross-entropy loss, Fig. 8 provides the MSE case with one-hot labels as target showcasing the same trends. |