Residual Relaxation for Multi-view Representation Learning
Authors: Yifei Wang, Zhengyang Geng, Feng Jiang, Chuming Li, Yisen Wang, Jiansheng Yang, Zhouchen Lin
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on different backbones show that our method can not only improve multi-view methods with existing augmentations, but also benefit from stronger image augmentations like rotation. |
| Researcher Affiliation | Academia | 1 School of Mathematical Sciences, Peking University, China 2 Key Lab. of Machine Perception, School of Artificial Intelligence, Peking University, Beijing, China 3 School of Engineering, The University of Sydney, Australia 4 Institute for Artificial Intelligence, Peking University, Beijing, China 5 Pazhou Lab, Guangzhou, China |
| Pseudocode | No | The paper describes its method, objective formulations, and practical implementations through detailed text and mathematical equations, but it does not include a distinct pseudocode block or a clearly labeled algorithm. |
| Open Source Code | No | The paper provides links to third-party resources such as 'https://github.com/fastai/imagenette' and 'https://github.com/deepmind/deepmind-research/tree/master/byol', which are datasets or codebases used by the authors. However, it does not contain an explicit statement or link indicating that the authors' own source code for the Prelax methodology is openly available. |
| Open Datasets | Yes | Due to computational constraint, we carry out experiments on a range of medium-sized real-world image datasets, including well known benchmarks like CIFAR-10 [15], CIFAR-100 [15], and two Image Net variants: Tiny-Image Net-200 (200 classes with image size resized to 32 32) [27] and Image Nette (10 classes with image size 128 128)3. |
| Dataset Splits | No | The paper mentions training epochs and linear evaluation on datasets like CIFAR-10 and Image Nette, which typically have standard train/test splits. However, it does not explicitly provide specific percentages, sample counts, or clear references for a separate validation split within the main text that would allow for precise reproduction of data partitioning beyond general training and testing. |
| Hardware Specification | No | The paper mentions using ResNet-18 as a backbone network for computational efficiency, which refers to the model architecture. However, it does not provide any specific details about the hardware used to run the experiments, such as GPU models, CPU specifications, or cloud computing instances. |
| Software Dependencies | No | The paper mentions 'using the Py Torch notations' but does not specify any version numbers for PyTorch or any other software libraries or dependencies used in the experiments. Therefore, the software environment cannot be reproducibly set up. |
| Experiment Setup | Yes | For Sim Siam and its Prelax variants, we follow the same hyperparameters in [5] on CIFAR-10. Specifically, we use Res Net-18 as the backbone network, followed by a 3-layer projection MLP, whose hidden and output dimension are both 2048. The predictor is a 2-layer MLP whose hidden layer and output dimension are 512 and 2048 respectively. We use SGD for pre-training with batch size 512, learning rate 0.03, momentum 0.9, weight decay 5 10 4, and cosine decay schedule [18] for 800 epochs. |