Functional Regularization for Representation Learning: A Unified Theoretical Perspective
Authors: Siddhant Garg, Yingyu Liang
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We also provide complementary empirical results to support our analysis. |
| Researcher Affiliation | Collaboration | Siddhant Garg Amazon Alexa AI Search Manhattan Beach, CA, USA sidgarg@amazon.com Yingyu Liang Department of Computer Sciences University of Wisconsin-Madison yliang@cs.wisc.edu |
| Pseudocode | No | The paper does not contain any pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any statements or links indicating the availability of open-source code for the described methodology. |
| Open Datasets | No | For the auto-encoder example, we randomly generate d orthonormal vectors ({u i }i=d i=1) in Rd, means µi and variances σi for i [d] such that σ1> >σr σr+1> >σd. We sample αi N(µi, σi) i [d] and generate x = Pd i=1 αiui. For generating y, we use a randomly generated vector a Rr. ... For the masked self-supervision example, we similarly generate x having the data property specified in Section 5.2 and then follow the other specifications in Section 5.2 (with regards to the hypothesis classes, reconstruction losses, etc.). |
| Dataset Splits | No | We use d = 100 and generate 104 unlabeled, 104 labeled training and 103 labeled test points. |
| Hardware Specification | No | The paper does not provide any specific hardware details used for running the experiments. |
| Software Dependencies | No | The paper does not provide specific software names with version numbers used in the experiments. |
| Experiment Setup | Yes | We use d = 100 and generate 104 unlabeled, 104 labeled training and 103 labeled test points. We use the quadratic activation function and follow the specification in Section 5.1 (with regards to the hypothesis classes, reconstruction losses, etc.). ... We report the MSE on the test data points averaged over 10 runs as the metric. |