Implicit Regularization with Polynomial Growth in Deep Tensor Factorization
Authors: Kais Hariz, Hachem Kadri, Stephane Ayache, Maher Moakher, Thierry Artieres
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Using numerical experiments, we demonstrate the benefits of this implicit regularization in yielding a more accurate estimation and better convergence properties. |
| Researcher Affiliation | Academia | 1Aix Marseille University, CNRS, LIS, Marseille, France 2LAMSIN, National Engineering School of Tunis, University of Tunis El Manar, Tunis, Tunisia 3Ecole Centrale de Marseille, Marseille, France. |
| Pseudocode | No | The paper does not include any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any explicit statements about releasing source code for the described methodology, nor does it include a link to a code repository. |
| Open Datasets | Yes | We used Meteo-UK2 and CCDS3 data sets (Lozano et al., 2009)... 2https://www.metoffice.gov.uk/ research/climate/maps-and-data/ historic-station-data. 3https://viterbi-web.usc.edu/~liu32/data.html. |
| Dataset Splits | Yes | Table 1 reports for the depth ranging from 0 to 5, and for a percentage of unobserved values ranging from 75% to 90%, the smallest loss obtained on validation data whatever the initialization, and the rank of the corresponding learned tensor (in brackets). |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | No | The paper does not list specific software dependencies with version numbers (e.g., Python 3.x, PyTorch 1.x) that would be needed to reproduce the experiments. |
| Experiment Setup | Yes | In all the experiments that we report here the percentage of observed and unobserved inputs in the tensor are 20% and 80%. ... With our deep formulation, the product of multiple small norm matrices may lead to numerical instabilities and/or to the well known vanishing gradient. However, we observed implicit regularization with highest values in matrix initialization. ... we use in our simulations zero mean and near zero standard deviation for initializing the parameters to be learned. |