Convergence rate of Bayesian tensor estimator and its minimax optimality
Authors: Taiji Suzuki
ICML 2015 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We investigate the statistical convergence rate of a Bayesian low-rank tensor estimator, and derive the minimax optimal rate for learning a lowrank tensor. Our problem setting is the regression problem where the regression coefficient forms a tensor structure. This problem setting occurs in many practical applications, such as collaborative filtering, multi-task learning, and spatiotemporal data analysis. The convergence rate of the Bayes tensor estimator is analyzed in terms of both in-sample and out-of-sample predictive accuracies. It is shown that a fast learning rate is achieved without any strong convexity of the observation. Moreover, we show that the method has adaptivity to the unknown rank of the true tensor, that is, the near optimal rate depending on the true rank is achieved even if it is not known a priori. Finally, we show the minimax optimal learning rate for the tensor estimation problem, and thus show that the derived bound of the Bayes estimator is tight and actually near minimax optimal. Numerical experiments showed that our theories indeed describe the actual behavior of the Bayes estimator. |
| Researcher Affiliation | Academia | Taiji Suzuki , S-TAIJI@IS.TITECH.AC.JP Tokyo Institute of Technology, O-okayama 2-12-1, Meguro-ku, Tokyo 152-8552, JAPAN PRESTO, Japan Science and Technology Agency, JAPAN |
| Pseudocode | No | The paper does not contain any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain an unambiguous statement that the authors are releasing the source code for the work described, nor does it provide a direct link to a source-code repository. |
| Open Datasets | No | The paper describes a 'tensor completion problem' where 'The true tensor A was randomly generated'. It does not specify the use of a publicly available dataset with concrete access information (link, DOI, formal citation). |
| Dataset Splits | No | The paper refers to 'in-sample predictive accuracy' and 'out-of-sample predictive accuracy' and mentions the number of samples used (n = ns * product of Mk), but it does not provide specific train/validation/test dataset splits (e.g., percentages or exact counts). |
| Hardware Specification | No | The paper does not provide any specific details about the hardware (e.g., GPU models, CPU types, memory) used to run the experiments. |
| Software Dependencies | No | The paper does not mention any specific software components with their version numbers (e.g., programming languages, libraries, frameworks, or solvers). |
| Experiment Setup | Yes | The true tensor A was randomly generated such that each element of U (k) (k = 1, . . . , K) was uniformly distributed on [ 1, 1]. σp was set at 5, and the true tensor was estimated by the posterior mean obtained by the rejection sampling scheme with R = 10. dmax and ξ were set at 10 and 0.5. The posterior sampling was terminated after 500 iterations. The experiments were executed in five different settings, called settings 1 to 5: {(M1, . . . , MK), d } = {(10, 10, 10), 4}, {(10, 10, 40), 5}, {(20, 20, 30), 8}, {(20, 30, 40), 5}, {(30, 30, 40), 6}. For each setting, we repeated the experiments five times and computed the average of the in-sample predictive accuracy and out-of-sample accuracy over all five repetitions. The number of samples was chosen as n = ns k Mk, where ns varied from 0.3 to 0.9. |