Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Low-rank Tensor Learning with Nonconvex Overlapped Nuclear Norm Regularization
Authors: Quanming Yao, Yaqing Wang, Bo Han, James T. Kwok
JMLR 2022 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, experiments are performed on both synthetic (Section 5.1) and real-world data sets (Sections 5.2-5.5), using a PC with Intel-i9 CPU and 32GB memory. To reduce statistical variation, all results are averaged over five repetitions. |
| Researcher Affiliation | Collaboration | Quanming Yao EMAIL Department of Electronic Engineering, Tsinghua University. Yaqing Wang EMAIL Baidu Research, Baidu Inc. Bo Han EMAIL Department of Computer Science, Hong Kong Baptist University. James T. Kwok EMAIL Department of Computer Science and Engineering, Hong Kong University of Science and Technology. |
| Pseudocode | Yes | Algorithm 1 Computing the pth element vp with index (i1 p . . . i M p ) in ξ(Xt). Algorithm 2 NOnconvex Regularized Tensor (NORT) Algorithm. |
| Open Source Code | No | The paper does not provide explicit links to source code for the NORT algorithm or an affirmative statement about its public release. It mentions using existing implementations for comparison: "We used our own implementations of LRTC, PA-APG and GDPAN as their codes are not publicly available." |
| Open Datasets | Yes | Experiments on a variety of synthetic and real-world data sets (including images, videos, hyper-spectral images, social networks, knowledge graphs and spatial-temporal climate observation records) show that the proposed algorithm is more efficient and has much better empirical performance than other low-rank tensor regularization and decomposition methods. We use the Windows, Tree and Rice images from (Hu et al., 2013). Experiments are performed on three hyper-spectral images (Figure 11): Cabbage (1312 432 49), Scene (1312 951 49) and Female (592 409 148).15: The third dimension is for the bands of images. 15. Cabbage and Scene images are from https://sites.google.com/site/ hyperspectralcolorimaging/dataset, while the Female images are downloaded from http: //www.imageval.com/scene-database-4-faces-3-meters/. Experiments are performed on the You Tube data set16 (Lei et al., 2009), which contains 15,088 users and five types of user interactions. 16. http://leitang.net/data/youtube-data.tar.gz. Experiments are performed on two benchmark data sets: WN18RR17 (Dettmers et al., 2018) and FB15k-23718 (Toutanova et al., 2015). 17. https://github.com/Tim Dettmers/Conv E 18. https://www.microsoft.com/en-us/download/details.aspx?id=52312. Three videos (Eagle19, Friends20 and Logo21) from (Indyk et al., 2019) are used. 19. http://youtu.be/ufnf_q_3Ofg 20. http://youtu.be/xm LZs Ef XEg E 21. http://youtu.be/L5HQo FIa T4I. We use the CCDS and USHCN data sets from (Bahadori et al., 2014). CCDS22 contains monthly observations of 17 variables (such as carbon dioxide and temperature) in 125 stations from January 1990 to December 2001. USHCN23 contains monthly observations of 4 variables (minimum, maximum, average temperature and total precipitation) in 1218 stations from from January 1919 to November 2019. 22. https://viterbi-web.usc.edu/ liu32/data/NA-1990-2002-Monthly.csv 23. http://www.ncdc.noaa.gov/oa/climate/research/ushcn. |
| Dataset Splits | Yes | We use 50% of them for training, and the remaining 50% for validation. Testing is evaluated on the unobserved elements in O. (Section 5.1) We randomly sample 5% of the pixels for training, which are then corrupted by Gaussian noise N(0, 0.012); and another 5% clean pixels are used for validation. The remaining unseen clean pixels are used for testing. (Section 5.2.1) We use 50% of the observations for training, another 25% for validation and the rest for testing. (Section 5.2.3) Following the public splits on entities in E and relations in R (Han et al., 2018), we split the observed triplets in S into a training set Strain, validation set Sval and testing set Stest. (Section 5.3) then randomly sample 10% of the locations for training, another 10% for validation, and the rest for testing. (Section 5.5) |
| Hardware Specification | Yes | In this section, experiments are performed on both synthetic (Section 5.1) and real-world data sets (Sections 5.2-5.5), using a PC with Intel-i9 CPU and 32GB memory. |
| Software Dependencies | No | All algorithms are implemented in Matlab, with sparse tensor and matrix operations performed via Mex files in C. All hypeprparamters (including the λi s in (14) and hyperparameter in the baselines) are tuned by grid search using the validation set. Note that the rank of Xi t+1 in step 11 is determined implicitly by the proximal step. As Xt and Zt are implicitly represented in factorized forms, Vt and Xt (in step 3) do not need to be explicitly constructed. As a result, the resultant time and space complexities are the same as those in Section 3.2.3. Using the coordinate format2 (Bader and Kolda, 2007).2. For a sparse M-order tensor, its pth nonzero element is represented in the coordinate format as (i1 p, . . . , i M p , vp), where i1 p, . . . , i M p are indices on each mode and vp is the value. by using sparse tensor packages such as the Tensor Toolbox (Bader and Kolda, 2007). |
| Experiment Setup | Yes | Recall from Corollary 10 that τ has to be larger than ρ + Dκ0. However, a large τ leads to slow convergence (Remark 11). Hence, we set τ = 1.01(ρ + Dκ0). Moreover, as in (Li et al., 2017), we set γ1 = 0.1 and p = 0.5 in Algorithm 2. All hypeprparamters (including the λi s in (14) and hyperparameter in the baselines) are tuned by grid search using the validation set. We early stop training if the relative change of objective in consecutive iterations is smaller than 10 4 or reaching the maximum of 2000 iterations. For the robust tensor completion, we take RTDGC (Gu et al., 2014) as the baseline, which adopts the ℓ1 loss and overlapped nuclear norm in (40) (i.e., κℓ(x) = x and φ is the nuclear norm). As this is non-smooth and non-differentiable, RTDGC uses ADMM (Boyd et al., 2011) for the optimization, which handles the robust loss and low-rank regularizer separately. As discussed in Section 4.1, we use the smoothing NORT (Algorithm 3, with δ0 = 0.9) to optimize (42), the smoothed version of (40). |