Geometrically regularized autoencoders for non-Euclidean data
Authors: Cheongjae Jang, Yonghyeon Lee, Yung-Kyun Noh, Frank C. Park
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In the experiments, we first demonstrate the geometric score estimation (Theorem 1) based on our geometrically regularized autoencoders for non-Euclidean data, providing a solid basis for future applications of autoencoders. We then utilize the proposed autoencoders for various applications, such as data sampling based on the Langevin Monte Carlo methods (Girolami & Calderhead, 2011) or clustering and noise filtering based on mode-seeking (Fukunaga & Hostetler, 1975; Cheng, 1995; Comaniciu & Meer, 2002), involving real-world non-Euclidean data sets. We also examine the usefulness of the proposed autoencoders in the representation learning perspective, using noisy point cloud data. |
| Researcher Affiliation | Collaboration | Cheongjae Jang1, Yonghyeon Lee2,3, Yung-Kyun Noh1,3, Frank Chongwoo Park2,4 1Hanyang University, 2Seoul National University, 3Korea Institute for Advanced Study, 4Saige Research cjjang@hanyang.ac.kr, ylee@kias.re.kr, nohyung@hanyang.ac.kr, fcp@snu.ac.kr |
| Pseudocode | Yes | Algorithm 1 DTI Filtering Algorithm Using GDAE |
| Open Source Code | Yes | Codes to train the proposed autoencoders: Supplementary Material. |
| Open Datasets | Yes | We train the GDAE and GRCAE using synthetic data sampled from m mixtures of isotropic tangent space Gaussians for which the ground truth geometric score values are obtainable. (...) We group documents in the Newsgroup20 data set (Lang, 1995) using the GDAE trained on the document embeddings. (...) Data used in these experiments were obtained from the Alzheimer s Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). (...) We train autoencoders using the Shape Net data set (Chang et al., 2015) and obtain representations for the Model Net data set (Wu et al., 2015). |
| Dataset Splits | Yes | The estimation errors measured on 10,000 test data points are averaged over five runs in Table 1. Note that the GDAE, GRCAE, and R-LSLDG methods show superior performance over DAE, RCAE, and LSLDG in estimating log ρg(x) x . (...) For a given data set {x1, . . . , x N} represented in local coordinates of P(n), the log-density gradient estimation error (Est. error) is evaluated as follows: (...) In this experiment, σ is selected among σ {0.01, 0.025, 0.05} to reduce the modified estimation error in (64), which does not require the true value of log ρg(x) x , on a randomly selected validation data set of sizes 20,000. (...) The number of training data is 800, that of validation data is 200, and that of test data is 1000. |
| Hardware Specification | Yes | We have used the Pytorch library (Paszke et al., 2017) and have utilized NVIDIA Tesla V100 GPU with Intel Xeon E5-2698 v4 2.2 GHz (20-Core) CPU (also for most of the other experiments). |
| Software Dependencies | No | The paper mentions using the "Pytorch library" but does not specify a version number for it. |
| Experiment Setup | Yes | We apply the Adam algorithm (Kingma & Ba, 2015) using the Pytorch library (Paszke et al., 2017) and update the parameters for 500,000 iterations. We use the batch size of 1,000 to train RCAE and GRCAE and the batch size of 10,000 to train DAE and GDAE. The learning rate starts at 2.5e-5 and is divided by ten after 250,000 iterations with a weight decay parameter of 1e-12. (...) For autoencoder-based models, we use the fully-connected neural network architecture that has 3-512-512-512-512-512-3 layers with Re LU-Re LU-linear-Re LU-Re LU-linear activation functions. For S-Flow that uses the Real NVP model (Dinh et al., 2016), the depth is eight and the length of the hidden feature is 512. For all cases, the learning rate is 1e-3 and the weight decay parameter is 1e-12. |