Delaunay Component Analysis for Evaluation of Data Representations
Authors: Petra Poklukar, Vladislav Polianskii, Anastasiia Varava, Florian T. Pokorny, Danica Kragic Jensfelt
ICLR 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We experimentally validate the proposed DCA method on representations obtained from neural networks trained with contrastive objective, supervised and generative models, and demonstrate various use cases of our extended single point evaluation framework. |
| Researcher Affiliation | Academia | Petra Poklukar , Vladislav Polianskii, Anastasia Varava, Florian T. Pokorny & Danica Kragic KTH Royal Institute of Technology, Stockholm, Sweden |
| Pseudocode | Yes | We provide an outline of our DCA framework in Algorithm 1 found in Appendix A. Moreover, by exploiting the nature of Delaunay graphs, DCA can be efficiently implemented (Section 3.2) and extended for evaluation of individual representations (Section 3.1). ... We summarize our DCA framework in Algorithm 1 and its q-DCA extension in Algorithm 2. |
| Open Source Code | Yes | Our implementation of the DCA algorithm is based on the C++/Open CL implementation of Delaunay graph approximation provided by Polianskii & Pokorny (2019) as well as on Python libraries HDBSCAN (Mc Innes et al., 2017) and Networkx (Hagberg et al., 2008). The code is available on Github4. ... Full implementation of our DCA algorithm together with the code for reproducing our experiments is available on our Git Hub4. |
| Open Datasets | Yes | We considered a similar experimental setup as in (Poklukar et al., 2021) and analyzed (i) representation space of a contrastive learning model trained with NT-Xent contrastive loss (Chen et al., 2020a), (ii) generation capabilities of a Style GAN trained on the FFHQ dataset (Karras et al., 2019), and (iii) representation space of the widely used VGG16 supervised model (Simonyan & Zisserman, 2015) pretrained on the Image Net dataset (Deng et al., 2009). ... We provide datasets used in all experiments in Section 4.1 and the q-DCA experiment in Section 4.2. |
| Dataset Splits | Yes | The training DR f and test DE f datasets each consisted of 5000 images containing four boxes arranged in 12 possible configurations, referred to as classes, recorded from the front camera angle (Figure 4 left). We created the sets R and E from 12-dimensional encodings of DR f and DE f , respectively, corresponding to the first 7 classes, c0, . . . , c6. ... We constructed R by randomly sampling 10000 points, and composed the query set Q1 of the remaining 2758 points. |
| Hardware Specification | Yes | DCA RUNTIME: Lastly, we additionally report empirical runtime (obtained on NVIDIA Ge Force GTX 1650 with Max-Q Design) |
| Software Dependencies | No | Our implementation of the DCA algorithm is based on the C++/Open CL implementation of Delaunay graph approximation provided by Polianskii & Pokorny (2019) as well as on Python libraries HDBSCAN (Mc Innes et al., 2017) and Networkx (Hagberg et al., 2008). The paper lists software libraries used but does not provide specific version numbers for them. |
| Experiment Setup | Yes | For all experiments, we set ηc = 0.0 and ηq = 0.0 as done in Section 4.2. We always fixed T = 104 in the approximation of GD, B = 0.7 and msc = 10 for HDBSCAN except in Ab-1) where we varied msc {3, 5, 10, 20}, in Ab-2) where we varied B {0.5, 0.6, 0.7, 0.8, 0.9, 1.0}, and in Ab-3) where we varied T {10, 102, 103, 105, 106}. ... We report the hyperparameters choices of the respective methods in Appendix B. |