On Matching Pursuit and Coordinate Descent
Authors: Francesco Locatello, Anant Raj, Sai Praneeth Karimireddy, Gunnar Raetsch, Bernhard Schölkopf, Sebastian Stich, Martin Jaggi
ICML 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 4. Empirical Evaluation In this section we aim at empirically validate our theoretical findings. In both experiments we use 1 and the intrinsic dimensionality of lin(A) as and 0 respectively. Note that a value of smaller than 0 represents the best case for the steepest update. We implicitly assume that the worst case in which a random update is as good as the steepest one never happens. Toy Data: First, we report the function value while minimizing the squared distance between the a random 100 dimensional signal with both positive and negative entries and its sparse representation in terms of atoms. We sample a random dictionary containing 200 atoms which we then make symmetric. The result is depicted in Figure 1. As anticipated from our analysis, the accelerated schemes converge much faster than the non-accelerated variants. Furthermore, in both cases the steepest update converge faster than the random one, due to a better dependency on the dimensionality of the space. Real Data: We use the under-sampled Urban HDI Dataset from which we extract the dictionary of atoms using the hierarchical clustering approached of (Gillis et al., 2015). This dataset contains 5 929 pixels, each associated with 162 hyperspectral features. The number of dictionary elements is 6, motivated by the fact that 6 different physical materials are depicted in this HSI data (Gillis & Luce, 2018). We approximate each pixel with a linear combination of the dictionary elements by minimizing the square distance between the observed pixel and our approximation. We report in Figure 2 the loss as an average across all the pixels: |
| Researcher Affiliation | Academia | 1Max Planck Institute for Intelligent Systems, Tübingen, Germany 2Dept. of Computer Science, ETH Zurich, Zurich, Switzerland 3EPFL, Lausanne, Switzerland. |
| Pseudocode | Yes | Algorithm 1 Generalized Matching Pursuit, Algorithm 2 Affine Invariant Generalized Matching Pursuit, Algorithm 3 Accelerated Random Pursuit, Algorithm 4 Accelerated Matching Pursuit |
| Open Source Code | No | The paper does not provide an explicit statement or link for the source code of the described methodology. |
| Open Datasets | Yes | Real Data: We use the under-sampled Urban HDI Dataset from which we extract the dictionary of atoms using the hierarchical clustering approached of (Gillis et al., 2015). This dataset contains 5 929 pixels, each associated with 162 hyperspectral features. The number of dictionary elements is 6, motivated by the fact that 6 different physical materials are depicted in this HSI data (Gillis & Luce, 2018). |
| Dataset Splits | No | The paper does not provide specific dataset split information (e.g., percentages, sample counts, or cross-validation details) for reproducibility. |
| Hardware Specification | No | The paper does not provide any specific hardware details (e.g., GPU/CPU models, memory amounts) used for its experiments. |
| Software Dependencies | No | The paper does not provide specific software dependency details (e.g., library or solver names with version numbers). |
| Experiment Setup | Yes | In both experiments we use 1 and the intrinsic dimensionality of lin(A) as and 0 respectively. |