Measures of diversity and space-filling designs for categorical data
Authors: Cedric Malherbe, Emilio Domı́nguez-Sánchez, Merwan Barlier, Igor Colin, Haitham Bou Ammar, Tom Diethe
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, a numerical assessment is provided to illustrate the potential of the designs with respect to state-of-the-art methods.In this section, we empirically demonstrate the advantages of GIPPR and GAC compared to the baselines 1) random designs, 2) discretized continuous design (d-Halton (Halton & Smith, 1964)) and 3) k-DPP designs (Kulesza et al., 2012). |
| Researcher Affiliation | Collaboration | 1DS&AI, Bio Pharmaceuticals R&D, Astra Zeneca, UK 2LTCI, T el ecom Paris 3University College London 4Huawei Noah s Ark Lab. |
| Pseudocode | Yes | Algorithm 1 Greedy Integer Programming Packing Radius (GIPPR) and Algorithm 2 Greedy Average Covering (GAC) |
| Open Source Code | No | The paper mentions using 'the open-source SCIP solver' and 'scipy QMC library (Virtanen et al., 2020)' for their implementations, but it does not provide an explicit statement or link to their own open-source code for GIPPR or GAC. |
| Open Datasets | Yes | We used three real-world systems f taken from the categorical benchmark of (Doerr et al., 2019): (1) LAB from telecommunication which measures the autocorrelation of a sequence of bits (2) MIS from combinatorics which consists of estimating the size of the largest independent set in a graph and (3) Ising Model from physics which measures the configuration s energy of the Ising spin glass model with d spins directing up or down. All the details can be found in Appendix D. and tuning (1) the architecture of a classification graph neural network over the standard Cora (Cora) and Cite Seer (Cite) datasets (Sen et al., 2008) |
| Dataset Splits | No | The paper states: 'For each design Dn, we labeled the corresponding dataset with a budget of n = 20 data and d = 10 parameters. Then, we learned a gaussian process model fθn by maximizing the marginal log-likelihood over the labeled data and evaluated its generalization capacities by computing its mean square error (MS) and log-likelihood (LL) over the whole space X in Figure 4.' It does not provide specific training, validation, or test dataset splits. |
| Hardware Specification | Yes | All the times have been recorded on the same laptop computer with an Intel i7 CPU @ 1.80GHz 1.99 GHz with 16GB of RAM and all the linear programs have been solved using the open-source SCIP solver, in python 3. and All the experiments were performed on a desktop computer with Intel i7 CPU @ 1.80GHz 1.99 GHz with 16GB of RAM. |
| Software Dependencies | No | The paper mentions 'python 3', 'open-source SCIP solver', and 'scipy QMC library', but does not provide specific version numbers for these components in the experimental setup or software dependencies section. While the SciPy reference points to SciPy 1.0, this is not stated directly as the version used in the experimental setup text. |
| Experiment Setup | Yes | For the gaussian process model, we used a standard ARD kernel used in (Wan et al., 2021) defined by: kθ(x, x ) = l exp Pd i=1 σi I{xi = x i} where θ = (l, σ1, . . . , σd) are d + 1 learnable parameters. ... using the Adam optimizer with 100 steps and a learning rate set to 10^-3. For GNN: The task here consists in choosing for each layer between two activation functions A {Re LU, Tan H}, to put a dropout of value 0.3 or not and finally to activate the skip-connection or not for each layer. ... network learned after T = 50 epochs, with the Adam optimizer with 1e-3 learning rate and a batch size of the size of the training set. |