Systematic Analysis of Cluster Similarity Indices: How to Validate Validation Measures
Authors: Martijn M Gösgens, Alexey Tikhonov, Liudmila Prokhorenkova
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We start with a series of experiments demonstrating the importance of the problem (Section 3). First, we construct simple examples showing the inconsistency of all pairs of different similarity indices. Then, we demonstrate that such disagreements often occur in practice when well-known clustering algorithms are applied to real datasets. Finally, we illustrate how an improper choice of a similarity index can affect the performance of production systems. |
| Researcher Affiliation | Collaboration | 1Eindhoven University of Technology, Eindhoven, The Netherlands 2Yandex, Berlin, Germany 3Yandex, Moscow, Russia 4Moscow Institute of Physics and Technology, Moscow, Russia 5HSE University, Moscow, Russia. Correspondence to: Martijn G osgens <research@martijngosgens.nl>. |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide concrete access to source code for the described methodology. It mentions using 'Scikit-learn', which is a third-party library, but not code written for this specific research. |
| Open Datasets | Yes | We ran 8 well-known clustering algorithms (Scikitlearn, 2020) on 16 real-world datasets from the UCI machine learning repository (Dua & Graff, 2017). |
| Dataset Splits | No | The paper mentions using real-world datasets but does not specify train, validation, or test splits. It refers to 'reference partition' for evaluation, but not general data splitting methodology. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running its experiments. |
| Software Dependencies | No | The paper mentions 'Scikit-learn, 2020' but does not provide specific version numbers for Scikit-learn or any other software dependencies needed for replication. |
| Experiment Setup | No | The paper describes experiments involving clustering algorithms and news aggregation, but it does not provide specific experimental setup details such as hyperparameters, learning rates, or detailed training configurations for the algorithms used. |