Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Solving Analogies on Words based on Minimal Complexity Transformation
Authors: Pierre-Alexandre Murena, Marie Al-Ghossein, Jean-Louis Dessalles, Antoine Cornuéjols
IJCAI 2020 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We illustrate our method on a large-scale benchmark dataset and compare with state-of-the-art approaches to demonstrate the interest of using complexity to solve analogies on words. |
| Researcher Affiliation | Academia | 1Helsinki Institute for Information Technology HIIT, Department of Computer Science, Aalto University, Espoo, Finland 2LTCI, T el ecom Paris, Institut Polytechnique de Paris, Palaiseau, France 3UMR MIA-518, Agro Paris Tech INRA, Paris, France |
| Pseudocode | No | The paper describes its algorithm textually but does not provide structured pseudocode or an algorithm block. |
| Open Source Code | Yes | The source code for a Python interpreter is available on the authors webpage. |
| Open Datasets | Yes | We base our work on one of the largest datasets available for solving analogies on words that we denote by SIGMORPHON 16 and that is presented in [Lepage, 2017]. This dataset is from the Track 1 Task 1 of SIGMORPHON 2016 Shared Task 2. |
| Dataset Splits | No | The paper uses the SIGMORPHON 16 dataset but does not explicitly detail the train/validation/test splits (e.g., percentages or specific counts) used for their experiments. |
| Hardware Specification | Yes | For instance, and with a reasonable implementation in C++1 run on a machine with one processor Intel Core i7 2.9 GHz and 8G of RAM, it requires a couple of seconds to find a solution for the analogy rosa:rosam::vita:x and about 20 minutes for the analogy orang:orang-orang::burung:x . |
| Software Dependencies | No | The paper mentions implementation in Python and C++ but does not provide specific version numbers for these languages or any other software dependencies (e.g., libraries, frameworks). |
| Experiment Setup | Yes | In our experiments, we fixed it to |c| + |b| |a|, taking our inspiration from the works on proportional analogy [Lepage, 2017]. |