Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Early Classification of Time Series: A Survey and Benchmark
Authors: Aurélien Renault, Alexis Bondu, Antoine Cornuéjols, Vincent Lemaire
TMLR 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this paper, we highlight the two components of an ECTS system: decision and prediction, and focus on the approaches that separate them. This document begins by situating these methods within a principle-based taxonomy. It defines dimensions for organizing their evaluation and then reports the results of a very extensive set of experiments along these dimensions involving nine state-of-the-art ECTS algorithms. In addition, these and other experiments can be carried out using an open-source library in which most of the existing ECTS algorithms have been implemented (see https://github.com/ML-EDM/ml_edm). |
| Researcher Affiliation | Collaboration | Aurélien Renault EMAIL Orange Research Agro Paris Tech Alexis Bondu EMAIL Orange Research Antoine Cornuéjols EMAIL Agro Paris Tech Vincent Lemaire EMAIL Orange Research |
| Pseudocode | No | The paper describes methods in prose and with diagrams (e.g., Figure 1: Proposed taxonomy for separable ECTS approaches), but does not contain explicitly labeled 'Pseudocode' or 'Algorithm' blocks. |
| Open Source Code | Yes | An open source library is made available1 to enable reproducible experiments, as well as to facilitate the scientific community s development of future approaches. ... (see https://github.com/ML-EDM/ml_edm). |
| Open Datasets | Yes | In addition to the reference data used in the ECTS field, a collection of some thirty non-z-normalized datasets is proposed and provided to the community. ... All original datasets of the paper can be downloaded, already prepared and split, from https://urlz.fr/q Rqu |
| Dataset Splits | Yes | Splitting strategy: When not using predefined splits, the train sets are split into two distinct sets in a stratified fashion: a first one to train the different classifiers, corresponding to 40% of the training set and another one to train the trigger model, trained over the 60% left. The set used to train the classifiers is itself split into two different sets in order to train calibrators, using 30% of the given data. |
| Hardware Specification | Yes | All the experiments have been performed using a Linux operating system, with an Intel Xeon E5-2650 2.20GHz (24cores) and 252GB of RAM. |
| Software Dependencies | No | The paper mentions implementation in Python and the use of specific algorithms and tools like Mini ROCKET, WEASEL 2.0, XGBoost, and tsfresh, but does not provide specific version numbers for the Python interpreter or core libraries like scikit-learn, numpy, or pandas. |
| Experiment Setup | Yes | Two groups of hyperparameters need to be set: (i) some of them are meta parameters independent of the dataset and have been fixed according to the original papers, (ii) others have to be optimized using a grid search based on the Avg Cost criterion. The optimization of the second group of hyperparameters has been carried out using the value bounds mentioned in the originally published papers. |