Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Towards Green Automated Machine Learning: Status Quo and Future Directions
Authors: Tanja Tornede, Alexander Tornede, Jonas Hanselle, Felix Mohr, Marcel Wever, Eyke Hüllermeier
JAIR 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Accordingly, most research contributions are of empirical nature: the proposition of a new technique or approach is accompanied by large experimental studies to show the benefits of the proposed techniques. As one is interested in techniques that work well in general, across a broad range of problems, a large number of datasets is required for evaluation. |
| Researcher Affiliation | Academia | Department of Computer Science, Paderborn University, Germany; Universidad de La Sabana, Chia, Cundinamarca, Colombia; Institute of Informatics, University of Munich (LMU), Germany; Munich Center for Machine Learning (MCML), Germany |
| Pseudocode | No | The paper is a survey and position paper that proposes concepts and strategies for 'Green Auto ML'. It discusses various approaches and their theoretical underpinnings but does not present any novel algorithms or procedures in the form of pseudocode or algorithm blocks. |
| Open Source Code | No | The paper is a survey and position paper that discusses concepts and existing tools for Green Auto ML. It refers to third-party tools like Carbontracker, Ir Ene, and Code Carbon and provides links to them, but it does not provide source code for any methodology presented in this paper. |
| Open Datasets | Yes | For example, the Open ML (Vanschoren et al., 2013) Auto ML benchmark (Gijsbers et al., 2019) features 39 datasets, for each of which an Auto ML system is given a total of 4 hours to search for a suitable pipeline. |
| Dataset Splits | No | The paper discusses concepts related to dataset splits in the context of benchmarking (e.g., "k-fold cross-validation", "Monte-Carlo Cross Validation"). However, as a survey and position paper, it does not conduct its own experiments and therefore does not provide specific training/test/validation dataset splits for its own work. |
| Hardware Specification | No | The paper is a survey and position paper and does not present its own experimental results. While it references hardware used in other research (e.g., "450 GPUs for 7 days", "800 GPUs for 28 days") and benchmarks (e.g., "Amazon AWS m5.2xlarge compute nodes, featuring an Intel Xeon Platinum 8000 Series Skylake-SP processor with 8 CPU cores and 32GB memory"), it does not provide hardware specifications for its own work. |
| Software Dependencies | No | The paper is a survey and position paper and does not present its own experimental results or methodology that would require specific software dependencies. It mentions various software tools used by other researchers (e.g., "Carbontracker", "Ir Ene", "Code Carbon") but does not list any specific software dependencies with version numbers for its own work. |
| Experiment Setup | No | The paper is a survey and position paper proposing concepts and future directions for Green Auto ML. It does not conduct its own experiments and therefore does not provide specific experimental setup details, such as hyperparameters or training configurations. |