Towards Making Learnware Specification and Market Evolvable
Authors: Jian-Dong Liu, Zhi-Hao Tan, Zhi-Hua Zhou
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Theoretical analysis and extensive experiments on a learnware market prototype encompassing thousands of models and covering six real-world scenarios validate the effectiveness and efficiency of our approach. Extensive experimental results on a learnware market encompassing thousands of models and covering six real-world scenarios validate the effectiveness and efficiency of our approach. |
| Researcher Affiliation | Academia | Jian-Dong Liu, Zhi-Hao Tan, Zhi-Hua Zhou National Key Laboratory for Novel Software Technology, Nanjing University, China School of Artificial Intelligence, Nanjing University, China {liujd, tanzh, zhouzh}@lamda.nju.edu.cn |
| Pseudocode | Yes | Algorithm 1: RKMEIndex Construction(S, l, r, s) Algorithm 2: Lf Specifications Generation and Indexing |
| Open Source Code | No | The paper does not provide an explicit statement about releasing the source code for the described methodology or a link to a code repository. |
| Open Datasets | Yes | We have developed a learnware market prototype comprising 3090 models of various model types, each trained on different data sets, covering six real-world scenarios involving classification and regression. These scenarios correlate with several real-world datasets: Postures (Gardner et al. 2014), Bank (Moro, Cortez, and Rita 2014), Mushroom (Wagner, Heider, and Hattab 2021), PPG-Da Li A (Reiss et al. 2019), PFS (Kaggle 2018) and M5 (Makridakis, Spiliotis, and Assimakopoulos 2022). |
| Dataset Splits | No | Each dataset is naturally split into multiple parts with different data distributions based on categorical attributes, and each part is then further subdivided into training and test sets. The paper mentions training and test sets but does not explicitly describe a separate validation set split or how it was used for model tuning. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware (e.g., CPU, GPU, or TPU models) used to run the experiments. |
| Software Dependencies | No | The paper mentions types of models used (e.g., linear models, Light GBM, neural networks) but does not provide specific software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | For all RKME-based methods, we set the specification size nc to 100 and use the Gaussian kernel k(x1, x2) = exp( γ x1 x2 2 2) with γ = 0.1. In the submitting stage, we set r = 150 and select s from {3, 4} for different scenarios. In the deploying stage, we set the number of iterations Tu = 5 and the constants s = s, K = 50. |