FrugalML: How to use ML Prediction APIs more accurately and cheaply
Authors: Lingjiao Chen, Matei Zaharia, James Y. Zou
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct systematic experiments using ML APIs from Google, Microsoft, Amazon, IBM, Baidu and other providers for tasks including facial emotion recognition, sentiment analysis and speech recognition. Across various tasks, Frugal ML can achieve up to 90% cost reduction while matching the accuracy of the best single API, or up to 5% better accuracy while matching the best API s cost. |
| Researcher Affiliation | Academia | 1Department of Computer Sciences, 2 Department of Biomedical Data Science Stanford University |
| Pseudocode | Yes | Algorithm 1 Frugal ML Strategy Training. |
| Open Source Code | Yes | We release our code and our dataset1 of 612,139 samples annotated by commercial APIs as a resource to aid future research in this area. 1https://github.com/lchen001/Frugal ML |
| Open Datasets | Yes | Table 2: Datasets sample size and number of classes... We release our code and our dataset1 of 612,139 samples annotated by commercial APIs as a resource to aid future research in this area. 1https://github.com/lchen001/Frugal ML. |
| Dataset Splits | No | The paper mentions 'training on half of FER+' but does not explicitly describe a separate validation dataset split or how it was used. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU/CPU models, processor types, or memory amounts used for running experiments. |
| Software Dependencies | No | The paper refers to various open-source models and tools (e.g., 'Bixin', 'Vader', 'Deep Speech', 'PyTorch' indirectly), but it does not specify version numbers for any software dependencies or libraries. |
| Experiment Setup | No | The paper describes the Frugal ML algorithm but does not provide specific experimental setup details such as learning rates, batch sizes, or other hyperparameters for its training or for the underlying models. |