Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Position: Enough of Scaling LLMs! Lets Focus on Downscaling
Authors: Yash Goel, Ayan Sengupta, Tanmoy Chakraborty
ICML 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically validate the scaling laws of carbon cost for different model and pre-training data sizes, as illustrated in Figures 2 and 3, respectively. |
| Researcher Affiliation | Academia | 1Indian Institute of Technology Delhi, India. Correspondence to: Ayan Sengupta <EMAIL>, Yash Goel <EMAIL>. |
| Pseudocode | No | The paper contains mathematical equations and propositions but does not include any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | The source code of our analysis can be found at https://github.com/LCS2-IIITD/Downscaling. |
| Open Datasets | No | The paper discusses concepts related to dataset size and refers to other works that use datasets but does not provide specific access information (links, DOIs, repositories, or formal citations) for any datasets used in its own analysis or empirical validation (e.g., for Figures 2 and 3). |
| Dataset Splits | No | The paper does not describe any specific datasets used for its own experimental validation, and therefore, no dataset split information (training/test/validation percentages or counts) is provided. |
| Hardware Specification | No | The paper discusses hardware considerations in general terms (e.g., 'hardware configuration', 'GPUs, TPUs') and includes formulas for calculating carbon footprints that involve hardware parameters, but it does not specify the exact models or configurations of hardware used for its own analysis or empirical validation. |
| Software Dependencies | No | The paper mentions some software frameworks like LLMCarbon but does not provide specific version numbers for any software dependencies used for its own analysis or empirical validation. |
| Experiment Setup | No | The paper focuses on theoretical propositions and discussions around scaling laws. Although it includes figures for empirical validation, it does not provide specific experimental setup details such as hyperparameters, training configurations, or system-level settings for reproducing its analysis. |