Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Proper Losses for Discrete Generative Models
Authors: Dhamma Kimpara, Rafael Frongillo, Bo Waggoner
ICML 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, we experimentally evaluate our losses as a proof of concept. |
| Researcher Affiliation | Academia | 1University of Colorado Boulder. |
| Pseudocode | No | The paper does not contain any pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | No | The paper does not provide an explicit statement about releasing source code or a link to a code repository for the methodology it describes. |
| Open Datasets | No | The paper conducts numerical experiments using theoretical distributions (e.g., 'power law distributions', 'Zipfians') rather than external public datasets that would require access information. Thus, there is no public dataset access information provided. |
| Dataset Splits | No | The paper does not provide specific training, validation, or test dataset splits. The experiments involve drawing samples from distributions for evaluation, not partitioning a fixed dataset for model training and evaluation in the typical machine learning sense. |
| Hardware Specification | No | The paper does not explicitly describe the hardware used to run its experiments, such as specific GPU or CPU models. |
| Software Dependencies | No | The paper does not provide a reproducible description of ancillary software with specific version numbers (e.g., programming languages or libraries with their versions). |
| Experiment Setup | Yes | For each pair of distributions p and q, at each number of total samples, we measured the absolute deviation between the loss value and the true distance between the distributions. We drew up to K1.5 total samples. We repeated this experiment for various batch sizes, where at each iteration, we drew the same batch size from p and q. |