Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
No Free Lunch: Fundamental Limits of Learning Non-Hallucinating Generative Models
Authors: Changlong Wu, Ananth Grama, Wojciech Szpankowski
ICLR 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | In this paper, we develop a theoretical framework to analyze the learnability of nonhallucinating generative models from a learning-theoretic perspective. Our results reveal that non-hallucinating learning is statistically impossible when relying solely on the training dataset, even for a hypothesis class of size two and when the entire training set is truthful. Our findings are primarily conceptual, they represent a first step towards a principled approach to addressing hallucinations in learning generative models. |
| Researcher Affiliation | Academia | Changlong Wu1, Ananth Grama1 & Wojciech Szpankowski1,2 1CSo I, Purdue University 2Jagiellonian University EMAIL EMAIL |
| Pseudocode | No | The paper describes learning rules in mathematical equations (e.g., equations 7 and 8) but does not present any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain any explicit statements about releasing code, nor does it provide links to source code repositories. |
| Open Datasets | No | The paper focuses on theoretical analysis of generative models and the learnability of non-hallucinating models. It does not mention or use any specific datasets for empirical evaluation, thus no information about public availability of datasets is provided. |
| Dataset Splits | No | The paper is theoretical and does not involve empirical experiments with datasets, therefore, no dataset splits are discussed or provided. |
| Hardware Specification | No | The paper is theoretical and does not describe any experimental setup or results that would require specific hardware. No hardware specifications are mentioned. |
| Software Dependencies | No | The paper is theoretical and does not describe any experimental setup that would require specific software dependencies. No software names with version numbers are mentioned. |
| Experiment Setup | No | The paper presents a theoretical framework and mathematical proofs. It does not include any experimental results or detailed setup configurations such as hyperparameters or training schedules. |