Hallo3D: Multi-Modal Hallucination Detection and Mitigation for Consistent 3D Content Generation
Authors: Hongbo Wang, Jie Cao, Jin Liu, Xiaoqiang Zhou, Huaibo Huang, Ran He
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments demonstrate that our method significantly improves the consistency and quality of generated 3D content, particularly in mitigating hallucinations common with 2D pretrained models. |
| Researcher Affiliation | Academia | Hongbo Wang1,2 Jie Cao1,2 Jin Liu1,3 Xiaoqiang Zhou1,4 Huaibo Huang1,2 Ran He1,2 1MAIS & NLPR, Institute of Automation, Chinese Academy of Sciences, Beijing, China 2School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China 3School of Information Science and Technology, Shanghai Tech University, Shanghai, China 4University of Science and Technology of China, Hefei, China |
| Pseudocode | No | The paper describes the methodology using text and equations but does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | No | Answer: [No] Justification: We are currently organizing the code and plan to release it as open-source in the future. |
| Open Datasets | Yes | Specifically, we selected 60 objects from the GSO [8] and Objaverse [6] datasets, replacing overly simple objects to ensure a more robust evaluation. |
| Dataset Splits | No | The paper discusses training and testing but does not explicitly provide specific dataset split information for validation (exact percentages, sample counts, or detailed splitting methodology). |
| Hardware Specification | Yes | We recorded the runtime using two baselines: Gaussian Dreamer [63] based on 3DGS with fewer iterations and faster speed, and Dream Fusion [40] based on Ne RF with more iterations and slower speed, on NVIDIA V100. |
| Software Dependencies | No | The paper mentions software like 'Threestudio library' and specific models but does not provide specific version numbers for software dependencies. |
| Experiment Setup | Yes | Identical parameter configurations and seed values were maintained for fair comparison, using default hyperparameters from the baselines open-source implementations. ...we set the batch size to 4 for all baselines. To ensure a fair comparison, we reduced the number of iterations to 1/4 of the original. For example, Dream Fusion originally trained for 10,000 iterations, and we adjusted it to 2,500 for optimization. ...we begin calculating LCG later in the training and only every 4 iterations in our experiments. ...we assigned a weight of w = 0.1 to LCG in Eq.8. |