On the Complexity of Bayesian Generalization
Authors: Yu-Zhe Shi, Manjie Xu, John E. Hopcroft, Kun He, Joshua B. Tenenbaum, Song-Chun Zhu, Ying Nian Wu, Wenjuan Han, Yixin Zhu
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results show that representations with relatively high subjective complexity outperform those with low subjective complexity in rule-based generalization, while the trend is the opposite in similarity-based generalization. In the remainder of the paper, we first present the new metrics, Representativeness of Attribute (Ro A), to measure the subjective complexity and analyze the computation-mode-shift in Sec. 2. Next, through a series of experiments, we provide strong evidence to support our hypotheses in Sec. 3 |
| Researcher Affiliation | Academia | 1Peking University 2National Key Laboratory of General Artificial Intelligence, BIGAI 3Cornell University 4Huazhong University of Science and Technology 5MIT 6UCLA 7Beijing Jiaotong University 8CUPK. |
| Pseudocode | No | The paper describes procedures and formulations mathematically, but it does not include any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | The experiment source code and the dataset information are available at https://github.com/Yuzhe SHI/ bayesian-generalization-complexity. |
| Open Datasets | Yes | Six groups of discriminative models were trained from scratch on six datasets (see Fig. A2) with the supervision of concept labels: LEGO (Tatman, 2017), 2D-Geo (El Korchi & Ghanou, 2020), ACRE (Zhang et al., 2021a), Aw A (Xian et al., 2018), Places (Zhou et al., 2017), and Image Net (Deng et al., 2009). |
| Dataset Splits | Yes | All models were opti-mized to converge on the training set and tuned to the best hyper-parameters on the validation set. We also employed early stopping based on the validation accuracy. |
| Hardware Specification | Yes | All models are trained on eight NVIDIA A100 80GB GPUs. |
| Software Dependencies | No | The paper mentions using 'Py Torch' and 'Res Net' (as model architecture) but does not provide specific version numbers for any software dependencies. |
| Experiment Setup | Yes | All models were opti-mized to converge on the training set and tuned to the best hyper-parameters on the validation set. During training, we used a cross-entropy loss function to optimize the model for classification performance. We also employed early stopping based on the validation accuracy. For both Image Net and Places365 datasets, we utilized the official pre-trained models from Py Torch and proceeded to finetune them over the course of 20 epochs. For other datasets, we train our model from scratch. |