Revisiting Discriminative vs. Generative Classifiers: Theory and Implications
Authors: Chenyu Zheng, Guoqiang Wu, Fan Bao, Yue Cao, Chongxuan Li, Jun Zhu
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Simulation results on a mixture of Gaussian validate our theoretical findings. Experiments on various pre-trained deep vision models show that naïve Bayes consistently converges faster as the number of data increases. |
| Researcher Affiliation | Collaboration | 1Gaoling School of AI, Renmin University of China; Beijing Key Laboratory of Big Data Management and Analysis Methods, Beijing, China 2School of Software, Shandong University 3Dept. of Comp. Sci. & Tech., Institute for AI, Tsinghua-Huawei Joint Center for AI, BNRist Center, THBI Lab, Tsinghua University 4Beijing Academy of Artificial Intelligence. Correspondence to: Chongxuan Li <chongxuanli@ruc.edu.cn>. |
| Pseudocode | No | The paper does not contain any pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code is available at https://github.com/ML-GSAI/Revisiting-Dis-vs Gen-Classifiers. |
| Open Datasets | Yes | Experiments on various pre-trained deep vision models... We systematically compare logistic regression and naïve Bayes on the CIFAR10 and CIFAR100 datasets (Krizhevsky et al., 2009). |
| Dataset Splits | Yes | We systematically compare logistic regression and naïve Bayes on the CIFAR10 and CIFAR100 datasets (Krizhevsky et al., 2009). |
| Hardware Specification | Yes | All experiments are done on a single Ge Force RTX 3090 GPU. |
| Software Dependencies | No | We train the logistic regression using scikit-learn s (Pedregosa et al., 2011) L-BFGS implementation... Res Net. We add the Res Net50 checkpoint released by Pytorch (Paszke et al., 2019). (Specific software versions are not provided). |
| Experiment Setup | Yes | For a fair comparison, we keep the linear evaluation setting in (Radford et al., 2021) throughout the experiments. Specially, we train the logistic regression using scikitlearn s (Pedregosa et al., 2011) L-BFGS implementation, with a maximum of 1000 iterations. We adjust the weight of ℓ2 regularization of logistic regression carefully... For each m, we obtain training samples randomly 5 times and record the mean test error of two models. |