Informative Dropout for Robust Representation Learning: A Shape-bias Perspective
Authors: Baifeng Shi, Dinghuai Zhang, Qi Dai, Zhanxing Zhu, Yadong Mu, Jingdong Wang
ICML 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Through extensive experiments, we observe enhanced robustness under various scenarios (domain generalization, few-shot classification, image corruption, and adversarial perturbation). |
| Researcher Affiliation | Collaboration | 1School of EECS, Peking University, China 2School of Mathematical Sciences, Peking University, China 3Microsoft Research Asia 4Center for Data Science, Peking University 5Beijing Institute of Big Data Research 6Wangxuan Institute of Computer Technology, Peking University. |
| Pseudocode | Yes | Algorithm 1 Informative Dropout (Info Drop) Input: input activation map zℓ 1 Parameters: convolutional kernel kℓ, bias bℓ, radius R, temperature T, bandwidth h, dropout rate r0 Output: output activation map zℓ for each element zℓ c,j in output do zℓ c,j σ(kℓ c pℓ 1 j + bℓ c) end for for c = 1 to cℓdo for i = 1 to r0 hℓ wℓ do sample j from [1, hℓ wℓ] with probability r(zℓ c,j) given by Eq. 1 zℓ c,j 0 end for end for |
| Open Source Code | Yes | Code is available at https: //github.com/bfshi/Info Drop. |
| Open Datasets | Yes | Following the literature (Carlucci et al., 2019), we use PACS (Li et al., 2017) as dataset... we evaluate Info Drop on two popular datasets: CUB (Wah et al., 2011) and mini-Image Net (Ravi & Larochelle, 2017)... we apply the same corruption functions on Caltech-256 dataset (Griffin et al., 2007)... we use CIFAR-10 (Krizhevsky et al., 2009) |
| Dataset Splits | Yes | Following the setting in Chen et al. (2019), we evaluate Info Drop on two popular datasets: CUB (Wah et al., 2011) and mini-Image Net (Ravi & Larochelle, 2017), meanwhile also test our model in the cross-domain scenario (Chen et al., 2019), where mini-Image Net is used for training and CUB for testing. |
| Hardware Specification | No | The paper mentions using convolutional neural networks (CNNs) and ResNet architectures for experiments, but it does not specify any particular hardware details such as GPU models, CPU types, or memory used for training or inference. |
| Software Dependencies | No | The paper references Pytorch in its bibliography (Paszke et al., 2019), suggesting its use, but it does not specify version numbers for PyTorch or any other software dependencies such as Python, CUDA, or other libraries. |
| Experiment Setup | No | Please refer to Appendix for specific experimental settings." and "Due to limited space, detailed experimental configuration and additional results are deferred to Appendix. |