Data Level Lottery Ticket Hypothesis for Vision Transformers
Authors: Xuan Shen, Zhenglun Kong, Minghai Qin, Peiyan Dong, Geng Yuan, Xin Meng, Hao Tang, Xiaolong Ma, Yanzhi Wang
IJCAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The experiments show that there is a clear difference between the performance of models trained with winning tickets and randomly selected subsets, which verifies our proposed theory. |
| Researcher Affiliation | Collaboration | 1Northeastern University 2Western Digital Research 3Peking University 4ETH Zurich 5Clemson University |
| Pseudocode | No | The paper describes methods in prose and uses diagrams (Figure 1, Figure 2) but does not include any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | The Source codes are available at https://github. com/shawnricecake/vit-lottery-ticket-input. |
| Open Datasets | Yes | In this paper, all of the models are trained on the Image Net [Deng et al., 2009] with approximately 1.2 million images in the training set |
| Dataset Splits | No | The paper states "approximately 1.2 million images in the training set and all of the results of accuracy are tested on the 50k images in the testing set" for ImageNet, but does not specify a distinct validation set split. |
| Hardware Specification | Yes | All the experiments are conducted on the NVIDIA A100 with 8 GPUs. |
| Software Dependencies | No | The paper refers to models like Dei T, Swin, and LV-Vi T and states "we follow the principles proposed by the original papers of Dei T [Touvron et al., 2021], Swin [Liu et al., 2021b] and LV-Vi T [Jiang et al., 2021]", but does not list specific software libraries or frameworks with version numbers in its own text. |
| Experiment Setup | No | The paper states "For the training strategies and optimization methods, we follow the principles proposed by the original papers of Dei T [Touvron et al., 2021], Swin [Liu et al., 2021b] and LV-Vi T [Jiang et al., 2021]" rather than providing specific hyperparameters or detailed training configurations within its own text. |