Improving Non-Transferable Representation Learning by Harnessing Content and Style
Authors: Ziming Hong, Zhenyi Wang, Li Shen, Yu Yao, Zhuo Huang, Shiming Chen, Chuanwu Yang, Mingming Gong, Tongliang Liu
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirically, the proposed H-NTL significantly outperforms competing methods by a large margin. |
| Researcher Affiliation | Collaboration | Ziming Hong1 Zhenyi Wang2 Li Shen3 Yu Yao4,5 Zhuo Huang1 Shiming Chen5 Chuanwu Yang6 Mingming Gong5,7 Tongliang Liu1 1Sydney AI Centre, The University of Sydney 2University of Maryland, College Park 3JD Explore Academy 4Carnegie Mellon University 5Mohamed bin Zayed University of Artificial Intelligence 6Huazhong University of Science and Technology 7The University of Melbourne |
| Pseudocode | Yes | Algorithm 1 Train target-specified H-NTL |
| Open Source Code | Yes | For t NTL and s NTL, we use their released code7 and the same hyperparameters settings reported in their paper to run experiments. 7https://github.com/condition Wang/NTL |
| Open Datasets | Yes | The digit tasks contain three random-selected pairs from four digit datasets: MNIST (MT) (Deng, 2012), MNIST-M (MM) (Ganin et al., 2016), SVHN (SN) (Netzer et al., 2011) and SYN-D (SD) (Roy et al., 2018). For challenging tasks, we involve CIFAR10 to STL10 (Coates et al., 2011) (C10 S10), Vis DA (Peng et al., 2017) (VT VV) and Office Home (Venkateswara et al., 2017) (OP OC). |
| Dataset Splits | Yes | Following Wang et al. (2022b), we randomly select 8,000 samples as training data and 1,000 samples as testing data without overlap for digit tasks, C10 S10, and VT VT. For OP OC, we use 3,000 for training data and 1,000 samples due to the limitation of the dataset size. |
| Hardware Specification | Yes | Our code is implemented in Python 3.8.8 and Py Torch 1.8.0. All experiments are conducted on a server running Ubuntu 20.04 LTS, equipped with an NVIDIA RTX A6000 GPU. |
| Software Dependencies | Yes | Our code is implemented in Python 3.8.8 and Py Torch 1.8.0. |
| Experiment Setup | Yes | For training SL, we employ the SGD as an optimizer with lr = 0.001 and set the batch size to 32. ... For our proposed H-NTL, we employ the SGD as an optimizer with lr = 0.1 and set the batch size to 128. The disentanglement VAE is trained for 20 epochs, and the dual-path knowledge distillation is trained for 30 epochs. The hyper-parameter λt is set to 1.0 for all datasets. |