Domain-Aware Fine-Tuning: Enhancing Neural Network Adaptability
Authors: Seokhyeon Ha, Sunbeom Jeong, Jungwoo Lee
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments demonstrate that our method outperforms other baseline methods, demonstrating its effectiveness in not only improving performance but also mitigating feature distortion. |
| Researcher Affiliation | Collaboration | Seokhyeon Ha1, Sunbeom Jeong1, Jungwoo Lee1,2 1 Seoul National University 2 Hodoo AI Lab {aoxm1231, sb3991, junglee}@snu.ac.kr |
| Pseudocode | Yes | Algorithm 1: Batch Normalization Conversion |
| Open Source Code | No | The paper does not contain an explicit statement about releasing the source code for the methodology or a link to a code repository. |
| Open Datasets | Yes | CIFAR-10 (Krizhevsky et al. 2009): A dataset that contains 10 categories of objects. For the OOD dataset, we use two additional datasets, CIFAR-10.1 (Recht et al. 2018) and STL (Coates, Ng, and Lee 2011). |
| Dataset Splits | Yes | Consistent with LP-FT (Kumar et al. 2022), we use ℓ2-regularized logistic regression classifier for linear probing and also choose the best ℓ2-regularization hyperparameter based on ID validation accuracy. For fine-tuning, we employ an SGD classifier with a cosine learning rate schedule and a batch size of 64. We fine-tune for 20 epochs on CIFAR-10, Entity-30, and Living-17, but extend the fine-tuning to 50 epochs on Domain Net and f Mo W due to their limited number of images. Early stopping is applied based on the ID validation accuracy. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., exact GPU/CPU models, memory amounts) used for running its experiments, only general statements about computational resources. |
| Software Dependencies | No | The paper mentions software components like 'SGD classifier' and 'Deep Labv3+ framework' but does not provide specific version numbers for these or other software dependencies. |
| Experiment Setup | Yes | We fine-tune for 20 epochs on CIFAR-10, Entity-30, and Living-17, but extend the fine-tuning to 50 epochs on Domain Net and f Mo W due to their limited number of images. For fine-tuning, we employ an SGD classifier with a cosine learning rate schedule and a batch size of 64. [...] For the segmentation task, we adopt the Deep Labv3+ (Chen et al. 2018) framework and initialize its backbone with a pre-trained Res Net-50 model from Mo Cov2. During fine-tuning, we use the SGD optimizer with a batch size of 16 and a polynomial learning rate schedule with power 0.9. We conduct the fine-tuning process for a total of 30,000 iterations, and the final model is evaluated thereafter. |