Exploiting Human-AI Dependence for Learning to Defer
Authors: Zixi Wei, Yuzhou Cao, Lei Feng
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Comprehensive experimental results on both synthetic and real-world datasets demonstrate the superiority of our proposed method. |
| Researcher Affiliation | Academia | 1College of Computer Science, Chongqing University, China 2School of Computer Science and Engineering, Nanyang Technological University, Singapore 3Information Systems Technology Design Pillar, Singapore University of Technology and Design, Singapore. |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any explicit statement about open-sourcing the code for its methodology or a link to a code repository. |
| Open Datasets | Yes | We conduct experiments for the proposed loss and baselines on widely used benchmark datasets with both synthetic and real-world experts. For synthetic experts, we perform experiments using CIFAR100 datasets (Krizhevsky, 2009) with the standard train-test split. For experiments involving real-world experts, we leverage the CIFAR-10N and CIFAR-100N datasets introduced by (Wei et al., 2021)... We also conduct experiments on Image Net-16H (Kerrigan et al., 2021)... |
| Dataset Splits | Yes | For synthetic experts, we perform experiments using CIFAR100 datasets (Krizhevsky, 2009) with the standard train-test split. We randomly partition the data into 80% training data and 20% test data for each trial. We also conduct experiments on Image Net-16H (Kerrigan et al., 2021)... using an 80-20 train-test split for each trial. |
| Hardware Specification | Yes | We train the model on each dataset for 400 epochs on 8 NVIDIA Ge Force 3090 GPUs. |
| Software Dependencies | No | The paper mentions using a 'wide residual network... to parameterize the scoring function g(x) and SGD is used for optimization' and mentions libraries like 'PyTorch' in its citations, but does not provide specific version numbers for software dependencies such as PyTorch or Python itself. |
| Experiment Setup | Yes | The learning rate is chosen from {3e 1, 1e 1, 3e 2, 1e 2} and the batch size is chosen from {512, 1024}, i.e., {64, 128} on each GPU. The weight decay is set as 5e 4. |