Alice Benchmarks: Connecting Real World Re-Identification with the Synthetic
Authors: Xiaoxiao Sun, Yue Yao, Shengjin Wang, Hongdong Li, Liang Zheng
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this paper, we detail the settings of Alice benchmarks, provide an analysis of existing commonly-used domain adaptation methods, and discuss some interesting future directions. An online server1 has been set up for the community to evaluate methods conveniently and fairly. To provide a baseline analysis, we evaluate some common DA methods at the content level (Yao et al., 2020; 2022), pixel level (Deng et al., 2018a), feature level (Vu et al., 2019; Tsai et al., 2018), and based on pseudo labels (Fan et al., 2018; Zhong et al., 2019; 2020). We report their accuracy not only on the Alice target datasets, but also on some existing real-world datasets, such as Market1501 (Zheng et al., 2015) and Ve Ri-776 (Liu et al., 2016). With extensive experiments, we identify a number of interesting findings... |
| Researcher Affiliation | Academia | Xiaoxiao Sun1, Yue Yao1, Shengjin Wang2, Hongdong Li1, Liang Zheng1 1 The Australian National University 2 Tsinghua University {first-name.last-name}@anu.edu.au1 wgsgj@tsinghua.edu.cn2 |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Attr. desc.: https://github.com/yorkeyao/Vehicle X Vehicle X: https://github.com/yorkeyao/Vehicle X |
| Open Datasets | Yes | We make the real data we collected for Alice benchmarks publicly available on the links below: Alice-train: https://drive.google.com/file/d/19s Qdx Fw F9LTm K8Bjh INj Wvp8o-Uwjn Rc/view?usp=sharing Person X: https://github.com/sxzrt/Instructions-of-the-Person X-dataset Vehicle X: https://github.com/yorkeyao/Vehicle X |
| Dataset Splits | Yes | For each task, we divide the real-world target data into three splits, i.e., training, validation and testing (details are shown in Table 2). Each task includes a fully annotated validation set for the model and its hyper-parameter selection, containing 7,978 (from 100 identities) and 1,753 (from 50 identities) images, respectively. |
| Hardware Specification | Yes | The experiments were conducted on a server equipped with four RTX-2080TI GPUs and a 16-core AMD Threadripper CPU @ 3.5Ghz. |
| Software Dependencies | No | The paper mentions various software tools and frameworks used (e.g., Open Un Re ID, Star GAN, Faster R-CNN) but does not provide specific version numbers for any of them. |
| Experiment Setup | No | The paper mentions that "For vehicle re-ID, the image size is set to 256 256," and refers to external papers for other settings (e.g., "In our experiments, the settings from (Deng et al., 2018a) and (Yao et al., 2020) are used for person and vehicle domain adaptations, respectively."). It does not provide comprehensive hyperparameter values or training configurations directly within the main text. |