Fake It Until You Make It : Towards Accurate Near-Distribution Novelty Detection
Authors: Hossein Mirzaei, Mohammadreza Salehi, Sajjad Shahabi, Efstratios Gavves, Cees G. M. Snoek, Mohammad Sabokrou, Mohammad Hossein Rohban
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We provide a quantitative as well as qualitative evaluation of this strategy, and compare the results with a variety of GAN-based models. Effectiveness of our method for both the near-distribution and standard novelty detection is assessed through extensive experiments on datasets in diverse applications such as medical images, object classification, and quality control. This reveals that our method considerably improves over existing models, and consistently decreases the gap between the near-distribution and standard novelty detection performance. |
| Researcher Affiliation | Academia | 1 Sharif University of Technology 2 University of Amsterdam 3 Institute for Research in Fundamental Sciences (IPM) 4 Okinawa Institute of Science and Technology |
| Pseudocode | No | The paper describes the proposed pipeline and provides equations but does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | The code repository is available at https://github.com/rohban-lab/FITYMI. |
| Open Datasets | Yes | The datasets that are tested in this setting include CIFAR-10 (23), CIFAR-100 (23), MVTec AD (3), Birds (46), Flowers (30). To evaluate the performance on near-ND benchmark, we use 4 different datasets that naturally show slight differences between their class distributions... The datasets that are tested in this setting include FGVC-Aircraft (25), Stanford-Cars (22), White Blood Cell (WBC) (49), and Weather (10). |
| Dataset Splits | No | For the WBC dataset, the paper states: 'WBC : we used the 4 big classes in Dataset 1 of the microscopy images of white blood cells, with a 80%/20% train-test split.' However, it does not consistently provide explicit validation splits or standard train/validation/test splits for all datasets or the overall experimental setup in a reproducible manner in the main text or appendix. |
| Hardware Specification | No | The paper does not explicitly state the specific hardware details (e.g., GPU models, CPU types, or memory) used to run the experiments. It mentions using 'a Vi T-B_16 as the feature extractor' which is a model architecture, not hardware. |
| Software Dependencies | No | The paper does not provide specific software dependency versions (e.g., library names with version numbers like PyTorch 1.9 or TensorFlow 2.x, CUDA versions, or Python versions) for the experimental setup. |
| Experiment Setup | Yes | We use a Vi T-B_16 as the feature extractor (pretrained on Image Net 21k), learning rate = 4e-4, weight decay = 5e-5, batch size = 16, optimizer = SGD, and a linear head with 2 output neurons. We freeze the first six layers of the network, and the rest is fine-tuned till convergence for all the experiments. The inputs are resized to 224 224, and as a rule of thumb, the data generation is stopped on FID 40 and FID 200 for low and high-resolution datasets. |