Adaptive Robust Evidential Optimization For Open Set Detection from Imbalanced Data
Authors: Hitesh Sapkota, Qi Yu
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experimental results on multiple real-world datasets demonstrate that the proposed model outputs uncertainty scores that can clearly separate samples from closed and open sets, respectively, and the detection results outperform the competitive baselines. We perform extensive experimentation to evaluate the effectiveness of the proposed AREO model. We first describe five real-world image datasets where a minority class is introduced to create an imbalanced setting. We then assess the OSD performance of the proposed technique by comparing with competitive baselines. Finally, we conduct some qualitative analysis, which uncovers deeper insights on the performance advantage of the proposed model. |
| Researcher Affiliation | Academia | Hitesh Sapkota & Qi Yu Rochester Institute of Technology {hxs1943, qi.yu}@rit.edu |
| Pseudocode | Yes | Algorithm 1: Multi-Scheduler Learning Process Algorithm 2: Alternative Optimization between f and p using Probabilistic SGD |
| Open Source Code | Yes | For the source code, please click here. |
| Open Datasets | Yes | Our experiments involve five real-world image datasets: Cifar10, Cifar100 (Krizhevsky, 2009), Image Net (Deng et al., 2009), MNIST (Deng, 2012), and Architecture Heritage Elements Dataset (AHED) (Llamas, 2017). |
| Dataset Splits | Yes | For all datasets, for the hyperparameter optimization, randomly selected 20% of the training set is used. It should be noted that in our case, we used closed set classification performance (MAP) as eval metric. |
| Hardware Specification | No | The paper does not explicitly describe the hardware used to run its experiments. It does not mention specific GPU or CPU models, or any cloud/cluster specifications. |
| Software Dependencies | No | The paper mentions software components like 'Le Net5 network', 'tanh activation', 'Re LU', 'Soft Plus', 'Adam optimizer', and 'l2 regularization' but does not provide specific version numbers for any of these components or underlying libraries/frameworks. |
| Experiment Setup | Yes | For all datasets, we use an Le Net5 network with tanh activation in the feature extractor and Re LU in the fully connected layers. For training, we use the Adam optimizer with a learning rate of 0.001 and l2 regularization with a coefficient of 0.001. The detailed hyperparameter setting is provided in Appendix. We initialize the uncertainty set λ0 = 100 for all datasets so that model gives the equal emphasis to the all data samples. In terms of MSF parameters (w and w ) for each model in P, we initialize them by uniformly sampling from [0, 1]. |