Contemplating Real-World Object Classification
Authors: ali borji
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We find that applying deep models to the isolated objects, rather than the entire scene as is done in the original paper, results in around 20-30% performance improvement. Relative to the numbers reported in Barbu et al., around 10-15% of the performance loss is recovered, without any test time data augmentation. Despite this gain, however, we conclude that deep models still suffer drastically on the Object Net dataset. We also investigate the robustness of models against synthetic image perturbations such as geometric transformations (e.g., scale, rotation, translation), natural image distortions (e.g., impulse noise, blur) as well as adversarial attacks (e.g., FGSM and PGD-5). Our results indicate that limiting the object area as much as possible (i.e., from the entire image to the bounding box to the segmentation mask) leads to consistent improvement in accuracy and robustness. |
| Researcher Affiliation | Industry | Ali Borji HCL America aliborji@gmail.com |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code and data are available at https://github.com/aliborji/Object Net Reanalysis.git2. |
| Open Datasets | Yes | Object Net contains 50,000 images across 313 categories, out of which 113 are in common with Image Net categories. ... To this end, we limit our analysis to the 113 overlapped categories between the two datasets. ... MS COCO dataset including chair, car, book, bottle, dinning table, umbrella, boat, motorcycle, sheep, and cow. |
| Dataset Splits | No | For the main Object Net reanalysis, the dataset is used as a test set for pre-trained models, and no train/validation split created by the authors is explicitly stated. For the MS COCO experiment, a train/test split is mentioned ('trained on 70 images per category... tested on the remaining 30 images'), but a separate validation split is not specified. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running its experiments, only discussing the software and datasets. |
| Software Dependencies | No | The paper mentions using 'Py Torch implementation' but does not specify a version number for PyTorch or any other software dependencies with their versions. |
| Experiment Setup | No | The paper provides some experimental setup details, such as 'trained on 70 images per category (700 in total) for 10 epochs' and mentions the architecture of the CNNs (2 conv layers, 2 fc layers) for MNIST/Fashion MNIST, but it does not include comprehensive specific hyperparameter values like learning rate, batch size, or optimizer settings for the main experiments. |