Towards Improving Calibration in Object Detection Under Domain Shift
Authors: Muhammad Akhtar Munir, Muhammad Haris Khan, M. Sarfraz, Mohsen Ali
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate TCD is capable of enhancing calibration with notable margins (1) across different DNN-based object detection paradigms both in in-domain and out-of-domain predictions, and (2) in different domain-adaptive detectors across challenging adaptation scenarios. Finally, we empirically show that our implicit calibration technique can be used in tandem with TCD during adaptation to further boost calibration in diverse domain shift scenarios. |
| Researcher Affiliation | Collaboration | Muhammad Akhtar Munir1,2 , Muhammad Haris Khan2, M. Saquib Sarfraz3,4, Mohsen Ali1 1 Information Technology University of Punjab, 2 Mohamed bin Zayed University of Artificial Intelligence, 3 Karlsruhe Institute of Technology, 4Mercedes-Benz Tech Innovation |
| Pseudocode | No | The paper describes the methods using mathematical formulations and descriptive text, but it does not include any explicit pseudocode blocks or algorithms. |
| Open Source Code | No | Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [No] |
| Open Datasets | Yes | Datasets: Cityscapes dataset [1] consists images of road and street scenes... KITTI dataset [4] offers images of road scenes... Foggy Cityscapes dataset [31] is constructed by simulating foggy-weather... Sim10k dataset [12] is a collection of 10K images... BDD100k dataset [39] contains 100k annotated images... |
| Dataset Splits | Yes | BDD100k dataset [39] contains 100k annotated images with bounding boxes and category labels. 70k images are used for training and 30k are used for validation. Following [38] we only consider daylight images to create a training subset of 36.7k images and a validation subset of 5.2k images. Note that, the validation subset is used as an evaluation set. |
| Hardware Specification | No | The paper states 'Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] In supplementary.' Since the supplementary material is not provided, specific hardware details are not available in the main paper. |
| Software Dependencies | No | The paper mentions loss functions like 'Focal loss' and 'IoU loss' and 'Smooth L1 loss and Cross-Entropy' used by the detectors. However, it does not specify software dependencies with version numbers (e.g., PyTorch 1.x, TensorFlow 2.x, Python 3.x). |
| Experiment Setup | Yes | Our TCD loss is developed to be used with the task-specific loss of DNN-based object detectors, including the SOTA domain-adaptive ones. For instance, the task specific losses for single-stage detectors use Focal loss [21] and Io U loss. Similarly Smooth L1 loss and Cross-Entropy are used in training the two-stage detectors. Let LD be the task-specific loss, then the total loss in our method is computed as: L = LD + LT CD. For further implementation details, refer to the supplementary. |