Metric Learning with Adaptive Density Discrimination
Authors: Oren Rippel, Manohar Paluri, Piotr Dollar, Lubomir Bourdev
ICLR 2016 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the effectiveness of this idea on several tasks. Our approach achieves state-of-the-art classification results on a number of fine-grained visual recognition datasets, surpassing the standard softmax classifier and outperforming triplet loss by a relative margin of 30-40%. In terms of computational performance, it alleviates training inefficiencies in the traditional triplet loss, reaching the same error in 5-30 times fewer iterations. Beyond classification, we further validate the saliency of the learnt representations via their attribute concentration and hierarchy recovery properties, achieving 10-25% relative gains on the softmax classifier and 25-50% on triplet loss in these tasks. |
| Researcher Affiliation | Collaboration | Oren Rippel MIT, Facebook AI Research rippel@math.mit.edu Manohar Paluri Facebook AI Research mano@fb.com Piotr Dollar Facebook AI Research pdollar@fb.com Lubomir Bourdev UC Berkeley lubomir.bourdev@gmail.com |
| Pseudocode | No | The paper does not contain explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not include a statement about providing open-source code for the described methodology or a link to a repository. |
| Open Datasets | Yes | We validate the classification efficacy of the learnt representations on a number of popular finegrained visual categorization tasks, including Stanford Dogs (Khosla et al., 2011), Oxford-IIIT Pet (Parkhi et al., 2012) and Oxford 102 Flowers (Nilsback & Zisserman, 2008) datasets. We also include results on Image Net attributes, a dataset described in Section 4.2. We train softmax, triplet and Magnet Loss objectives on a curated dataset we refer to as Image Net Attributes... In Appendix C we describe it in detail. |
| Dataset Splits | Yes | We seek to compare optimal performances of the different model spaces, and so perform hyperparameter search on validation error generated by 3 classes of objectives: a standard softmax classifier, triplet loss, and Magnet Loss... The Image Net Attributes training and validation sets then comprise all examples of all classes for which annotated examples exist. |
| Hardware Specification | Yes | We run all experiments on a cluster of Tesla K40M GPU s. |
| Software Dependencies | No | The paper mentions using 'Goog Le Net' as the parameterized map to representation space but does not specify software dependencies with version numbers (e.g., deep learning frameworks like TensorFlow/PyTorch, CUDA versions, or other libraries). |
| Experiment Setup | Yes | We add an additional fully-connected layer to map to a representation space of dimension 1024. We augment all experiments with random input rescaling of up to 30%, followed by jittering back to the original input size of 224 224. At test-time we evaluate an input by averaging the outputs of 16 random samples drawn from this augmentation distribution. The hyperparameter search setup, including optimal configurations for each experiment, is specified in full detail in Appendix B. For Magnet Loss we observed empirically that it is beneficial to increase the number of clusters per minibatch to around M = 12 in the cost of reducing the number of retrieved examples per cluster to D = 4. The optimal gap has in general been α 1, and the value of K varied as function of dataset cardinality. For all models, we tune optimization hyperparameters consisting of learning rate and its annealing factor which we apply every epoch. We fix the momentum as 0.9 for all experiments. |