Mean Field Theory in Deep Metric Learning
Authors: Takuya Furusawa
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We extensively evaluate these derived loss functions on three image-retrieval datasets and demonstrate that our loss functions outperform baseline methods in two out of the three datasets. |
| Researcher Affiliation | Industry | Takuya Furusawa ZOZO Research 1-3-22 Kioicho, Chiyoda-ku, Tokyo, Japan takuya.furusawa@zozo.com |
| Pseudocode | Yes | Algorithm 1 Mean field losses training procedure. |
| Open Source Code | No | The paper states that they utilize the PyTorch Metric Learning library (Musgrave et al., 2020b) to implement baseline models, which is an open-source library. However, it does not provide an explicit statement or link indicating that the source code for *their own* proposed methodology is publicly available. |
| Open Datasets | Yes | In our experiments, we utilize four publicly available image-retrieval datasets, CUB-200-2011 (CUB) (Wah et al., 2011), Cars-196 (Cars) (Krause et al., 2013), Stanford Online Products (SOP) (Oh Song et al., 2016), and In Shop (Liu et al., 2016). |
| Dataset Splits | Yes | We split a dataset into train valid (the first half classes) and test datasets (the remaining). The train valid set was further divided into four partitions in a class disjoint manner, and we performed four-fold cross-validation based on the leave-one-out method in each iteration. |
| Hardware Specification | Yes | We carry out the experiments on a single NVIDIA V100 GPU. ... The experiments on the CUB and Cars (SOP and In Shop) datasets are carried out on a single NVIDIA V100 (A100) GPU. |
| Software Dependencies | No | We implement our experiments in Py Torch (Paszke et al., 2019) and utilize the Py Torch Metric Learning library (Musgrave et al., 2020b) to implement baseline models. While specific software names are mentioned, no specific version numbers for PyTorch or PyTorch Metric Learning are provided in the text. |
| Experiment Setup | Yes | In the modern benchmarking protocol, we perform 50 iterations of Bayesian optimization for hyperparameters in loss functions including the learning rate for proxies and mean fields for a fair comparison. ... we train a model with embedding dimensions set to 128 and batch size set to 32... we utilize the RMSprop optimizer with learning rate 10^-6 for the embedding model. ... we use AdamW optimizer (Loshchilov & Hutter, 2017) with the learning rate 10^-4 for the embedding model, setting the learning rate for proxies to 10^-2 and that for mean fields to 2 * 10^-1. The hyperparameters for Proxy Anchor loss are fixed to (α, δ) = (32, 10^-1), while we set (m P, m N, λMF) = (0.02, 0.3, 0) for MFCont. loss and (α, β, δ, λMF) = (0.01, 80, 0.8, 0) for MFCWMS loss in default. |