An end-to-end approach for the verification problem: learning the right distance
Authors: Joao Monteiro, Isabela Albuquerque, Jahangir Alam, R Devon Hjelm, Tiago Falk
ICML 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirical evaluation shows such method defines an end-toend approach for the verification problem, able to attain better performance than simple scorers such as those based on cosine similarity and further outperforming widely used downstream classifiers. Evaluation on large scale verification tasks provides empirical evidence of the effectiveness in directly using outputs of the learned pseudo-distance for inference, outperforming commonly used downstream classifiers. We proceed to evaluation of the described framework and do so with four sets of experiments. |
| Researcher Affiliation | Collaboration | 1INRS-EMT, Universit e du Qu ebec, Montreal, Canada. 2Centre de Recherche Informatique de Montr eal, Montreal, Canada. 3Microsoft Research. 4Quebec Artificial Intelligence Institute, Universit e de Montr eal, Montreal, Canada.. |
| Pseudocode | Yes | Algorithm 1 Training procedure. E, D = Initialize Models() repeat x, y = Sample Minibatch() z = E(x) z+ = Get All Positive Pairs(z, y) z = Get All Negative Pairs(z, y) y = Project Onto Simplex(z) L = L(z+, z , D) + LCE(y , y) E, D = Update Rule(E, D, L ) until Maximum number of iterations reached return E, D |
| Open Source Code | Yes | Code for reproducing our experiments can be found at: https://github.com/joaomonteirof/e2e_verification |
| Open Datasets | Yes | We run proof-of-concept experiments and make use of standard image datasets to simulate verification settings. We report results on all trials created for the test sets of Cifar-10 and Mini-Image Net. We make use of the recently introduced Vox Celeb corpus (Nagrani et al., 2017; Chung et al., 2018) |
| Dataset Splits | Yes | For the case of Mini-Image Net, since that dataset was designed for few-shot learning applications, we have an open-set evaluation for verification since there are 64, 16, and 20 disjoint classes of training, validation, and test examples. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used, such as GPU models, CPU models, or memory specifications. It only mentions model architectures like ResNet-18 and ResNet-50. |
| Software Dependencies | No | The paper mentions software components implicitly (e.g., neural networks implying deep learning frameworks) but does not provide specific software dependencies with version numbers within the main text. |
| Experiment Setup | Yes | The learning rate is set to 0.001 and is reduced by a factor of 0.1 every 10 epochs. Training is carried out for 50 epochs. All hyperparameters are selected with a random search over a pre-defined grid. ... The grid used for hyperparameters selection along with the values chosen for each evaluation are presented in the appendix. |