Enhancing Tail Performance in Extreme Classifiers by Label Variance Reduction
Authors: Anirudh Buvanesh, Rahul Chand, Jatin Prakash, Bhawna Paliwal, Mudit Dhawan, Neelabh Madan, Deepesh Hada, Vidit Jain, SONU MEHTA, Yashoteja Prabhu, Manish Gupta, Ramachandran Ramjee, Manik Varma
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Comprehensive experiments are conducted on a diverse set of XC datasets, demonstrating that LEVER can enhance tail performance by around 5% and 6% points in PSP and coverage metrics, respectively, when integrated with leading extreme classifiers. |
| Researcher Affiliation | Industry | Microsoft {t-abuvanesh, t-rahulchand, t-japrakash, bhawna, t-mdhawan t-nmadan, deepeshhada, jainvidit, sonu.mehta, yprabhu gmanish, ramjee, manik}@microsoft.com |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code for LEVER is available at: aka.ms/lever. |
| Open Datasets | Yes | Traditionally, performance in XC is mostly assessed on the public datasets available from (Bhatia et al., 2016). Additionally, the datasets we release exhibit significant imbalances compared to existing benchmarks... Code to create the dataset from raw AOL search logs is available here https://github.com/anirudhb11/LEVER/tree/main/datasets/AOL. The LF-Wiki Hierarchy-1M dataset is available here https://github.com/anirudhb11/LEVER/tree/main/datasets/Wiki Hierarchy. |
| Dataset Splits | Yes | The hyperparameter τ is tuned using a validation set that contains 5% of the training data. |
| Hardware Specification | Yes | Training time (in hours) for different models on a single NVIDIA V100 GPU. |
| Software Dependencies | No | The paper mentions software components like Distil BERT, Mini LM, and implies the use of frameworks for deep learning (e.g., PyTorch by common practice for such models, though not explicitly stated with version). However, it does not provide specific version numbers for any of these software dependencies. |
| Experiment Setup | Yes | D MODEL DETAILS AND HYPERPARAMETERS. Table 16: Hyperparameters of tail-expert NGAME module. Table 17: Hyperparameters of ELIAS. Table 18: Hyperparameters of Cascade XML. Table 19: Hyperparameters of Ren ee. |