Contextual Memory Trees
Authors: Wen Sun, Alina Beygelzimer, Hal Daumé Iii, John Langford, Paul Mineiro
ICML 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimentally ( 4), we show this is useful in three different settings. |
| Researcher Affiliation | Collaboration | 1Robotics Institute, Carnegie Mellon University, USA 2Yahoo! Research, New York, NY, USA 3Microsoft Research, New York, NY, USA 4Microsoft, USA. |
| Pseudocode | Yes | Algorithm 1 PATH(query x, node v)... Algorithm 2 QUERY(query x, items to return k, exploration probability )... Algorithm 3 UPDATE((x, z, r), (v, a, p))... Algorithm 4 INSERT(node v, memory z, Reroute d)... Algorithm 5 INSERTLEAF(leaf node v, memory z)... Algorithm 6 REMOVE(x)... Algorithm 7 REROUTE() |
| Open Source Code | Yes | 1https://github.com/LAIRLAB/vowpal_wabbit/tree/master/demo/memory_tree |
| Open Datasets | Yes | We test the online learning ability of CMT on two multiclass classification datasets, ALOI (1000 labels with 100 examples per label) and Wiki Para 3-shot (10000 labels with 3 examples per label)... We compare CMT-augmented OAS to OAA on three multi-label datasets, RCV1-1K (Prabhu & Varma, 2014), Amazon Cat-13K (Mc Auley & Leskovec, 2013), and Wiki-31K (Zubiaga, 2012; Bhatia et al., 2015)... We used three benchmark datasets, (1) UIUC Pascal Dataset (Rashtchian et al., 2010), (2) Flickr8k dataset (Hodosh et al., 2013), and (3) MS COCO (Lin et al., 2014)... |
| Dataset Splits | Yes | Used online, we QUERY for an example, evaluate its loss, then apply UPDATE with the observed loss followed by INSERT of the data point. |
| Hardware Specification | No | The paper does not provide specific hardware details such as CPU/GPU models, memory specifications, or cloud instance types used for running the experiments. |
| Software Dependencies | No | The paper mentions software like 'Vowpal Wabbit' and 'scikit-learn' but does not specify their version numbers. |
| Experiment Setup | Yes | We implemented CMT as a reduction to Vowpal Wabbit’s (Langford et al., 2007) default learning algorithm. The routers (g) and the learning scorer (f) are all linear functions and are incrementally updated by an Adagrad (Duchi et al., 2011) gradient method in VW. |