The Traveling Observer Model: Multi-task Learning Through Spatial Variable Embeddings
Authors: Elliot Meyerson, Risto Miikkulainen
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5 EXPERIMENTS This section presents a suite of experiments that evaluate the behavior of the implementation introduced in Section 4. See Appendix for additional experimental details. ... The results show that the learned embeddings outperform zero and random embeddings, achieving performance on par with the Oracle (Table 2). |
| Researcher Affiliation | Collaboration | Elliot Meyerson Cognizant AI Labs elliot.meyerson@cognizant.com Risto Miikkulainen UT Austin & Cognizant AI Labs risto@cs.utexas.edu |
| Pseudocode | Yes | Figure 7: Pytorch code for the forward pass of the TOM implementation. |
| Open Source Code | No | Code for TOM will be available at https://github. com/leaf-ai/tom-release. |
| Open Datasets | Yes | The first experiment is based on the CIFAR dataset (Krizhevsky, 2009). ... The second experiment is based on the Melbourne minimum daily temperature dataset (Brownlee, 2016)... The set of tasks used is UCI-121 (Lichman, 2013; Fern andez-Delgado et al., 2014)... |
| Dataset Splits | Yes | For Daily Temperature, the second-to-last year of data is withheld for validation, and the final year is withheld for testing. ... The UCI-121 experiments use the preprocessed versions of the official train-val-test splits (https://github.com/bioinf-jku/SNNs/tree/master/UCI). |
| Hardware Specification | No | The paper states that models are implemented in PyTorch and use Adam for optimization, but it does not specify any hardware details such as GPU models, CPU types, or memory used for running the experiments. |
| Software Dependencies | No | all models are implemented in pytorch (Paske et al., 2017), use Adam for optimization (Kingma & Ba, 2014)... t-SNE (van der Maaten & Hinton, 2008) was used to reduce the dimensionality to two. t-SNE was run for 10K iterations with default parameters in the scikit-learn implementation (Pedregosa et al., 2011)... While software is mentioned, specific version numbers for PyTorch, scikit-learn, or other libraries are not provided. |
| Experiment Setup | Yes | The number of blocks in the encoder, core, and decoder is N = 3 for all problems except UCI-121, for which it is N = 10. All experiments use a hidden size of 128 for all dense layers aside from the final decoder layer that maps to the output space. ... Adam is used for all experiments, with all parameters initialized to their default values. ... The learning rate is kept constant at 0.001 throughout training. ... Models are trained for 500K steps for CIFAR, 100K steps for Daily Temperature, and 250K for Transposed Gaussian Process and Concentric Hyperspheres. ... Dropout is set to 0.0 for CIFAR, Daily Temperature, and Concentric Hyperspheres; and 0.5 for Transposed Gaussian Process and UCI-121. ... The batch size was 32 for CIFAR and Daily Temperature, and max(200, # train samples) for all other tasks. |