Focused Hierarchical RNNs for Conditional Sequence Processing
Authors: Nan Rosemary Ke, Konrad Żołna, Alessandro Sordoni, Zhouhan Lin, Adam Trischler, Yoshua Bengio, Joelle Pineau, Laurent Charlin, Christopher Pal
ICML 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate this method on several types of tasks with different attributes. First, we evaluate the method on synthetic tasks which allow us to evaluate the model for its generalization ability and probe the behavior of the gates in more controlled settings. We then evaluate this approach on large scale Question Answering tasks including the challenging MS MARCO and Search QA tasks. Our models shows consistent improvements for both tasks over prior work and our baselines. |
| Researcher Affiliation | Collaboration | 1Montreal Institute for Learning Algorithms, Montreal, Canada 2Polytechnique Montreal 3Microsoft Research, Montreal 4Jagiellonian University, Cracow, Poland 5Adept Mind Scholar 6University of Montreal 7Senior Cifar Member 8Mc Gill University 9Facebook AI Research, Montreal 10HEC Montreal. |
| Pseudocode | No | The paper describes the architecture and mathematical formulations but does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide an explicit statement about releasing source code or a link to a code repository for its methodology. |
| Open Datasets | Yes | We then move on to challenging large scale QA tasks such as MS MARCO (Nguyen et al., 2016) and Search QA (Dunn et al., 2017). We adapt the Pixel-by-Pixel MNIST classification task (Le Cun et al., 1998; Le et al., 2015) to the question and answering setting. |
| Dataset Splits | Yes | The LSTM2 reached an accuracy of 98.4% on the validation set, and FHE3 outperformed the baseline by having an accuracy of 99.1%. Table 6. Search QA results measured in F1 and Exact Match (EM) for validation and test set. Ablation studies are evaluated on the validation set only. |
| Hardware Specification | No | The paper does not provide specific details on the hardware used for running the experiments (e.g., GPU models, CPU types, or memory specifications). |
| Software Dependencies | No | The paper mentions using Theano and the Adam optimizer, but it does not specify version numbers for any software dependencies or libraries. |
| Experiment Setup | Yes | Hyper-parameters All models (FHE, LSTM1 and LSTM2) for a certain task has the same number of hidden units (256 for picking task and 128 for Pixel-by-Pixel MNIST QA task). ... Learning rates used for all models are 0.0001 with the Adam optimizer (Kingma & Ba, 2014). ... FHE hyper-parameters were fixed (α = 0.001, β = 0.5, γ = 50%). We use the Adam optimizer (Kingma & Welling, 2014) with a learning rate of 0.001. |