Online Deep Learning: Learning Deep Neural Networks on the Fly
Authors: Doyen Sahoo, Quang Pham, Jing Lu, Steven C. H. Hoi
IJCAI 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We validate the efficacy on large data sets (both stationary and concept drifting scenarios).4 Experiments 4.1 Datasets We consider several large scale datasets. Higgs and Susy are Physics datasets from UCI repository. For Higgs, we sampled 5 million instances. We used 5 million instances from Infinite MNIST [Loosli et al., 2007]. We also evaluated on 3 synthetic datasets. |
| Researcher Affiliation | Collaboration | Doyen Sahoo1, Quang Pham1, Jing Lu2, Steven C. H. Hoi1 1 School of Information Systems, Singapore Management Univerity, 2 JD.com {doyens,hqpham.2017}@smu.edu.sg, lvjing12@jd.com, chhoi@smu.edu.sg |
| Pseudocode | Yes | Algorithm 1 Online Deep Learning (ODL) using HBP |
| Open Source Code | Yes | Source code available at https://github.com/LIBOL/ODL |
| Open Datasets | Yes | Higgs and Susy are Physics datasets from UCI repository. For Higgs, we sampled 5 million instances. We used 5 million instances from Infinite MNIST [Loosli et al., 2007]. |
| Dataset Splits | No | The paper describes evaluating performance in different 'windows' or 'stages' of the learning process (e.g., 'first 0.5% of data', '[10-15]%', '[60-80]%'), which is characteristic of online learning where data arrives sequentially. It does not specify explicit train/validation/test dataset splits as used in batch learning for reproduction. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | No | Implementation was in Keras [Chollet, 2015]. However, no specific version number for Keras or any other software dependencies is mentioned. |
| Experiment Setup | Yes | Configuration across all methods: Re LU activation, fixed learning rate of 0.01 (finetuned on the baselines). For momentum, a fixed learning rate of 0.001 was used, and momentum parameters were finetuned to give the best performance on the baselines. For HBP, we set β = 0.99 and the smoothing parameter s = 0.2. |