Nonlinear Dynamic Boltzmann Machines for Time-Series Prediction
Authors: Sakyasingha Dasgupta, Takayuki Osogami
AAAI 2017 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Numerical experiments with synthetic datasets show that the RNN-Gaussian Dy BM improves predictive accuracy upon standard VAR by up to 35%. On real multi-dimensional time-series prediction, consisting of high nonlinearity and non-stationarity, we demonstrate that this nonlinear Dy BM model achieves significant improvement upon state of the art baseline methods like VAR and long short-term memory (LSTM) networks at a reduced computational cost. |
| Researcher Affiliation | Industry | Sakyasingha Dasgupta and Takayuki Osogami IBM Research Tokyo {sdasgup, osogami}@jp.ibm.com |
| Pseudocode | No | No explicit pseudocode or algorithm blocks found. |
| Open Source Code | Yes | See algorithmic description in supplementary (Dasgupta and Osogami 2016). LSTM was implemented in Keras see supplementary (Dasgupta and Osogami 2016). |
| Open Datasets | Yes | Monthly sunspot number prediction4: In the second task, we use the historic benchmark of monthly sunspot number (Hipel and Mc Leod 1994) collected in Zurich from Jan. 1749 to Dec. 1983. This is a one-dimensional nonlinear time series with 2820 time steps. ... 4Publicly available at https://datamarket.com/data/set/22t4/, Weekly retail gasoline and diesel prices in U.S.3: This dataset consists of real valued time series of 1223 steps... 3Data obtained from http://www.eia.gov/petroleum/ |
| Dataset Splits | No | We use the first 67% of the time series observations (819 time steps) as the training set and the remaining 33% (404 time steps) as the test data set. |
| Hardware Specification | Yes | All the experiments were carried out with a Python 2.7 implementation (with numpy and theano backend) on a Macbook Air with Intel Core i5 and 8 GB of memory. |
| Software Dependencies | No | All the experiments were carried out with a Python 2.7 implementation (with numpy and theano backend) on a Macbook Air with Intel Core i5 and 8 GB of memory. |
| Experiment Setup | Yes | The learning rates, η and η , in (14)-(18) is adjusted for each parameter according to RMSProp (Tieleman and Hinton 2012), where the initial learning rate was set to 0.001. Throughout, the initial values of the parameters and variables, including eligibility traces and the spikes in the FIFO queues, are set to 0. However, we initialize σ j = 1 for each j to avoid division by 0 error. All RNN weight matrices were initialized randomly as described in the RNN-G Dy BM model section. The RNN layer leak rate ρ was set to 0.9 in all experiments. Wrnn is initialized from a Gaussian distribution N (0,1) and Win is initialized from N (0,0.1). The sparsity of the RNN weight matrix can be controlled by the parameter φ and it is scaled to have a spectral radius less than one, for stability (Jaeger and Haass 2004). For all results presented here, the RNN weight matrix was 90% sparse and had a spectral radius of 0.95. |