Online Learning with Optimism and Delay
Authors: Genevieve E Flaspohler, Francesco Orabona, Judah Cohen, Soukayna Mouatadid, Miruna Oprescu, Paulo Orenstein, Lester Mackey
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conclude by benchmarking our algorithms on four subseasonal climate forecasting tasks, demonstrating low regret relative to state-of-the-art forecasting models. We validate our algorithms on the problem of subseasonal forecasting in Sec. 7. Subseasonal forecasting predicting precipitation and temperature 2-6 weeks in advance is a crucial task for allocating water resources and preparing for weather extremes (White et al., 2017). We evaluate the relative merits of the delayed online learning techniques presented by computing yearly regret and mean RMSE for the ensemble plays made by the online leaner in each year from 2011-2020. The average yearly RMSE for the three online learning algorithms and the six input models is shown in Table 1. |
| Researcher Affiliation | Collaboration | Dept. of EECS, Massachusetts Institute of Technology 2Dept. of AOSE, Woods Hole Oceanographic Institution 3Dept. of ECE, Boston University 4Atmospheric and Environmental Research 5Dept. of CS, University of Toronto 6Microsoft Research New England 7Instituto de Matem atica Pura y Aplicada. |
| Pseudocode | No | The paper describes algorithms (e.g., ODFTRL, DOOMD, DORM, DORM+) using mathematical formulas for their updates, but it does not present them in clearly labeled pseudocode blocks (e.g., Algorithm 1, Algorithm box). |
| Open Source Code | Yes | Our Python library for Optimistic Online Learning under Delay (Pool D) and experiment code are available at https://github.com/geflaspohler/poold. |
| Open Datasets | Yes | Our experiments are based on the subseasonal forecasting data of Flaspohler et al. (2021) that provides the forecasts of d = 6 machine learning and physics-based models for both temperature and precipitation at two forecast horizons: 3-4 weeks and 5-6 weeks. Flaspohler, G., Orabona, F., Cohen, J., Mouatadid, S., Oprescu, M., Orenstein, P., and Mackey, L. Replication Data for: Online Learning with Optimism and Delay, 2021. URL https://doi.org/ 10.7910/DVN/IOCFCY. |
| Dataset Splits | No | The paper describes an evaluation period (2011-2020) for its online learning algorithms, calculating yearly regret and RMSE. However, it does not explicitly provide traditional training/validation/test dataset splits with percentages or sample counts. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU/CPU models, memory) used to run the experiments. |
| Software Dependencies | No | The paper mentions "Our Python library for Optimistic Online Learning under Delay (Pool D)" but does not specify the version of Python or any other software dependencies with version numbers. |
| Experiment Setup | Yes | Our experiments are based on the subseasonal forecasting data... forecasts of d = 6 machine learning and physics-based models... In operational subseasonal forecasting, feedback is delayed; models make D = 2 or 3 forecasts... Unless otherwise specified, all online learning algorithms use the recent g hint gs, which approximates each unobserved subgradient at time t with the most recent observed subgradient gt D 1. See App. L for full experimental details, App. N for algorithmic details, and App. M for extended experimental results. |