Batch Reinforcement Learning for Smart Home Energy Management
Authors: Heider Berlink, Anna HR Costa
IJCAI 2015 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | RLb EMS was tested with real operational data from two different places. As these data have a strong dependence on location, we demonstrate that our proposed system can be used in situations characterized by distinct levels of uncertainty. The remainder of this paper is structured as follows. In Section 2, we briefly describe our mathematical framework. Our proposal, the RLb EMS system, is described in Section 3. In Section 4 we experimentally evaluate and analyze the RLb EMS system and, in Section 5, we conclude and discuss future steps. |
| Researcher Affiliation | Academia | Heider Berlink, Anna Helena Reali Costa Universidade de S ao Paulo S ao Paulo, SP, Brazil {heiderberlink, anna.reali}@usp.br |
| Pseudocode | Yes | The Fitted Q-iteration (FQI) algorithm [Ernst et al., 2005a] is one of the most popular algorithms in BRL due to its simple implementation and its excellent results. FQI converts the learning from interaction scheme to a series of supervised learning problems. The FQI Training algorithm is shown in Algorithm 1. |
| Open Source Code | No | The paper does not provide any statement or link indicating that the source code for the described methodology is publicly available. |
| Open Datasets | Yes | To test RLb EMS, we have gathered real historical data of energy generation and price for Brazil and the USA. The energy consumption data were generated from real statistical data on the usage of home appliances, available in [PROCEL, 2014]. ... The energy generation data were acquired from a real solar power plant in S ao Paulo, Brazil [Berlink et al., 2014]. ... For this test, we used the hourly pricing data of five consecutive years (2008-2013). For the simulation, we used the Solar PV System given by Chen et al. [2013]. |
| Dataset Splits | Yes | To perform our tests, we simulated the RLb EMS on-line execution using the Validation Data for Brazil and the USA. As we will see, the first one was a simplified version of the second one and was used as a preliminary test, to confirm the benefits of the proposed approach. Both smart homes, in Brazil and in the USA, generated their own energy with a Solar PV System and stored energy in a rechargeable battery. ... The annual historical data were divided into four sets of data, each one corresponding to one season of the year. After this, each database was divided into two subsets of data: Training Data and Validation Data. |
| Hardware Specification | No | The paper describes the components of the simulated smart home (e.g., Solar PV System details, battery model), but it does not specify any hardware used for running the experiments or training the models (e.g., CPU, GPU, memory, or cloud resources). |
| Software Dependencies | No | The paper mentions using a 'Radial Basis Neural Network' and 'Fitted Q-iteration (FQI) algorithm' but does not specify any software libraries, frameworks, or programming languages with their version numbers that were used for implementation or experimentation. |
| Experiment Setup | Yes | We define d Q = 5%, which means that the convergence criterion is met if the estimated Q-value for each sample of F varies less than 5 % from one iteration to the next. We use H = 200. |