Day-ahead energy market trading strategies using reinforcement learning and evolutionary algorithms

Łukasz Lepak

supervisor: Paweł Wawrzyński



Energy is a popular topic today. Rising prices, move towards renewable sources, atomic energy – these are just some areas that are wildly discussed. In this work, we focus on day-ahead energy markets and trading strategies. A hypothetical prosumer is considered who is able to generate energy and store it in batteries or use it for his own needs. He is also a market participant, which allows him to sell excess energy or buy deficit energy. The goal of the prosumer is to increase his profit, which may be accomplished by an appropriate trading strategy. In this work, two approaches are proposed. The first one is a fully automatic trading strategy, which is learned with reinforcement. It uses a set of observations, like previous prices, weather forecasts or battery state forecasts, to optimize its policy so that it creates the best buy and sell orders possible. The second approach uses hand-crafted strategies, in which parameters are optimized by a 1+1 evolutionary algorithm, to optimize the results obtained by these created strategies. Both approaches operate on a day-ahead energy market, where orders for each hour need to be submitted on a day preceding actual energy deliveries. We assess the results of proposed approaches by comparing various metrics, including profit/loss during the operation, the characteristics of submitted orders or the changes in battery states of the hypothetical prosumer.