Markov decision process formulations for management of pumped hydro energy storage systems

Abstract

Renewable energy sources have received much attention to mitigate the high dependence on fossil fuels and the resulting environmental impacts. Since the variability and intermittency of such renewable sources lower the reliability and security of energy systems, they should often be accompanied by efficient and flexible storage units. This dissertation focuses on pumped hydro energy storage (PHES) facilities, which are one of the most commonly used large-scale storage technologies. We study the energy generation and storage problem for PHES facilities with two connected reservoirs, where water is pumped from the lower reservoir to the upper reservoir to store energy during low-demand/low-electricity price periods, and released back to the lower reservoir to generate energy during high-demand/high-electricity price periods. The first part of this dissertation investigates the potential benefits of transforming conventional cascading hydropower stations into PHES facilities by replacing turbines with reversible ones. The second part compares the short-term cash flows obtained from different PHES configurations (cascading vs. non-cascading facilities, upstream vs. downstream inflows, and closed-loop facilities). We formulate both problems as Markov decision processes under uncertainty in the streamflow rate and electricity price. We include the streamflow rate and electricity price as exogenous state variables in our formulation. We analytically derive bounds on the profit improvement obtained from PHES transformation in the first part and bounds on the revenue differences obtained from different configurations in the second part. In the last part, we establish several structural properties of the optimal profit function for general two-reservoir PHES systems. We show the optimality of a state-dependent threshold policy for non-cascading PHES facilities when the electricity price is always positive. Leveraging our structural results, we construct a heuristic solution method for more general settings when the electricity price can also be negative. In this dissertation, we also conduct comprehensive numerical experiments with data-calibrated time series models to provide insights into the optimal operation of PHES facilities, considering distinct seasons with different streamflow rates, different negative electricity price occurrence frequencies, and different system parameters.