Burning fossil fuels is responsible for a large portion of the greenhouse gases released into the atmosphere. In addition to their negative impacts on the environment, fossil fuels are limited, which makes the integration of renewable energy sources into the grid inevitable. However, the intermittent nature of renewable energy sources makes it challenging to regulate energy output, resulting in low system flexibility. Adoption of an energy storage system, such as pumped hydro energy storage (PHES) and batteries, is necessary to fully utilize and integrate a larger proportion of variable renewable energy sources into the grid. On the other hand, in investment planning problems, satisfying the demand for certainty for even infrequently occurring events can lead to considerable cost increases. In this study, we propose a chance constrained two-stage stochastic program for designing a hybrid renewable energy system where the intermittent solar energy output is supported by a closed-loop PHES system. The aim of this study is to minimize the total investment cost while meeting the energy demand at a predetermined service level. For our computational study, we generate scenarios for solar radiation by using an Auto-Regressive Integrated Moving Average (ARIMA) based algorithm. In order to exactly solve our large scale problem, we utilize a Benders based branch and cut decomposition algorithm. We analize the efficiency of our proposed solution method by comparing the CPU times provided by the proposed algorithm and CPLEX. The findings indicate that the proposed algorithm solves the problem faster than CPLEX.