Solar forecasting based on all sky imaging, long short-term memory

Researchers from the Netherlands’ Utrecht University and EKO Instruments Europe developed a novel machine learning and all-sky imaging-based short-term solar irradiance forecasting.

The model is based on the long short-term memory (LSTM) model, which is a kind of recurrent neural network capable of learning order dependence in sequence prediction problems. The LSTM technique takes the relevant parts of a pre-trained machine learning model and applies it to a new but similar problem.

“We aim to enhance prediction accuracy by implementing data augmentation and possibly expanding the network of sensor sites to include real-time data from multiple locations,” the research's lead author, Khadija Barhmi, told pv magazine.

The academics explained that the LSTM uses memory cells, which can store information for a long period, and gates, which control the flow of information and make decisions about what is forgotten and what is kept.

“LSTM networks, a subset of Recurrent Neural Networks (RNNs), are known for their exceptional ability to model temporal sequences,” they stated. “By integrating LSTM with all-sky imager images and local meteorological data, our research formulates a novel approach that method enables us to create future representations of the sky’s evolution.”

The prediction method developed by the scientist utilized all-sky imaging, capturing sky images from which the model can learn about cloud placement, movement, and formation. It employs the threshold algorithm, classifying pixels as cloud or clear-sky based on pre-defined thresholds. Cloud movement is calculated using a two-frame flow estimation algorithm.

The team placed all-sky cameras at the Plataforma Solar de Almería (PSA) facility in southern Spain, approximately 880.2 meters apart. Each took images that covered a 180-degree field of view, with a sampling rate of 15 seconds. In addition, sensors measuring ambient temperature, global horizontal irradiance (GHI), and relative humidity were placed in the same location. The LSTM model was also fed with external meteorological data collected from open sources, such as precise sky index and sun-earth distance.

“As a data set, we consider data from the 1st of August 2019 to 31 December 2019. These comprise 121 sunny days, 29 partially cloudy days, and three cloudy days,” said the research group. “To assess the effectiveness of various parameters, we conducted an extensive comparison over nine days.”

The validation set comprised different weather conditions, including sunny days, partially cloudy days, and cloudy days.

The novel method has been measured against other machine learning methods, such as Random Forest (RF) and Artificial Neural Networks (ANN) with this validation set. It was also benchmarked against the persistence model, a standard baseline model in solar irradiance forecasting, and against the state-of-the-art model SKIPP’D. They were all tested with variations of input data, with different combinations of sky images, sensor data, and meteorological data.

“The RF and ANN models achieve their best performance when using only the ‘onsite’ feature subset, indicating that they fail to capture valuable information from features extracted from images,” the scientists said. “However, LSTM outperforms them when given access to this subset. By grasping the complexity of these features, LSTM delivers the best predictions among all models tested. Furthermore, the ‘all-data’ feature set performs best for LSTM on average, under all weather conditions.”

The ramp score (RS) of the LSTM model – utilized to measure the forecasting ability of GHI fluctuations – was 39% in sunny and 25% in partially cloudy conditions. “Our benchmarking activities entailed a comparative analysis between our deep learning model and the SKIPP’D method,” the group added. “The LSTM model outperformed in validation, demonstrating superior temporal dynamics capture, crucial for solar forecasting.”

The scientists presented their model in the study “All sky imaging-based short-term solar irradiance forecasting with Long Short-Term Memory networks,” published on Solar Energy.