New disaggregation methodology quantifies uncertainty in behind-the-meter PV generation

The rapid adoption of behind-the-meter (BTM) distributed energy resources, such as photovoltaic (PV) systems and heat pumps (HPs), is creating new challenges for distribution system operators (DSOs). Since these assets are located behind customer meters, their individual generation and consumption profiles are not directly observable from conventional smart meter measurements. This lack of visibility limits DSOs’ ability to accurately assess network conditions, forecast demand, and manage emerging flexibility opportunities.

Addressing this challenge, a research team from Spain’s Universitat Politècnica de Catalunya (UPC) has introduced a generative diffusion-based methodology for probabilistic BTM PV disaggregation. The method extracts the probabilistic characteristics of PV generation from low-resolution smart meter data, enabling reconstruction of solar production profiles from measurements available at 30-minute intervals.

“Our work started from a real request,” corresponding author Marc Jené Vinuesa told pv magazine. “A Spanish DSO asked us for a service to estimate voltage volatility on their network, but it had no knowledge of which PV systems were installed behind the meter in their grid, which is exactly the input such a service needs. That gap is what the method addresses: the DSO first gains visibility of what is installed, and from there, several data-driven services become possible. We like to say the method serves as a service enabler.”

Dubbed conditional denoising diffusion probabilistic model (cDDPM), the proposed method learns the distribution of daily PV generation profiles using low-resolution smart meter data, net consumption, and local irradiance information. During inference, the model generates multiple plausible PV profiles, enabling probabilistic estimates with uncertainty intervals. These generated samples enable both point estimates and uncertainty quantification through probabilistic intervals.

The model is conditioned on available information such as net consumption, irradiance, and location-related features to improve reconstruction accuracy. During inference, the diffusion model generates PV profiles by progressively denoising random noise while preserving temporal characteristics of daily generation patterns. A 1D U-Net architecture is employed as the noise prediction network, incorporating time embeddings and conditioning signals to capture complex temporal dependencies.

The new method was trained and evaluated using two real-world residential datasets from Australia and the Netherlands. The Australian dataset includes 257 PV-equipped households with one year of half-hourly smart meter and PV generation data, while the Dutch dataset is used to assess cross-region generalization. The model performance is evaluated using both deterministic metrics, such as RMSE, MASE, MAPE, and SMAPE, and probabilistic metrics, including prediction interval coverage, sharpness, and quantile accuracy.

Several baseline methods are considered, including physics-based, proxy-based, ensemble, quantile regression, and conditional variational autoencoder approaches. The proposed cDDPM model is trained using a 1D U-Net architecture and optimized through extensive hyperparameter tuning. Experiments are conducted on high-performance computing infrastructure to ensure reliable training and evaluation. The methodology is tested across multiple train–test splits to assess robustness and seasonal variability. The evaluation focuses on both the accuracy of PV reconstruction and the reliability of uncertainty estimates. Overall, the experimental framework provides a comprehensive comparison between the diffusion-based approach and existing deterministic and probabilistic disaggregation techniques.

The model performance is evaluated using both deterministic metrics, such as Root Mean Square Error (RMSE), Mean Absolute Scaled Error (MASE), Mean Absolute Percentage Error (MAPE), and Symmetric Mean Absolute Percentage Error (SMAPE), as well as probabilistic metrics, including Prediction Interval Coverage Probability (PICP), Prediction Interval Normalized Average Width (PINAW) for uncertainty sharpness, and Pinball Loss (PL) for quantile accuracy.

Experiments were conducted on high-performance computing infrastructure to ensure reliable training and evaluation, with the methodology being tested across multiple train–test splits to assess robustness and seasonal variability.

The analysis showed that the cDDPM offers superior performance for behind-the-meter (BTM) PV disaggregation compared with deterministic and probabilistic benchmark methods. Calibration analysis confirmed that the uncertainty estimates remain reliable, particularly during periods of active PV generation. Further analysis showed that combining irradiance with current and previous-day net consumption improves both accuracy and uncertainty estimation.

Vinuesa explained that the model could also be used for PV-driven homes including batteries and electric vehicles (EVs). “Accurate PV disaggregation in PV-plus-battery homes would require training the model with representative datasets that include battery-equipped households,” he went on to say. “Our previous work demonstrated that conditioning the model on irradiance alone can still achieve competitive PV disaggregation performance, which may become particularly relevant in scenarios where battery operation obscures the relationship between PV generation and net consumption.”

He also stated that, compared with batteries, EV charging is expected to have a smaller impact on PV disaggregation because most residential charging occurs during evening and nighttime periods, when PV generation is negligible. “However, the interaction becomes more complex as smart charging strategies increasingly align EV charging with periods of high solar availability,” he added. “In such cases, additional information, such as electricity prices, charging behavior, or user preferences, may be required. Furthermore, if the objective shifts from PV disaggregation alone toward the simultaneous identification of multiple flexible resources—including EV consumption, battery charging/discharging, and heat pump operation—the model would need to incorporate additional exogenous variables and be trained on more diverse multi-resource datasets.”

The new methodology was presented in “Conditional diffusion modeling for probabilistic behind-the-meter PV disaggregation,” published in Energy and AI.