Uncertainty Flagging Analysis
The most commonly used NWP model for power production forecasting is ECMWF. However, if the consensus of other models deviates beyond a certain threshold from ECMWF, it often signals a potential error in the ECMWF forecast. Our uncertainty flagging analysis showed, that in over two-thirds of cases, when the consensus forecast is higher or lower than ECMWF, actual observations tend to follow that trend.
Introduction to Uncertainty Flagging
It is well known that uncertainties in wind power forecasts are related to several different sources of error. The four main causes of these errors are:
Reliable Numerical Weather Prediction (NWP) models are crucial for accurate production forecasting. These models provide surprisingly accurate weather data for short-term forecasts. However, there are limitations to how detailed these advanced models can be, and very small-scale events (minutes and <1 km) are not perfectly modeled.
In the power market, the most challenging parameters to predict accurately are wind speed and solar radiation. To address these challenges, a combination of different weather models is key. Currently, the ECMWF is considered the best NWP model, on which the energy market closely relies. However, any errors or significant changes in this model can lead to high volatility, affecting both the physical and financial energy markets.
At ConWX, we combine different weather models and use the best ones available for day-ahead and intraday markets. Our weather model mix typically includes 4-5 NWPs, including 2 in-house models.
In one analysis, for a small continental portfolio of 50 MW, we found hundreds of instances over a year where our forecasts deviated by more than 10% from the best-in-class predictions, highlight periods of potential over- or underestimation of next-day power production.
These uncertainties are not evenly distributed throughout the year but occur in clusters depending on various weather regimes. The questions are: what should be done in these situations, and how accurate are these flags?
Analysis
In 2023, we analyzed 40 wind farms with a total capacity of more than 1,4 GW. Using a threshold of 25% potential error on the forecast, we evaluated both the risk of over- and under-predictions of this portfolio and compared the analysis with realized production data.
Our uncertainty flagging for overprediction was successful 73% of the time, and for underprediction, the rate was 71%. Nearly three out of four times, the trades would have been successful by following our flags.
In the figure below, you can see potential overprediction from July 2023, where the ECMWF power model was out of sync with the realized production data.
In 2024, we analyzed 11 solar parks located in Balkan countries. Using a threshold of a 10% potential error on the forecast, we evaluated both the risk of overprediction and underprediction of this portfolio and compared the analysis with realized production data.
Our uncertainty flagging for overprediction was successful 66% of the time, and for underprediction, the rate was 71%.
