Houston, TX, USA
September 28 2020
We are thrilled to announce that Innowatts was the overall top performer in the Electrical Power Research Institute (EPRI)’s newest independent trial.
In the year-long trial, EPRI compared 12 short-term energy forecasting models from six leading energy forecasters. Each vendor provided forecasts against real-world data from nine commercial building sites. Across most measures, the three models provided by Innowatts consistently outperformed all other participating vendors.
We attribute this success to our modeling software, which utilizes machine learning-enhanced intelligence. Our AI-powered platform is built on diverse meter data from all over the world, allowing us to apply that depth of experience and knowledge to forecasting.
Below are highlights from the study report.
NOTE: Each participating model was assigned an anonymous ID. Innowatts’ models in this report are referred to as Magnolia, Neem, and Sequoia.
Innowatts’ Models Outperform Across The Board
Throughout the winter and summer seasons, Innowatts’ models had the lowest error rates in 21 out of 24 test categories. Across the entire study year, models “Neem” and “Magnolia” never performed lower than second or third among all participating models.
Summer 2019 – Performance comparison with the benchmark, excluding participant missing uploads. (Appendix C-9)
Winter 2019/20 – Performance comparison with the benchmark, excluding participant missing uploads. (Appendix D-7)
Innowatts also showed markedly lower error rates than other participating models across both whole-day and peak-hour forecasting.
Overall Daily Performance – Load forecasting results for the full year of forecasts.
Hashed bars include in the error calculation the participants’ missed uploads; solid bars exclude these missed uploads from the error calculation. (Figure 3-1 & 3-2)
Innowatts’ Models Display Agility During COVID-19 Shut Downs
During March and April 2020, businesses experienced unprecedented shifts in energy consumption due to nationwide “shelter-in-place” orders. During this period, the trial evaluated all vendor forecasts for six commercial sites. Two of Innowatts’ models had the lowest error for all forecast horizons. In addition, Innowatts’ models posted single-digit percentage error rates in all but three of the 24 test conditions, markedly outperforming competitors’ models.
SHA performance during the timeframe of COVID-19 shutdowns. See the report for other performance metrics during that time frame. (Appendix D-4)
AI Modeling Accuracy Far Outpaces Historical Benchmarking
All of Innowatts’ projections also outperformed EPRI’s benchmark forecast, which was generated by assuming that energy use would remain identical from one day to the next. In several cases, Innowatts forecasting errors were just half those of the benchmark figures.
Innowatts also outperformed the other six models against the benchmark, showing the lowest error across the full range of forecasting scenarios once missing uploads were excluded from error calculations.
Load forecasting results relative to the EPRI benchmark for the full year of forecasts. Positive values (greyed area) mean participant performance was better than the benchmark. (Figure 3-3)
About The Study Methodology
The objective of EPRI’s research was to evaluate the effectiveness of existing software products for customer-level, behind-the-meter load forecasting. To that end, continuous load data from nine commercial customer sites were provided to six load forecasting vendors. Forecast participants in turn provided customer-level load forecasts for four different horizons, and their accuracies were evaluated.
In addition to the continuous load data, vendors were also provided with at least one year of historical load data for each of the nine commercial sites, as well as the location (zip code), the type of commercial enterprise it was, and any rate information known about each site.
12 forecast “participants” were part of the trial, representing the six vendors, nine of which participated for the full year of the trial. Although the research team (EPRI, Southern Company, and PowerSecure) knew the identity of each of the six vendors, they did not know which vendor mapped to which of the 12 participants.
EPRI developed a simple benchmark forecast to provide a basis for comparison. The benchmark applied a persistent forecast approach—it used the most recently measured day of data as the forecast of the future periods. In other words, for the Hour Ahead forecast it assumed “the load in the next hour today will be the same as it was for that hour was yesterday.” Likewise, for the Four Day Ahead forecast, it assumed “the load for each hour four days from now the same as it was for each hour yesterday.”
You can find more information about the study methodology in the report, here: https://www.epri.com/research/products/000000003002019632