Designing the grid of the future requires understanding the grid of the present — not just for utilities, but for technology providers as well.

An effort in New York shows how difficult that can be.

Fundamental to optimizing DERs like rooftop solar and behind-the-meter storage is a common understanding of the distribution system. Currently, only utilities have full access to the data needed to fully understand the system’s limits and potential, and even they often lack visibility to understand exactly where all their assets are located.

But New York’s Reforming the Energy Vision (REV) docket includes an effort to enhance and spread the knowledge of the utility grid, part of its distribution system planning proceeding. If stakeholders there can agree on a common approach, it could become a model to enhance grid planning and DER collaboration across the country.

“The sharing of distribution-level data by New York utilities can fundamentally change the way utilities and third-parties operate not just in New York, but throughout the whole country,” said Jon Wellinghoff, former chair of the Federal Energy Regulatory Commission and chief policy officer for SolarCity. Data sharing by utilities will “maximize meaningful stakeholder engagement” and allow DER providers like SolarCity to offer “market-based solutions.”

By collecting and sharing system data, utilities can indicate to DER providers where their technologies can add the most value to the grid and displace expensive traditional power system upgrades. DER providers, meanwhile, can access a larger consumer market and work with utilities to enhance consumer choice.

In New York, the end goal is to amass enough accurate system data to allow real-time transactive energy marketplaces on the distribution system — similar to larger wholesale market constructions. But the effort is still in its early stages, and both utilities and DER providers say there are a number of hurdles for regulators to address before the vision becomes a reality.

Data sharing in the REV docket

An update on data sharing was just filed by New York’s utilities, participating as the Joint Utilities (JU), in their “Supplemental Distributed System Implementation Plan.”

The JU is made up of Central Hudson Gas & Electric, Consolidated Edison (ConEd), New York State Electric & Gas, National Grid New York, Orange and Rockland Utilities, and Rochester Gas and Electric. The distribution plan was mandated by New York regulators in the REV proceeding to move the traditional utility business model to that of a distribution system platform provider — an impartial integrator that facilitates the interconnection and operation of DERs on its grid.

“We are working on where this will go,” said John Borchert, director of energy policy at Central Hudson Gas and Electric. “We are taking another look at the data providers want and how we are providing it and the needs and access to data that is available.”

The JU filing details a three-stage process now unfolding.

In the current Stage 1, utilities are upgrading New York’s grid and beginning to incorporate the advanced technologies needed “to enable more robust integration of DER,” including smart inverters, grid sensors and distribution system software. For the first time, system data needed by DER providers to make competitive bids in utility solicitations for grid upgrades is being made available by the utilities.

In Stage 2, that data will enable advanced functions and capabilities to allow “an operational marketplace based on transactions between utilities and DER providers.” DER will be valued in “utility pricing, programs, and procurement” that opens DER market participation.

That stage will require a standardized procurement process possible only if DER providers have access to complete system data, said Ryan Katofsky, vice president of industry analysis at Advanced Energy Economy, a clean energy trade group and stakeholder in the DSIP proceeding.

“Opening up data for all 8760 hours of operation across the entire network will allow providers to develop innovative products and grid services,” he said.

In Stage 3, high DER penetrations will lead to a transactive marketplace. That could be as much as five years out, according to the filing.

Complete system data and much more precise price data will be necessary to allow real time market opportunities for DERs, Katofsky said. It could come through wholesale markets or DSPs and it could be through solicitations or new tariffs with locational and temporal value price signals.

“It is not yet clear how specific it will be but stakeholders want more granular data and values,” Katofsky said.

What data, exactly?

Transforming utilities into distribution platform providers requires the development of new functions and capabilities, the Joint Utilities filing reports. Each utility has provided “a roadmap for technology investments to improve grid intelligence and prepare it for higher DER penetrations, and provided data to bring greater transparency to the planning process and support distribution market development.”

A high-DER future will require utilities to have standards through which utilities can compare traditional grid upgrades like lines and substations to non-wire alternatives (NWAs) like rooftop solar, storage or demand-side management. To do this without compromising reliability and affordability, new tools and processes for data access, will be initiated by the end of 2018.

For grid operations, complete and publicly available system data will help utilities incorporate DERs in “managing real and reactive power in normal, outage, and emergency conditions, maintaining distribution equipment, providing power quality, and improving operational efficiency,” the JU filing reports.

And to incorporate DERs into market operations, accessing and assimilating real-time pricing data through procurements, markets, or dynamic tariffs will also necessary. A key role for DERs is expected to be as non-wire alternatives, and more than 35 NWA opportunities have already been identified, according to the JU filing.

The data utilities are collecting to support these aims fall into two categories.

One is system data, which describes grid performance at the system, substation, or feeder level, the JU filing reports. The other category is customer energy usage data, which can be about individual or aggregate usage.

Utilities already make system data — including capital investment plans, load forecasts, reliability statistics, and planned reliability and resiliency projects — public in regulatory filings, according to the JU document. Customer data is available, with authorization, through utility bills, through Green Button Connect My Data and other online tools, and through Electronic Data Interchange and similar online platforms.

Collecting and sharing data could also be a source of new revenue for utilities, the JU filing observes.

“Basic data” about loads, circuits, and power quality would be available at no cost. But regulators could choose to allow utilities to require remuneration for a data analysis service that makes granular and customized "value-added" data, like aggregated customer usage information, available to DER providers.

Regulators indicated their desire for utilities to monetize some system information services last year, opening up new revenue opportunities to utilities in a May order in the REV docket.

Hosting Capacity Analysis — the next frontier

Consolidated Edison has so far set the bar data transparency in New York by making its substation forecasts and its historical hourly substation data accessible online, Wellinghoff said.

The five other utilities are expected to make 90% of their system data public within five years.

There is a significant difference between the data made available by ConEd and the other Joint Utilities.

“Con Edison has provided 24-hour peak load duration forecasts and 24-hour minimum load historical curves,” as well as underlying historical hourly load data, the JU filing reports. It has also provided network-level hosting capacity maps “where sufficient minimum load exists to enable DER interconnection at little to no additional cost.” Going forward, the utility plans to make feeder hosting capacity maps available.

The Hosting Capacity Analysis that underlies the maps is “an assessment of how the power flows from the point of interconnection onto the system and the key categories of impacts,” said Sky Stanfield, attorney for the Interstate Renewable Energy Council (IREC) in the REV docket. “[It] shows how what is on the system interacts with everything else on the system and determines the amount of distributed resources that can be added without causing a problem. That benefits the utility, the provider, and the customer.”

IREC helped develop the concept of integrated distribution system planning as part of early efforts to streamline solar interconnections, according to Stanfield. To complete that vision, what DER providers need —what utilities are only beginning to share — is circuit-level hosting capacity data and the underlying information to work with it, Stanfield said.

That’s where the HCA comes in, assimilating the megabytes of load data utilities have into the actionable information DER providers need.

“California is already working to deliver that information in the form of system maps,” she said.

In New York, ConEd is doing the same, while other utilities are still working to build their capabilities.

Central Hudson has made hour-by-hour historic and forecasted substation load data available where it has assimilated and processed it, Borchert said. Capturing and refining it is a labor-intensive task but the utility recognizes the importance of doing it.

For the other utilities in the JU, analysis of at least 50% of circuits will be completed by the end of 2017 and full-circuit analysis will be completed by all utilities by mid-2018, the filing reports.

They also expect to be able to make the majority of their substation data publicly available within the next five years, according to Wellinghoff.

But while DER providers are pleased New York utilities are working on HCA, some stakeholders have misgivings.

“The New York utilities’ proposed methodology is of questionable accuracy and would make the data not useful,” Stanfield said. “It is critical that the [HCA] analysis be done accurately and transparently.”

In Stage 2, the utilities will provide more complete hosting capacity data using the Distribution Resource Integration and Value Estimation (DRIVE) methodology from the Electric Power Research Institute (EPRI), the JU filing reports. It “leverages existing circuit models in a utility’s native distribution planning software to carry out a streamlined analysis of hosting capacity.”

IREC’s filing argues utilities should find a better methodology to accomplish the DSIP goals. It should enable production and sharing of HCA-level data for interconnecting and optimizing DER and guide distribution planning.

The EPRI DRIVE methodology has “potential flaws” the utilities do not acknowledge, IREC argues. It is not clear what “use cases” its “streamlined” approach satisfies or what outputs it will deliver.

EPRI VP Mark McGranaghan said IREC's concerns seem more informed by the utilities' inadequately transparent data than by any DRIVE methodology shortcomings.    

A comparison of HCA methodologies performed in a similar California Public Utilities Commission (CPUC) proceeding concluded a streamlined method “has significant accuracy issues” and developers “would have a difficult time relying on the information provided in hosting capacity maps,” according to the IREC filing.

​The alternate “iterative” methodology would be “significantly more accurate” but its complexity raises “practical concerns,” the CPUC found. “Some combination of the two methodologies may be the best way to proceed.”

IREC describes EPRI's DRIVE as a "streamlined" methodology similar to the one being evaluated in California. McGranaghan insisted it is like neither the "streamlined" or "interative" California methodologies. It is a more detailed, non-iterative alternative to both, he told Utility Dive.    

IREC calls for an HCA in New York that optimizes all DER and not just distributed solar. It should be updated at least monthly, and underlying data should be accessible and downloadable.

“If the utilities choose a tool that does not ultimately serve this full suite of needs, it will be extremely costly to ratepayers to re-route at a later date,” IREC adds.

McGranaghan insisted EPRI's DRIVE is adequate to the needs of both the New York utilities and stakeholders if it is called on to meet them.

Other barriers to data sharing

The benefits of making system data more transparent are clear and stakeholders continue to demand it. But Central Hudson has not yet had requests for its load data, Borchert said. Nevertheless, each utility has disclosed its available basic data and more will be forthcoming, once utilities’ privacy issues are resolved.

Borchert described two privacy concerns. One is granular load data. It could include usage information about one or two circuits that feed only a single large customer, essentially profiling the customer, he said. “Making that data available without customer authorization violates our privacy commitment unless it is aggregated enough to mask what is commercially sensitive.”

Another privacy concern is sufficiently detailed substation, feeder, and infrastructure data for airports or other critical infrastructure made available through HCA maps, he said. That data, if used improperly, could put those facilities at some risk.

As some DER advocates have argued, much of that information is already publicly available to people who know how to search it out, Borchert acknowledged. “But unless it is mapped out in the detail only utilities have access to, it does not offer enough information to compromise critical infrastructure.”

IREC’s Stanfield described other barriers to data sharing.

“It requires utilities to gather, organize, and clean up their data, and figure out how to apply the methodology,” she said. “It is a lot of work, it is complicated, and it takes time, even if the utility dedicates enough resources to get it done.”

The hope is that the time and costs invested in the upfront analysis of a utility system’s hosting capacity will more than pay off in streamlined processing of DERs over the long term, Stanfield said. That would benefit the DER provider, the utility, and its customers.

It would also make the transition to a DSP platform more efficient by allowing utilities to enter that role with a detailed sense of how to optimize their systems, she added.

A more challenging barrier might be the fundamental competitive struggle between utilities and DER providers inherent in the utility business model, Stanfield said. It is contradictory to a utility’s basic incentive structure to streamline the building of NWAs that incorporate DER because the utility's returns come from capital expenditures for infrastructure.

“That perceived competitive relationship creates a resistance to providing information,” she said.

The REV docket aims to change that by allowing utilities to earn extra revenue for meeting system needs with DERs, rather than traditional investments. And in the longer term, utilities also aim to drum up more revenue from marketing their system data.

“Basic data should be available to all providers,” Borchert said. “But utilities might legitimately earn a return for costs incurred in producing and cleaning up data that delivers value to providers.”

Much work remains to understand its potential impacts on utilities and providers, determine how the data might be used and how available it is, and the costs, benefits, and appropriate fee structures, Borchert said.

“If it is has significantly more value to the third parties than to the utility, it might be marketable,” he said. “It might be done through an unregulated arm of the utility or the commission could decide it is an appropriate revenue stream for regulated utilities.”

When will the future arrive?

The initial and supplemental distribution system implementation plans “accelerate Stage 1 progress and move the utilities deeper into Stage 2 over the next five years,” the JU filing reports. That “will establish a foundation for moving into Stage 3.”

At present, it adds, the utilities are in “the early stages of what will be an iterative process.”

AEE’s Katofsky agrees the current proceeding is “pretty close to the beginning of this evolution.”

Now supported largely by federal and state policies, DER cannot fully mature until their services are properly valued by markets, he said. That will not happen until their unique locational and temporal values can be monetized and those values will only be clearly revealed by HCAs.

“By making the data available, third parties may develop and deliver ‘killer apps’ in the form of products and services that have value nobody knew was there,” Katofsky said. “It is important to enable that innovation.”

Though the utilities are moving slowly to protect reliability, privacy, and security, they are moving in the right direction, he added. “The DSIP is updated every two years and that is the cadence of this part of the REV, so this will take years, not months.”

As for the long-term goal of transactive energy markets, most stakeholders think that future is at least a decade away. But at least one important figure is more bullish.

Last week, outgoing New York PSC Chair Audrey Zibelman told Utility Dive she expects transactive energy markets to emerge in the state in the next five to ten years. Though she’s leaving REV to run Australia’s grid operator, she said the evolution started by the regulatory docket could progress faster than many think is possible.

“I think we're going to see these markets continue to develop and I think anyone who thinks in decades anymore is not looking at the rest of the world and how quickly things are changing,” she said.