Today we are pleased to publish a major report[i] the Australian Energy Council commissioned into the question of Scheduling and Ahead Markets, prepared by the highly experienced National Electricity Market (NEM) economists, David Smith of Creative Energy Consulting and Veronika Nemes of MarketWise Solutions.
This is a matter being deeply investigated by the Energy Security Board (ESB) as part of its “Post 2025 NEM review”[ii] as it has been advocated in many Australian Energy Market Operator (AEMO) technical reports[iii]. In fact, the March COAG Energy Council meeting[iv] went as far as directing the ESB to develop “a new framework for … ahead market arrangements” for the NEM.
But this was a very open direction with an infinite range of possible interpretations. Australian Energy Council (AEC) members were unsurprisingly keen to receive expert advice on:
These are the questions we put to our advisers.
We briefly discuss the report below, however we urge readers to invest the time to give the report full justice, which is a deeply engaging and enlightening read.
How the NEM works now
Much of Creative Energy’s report is dedicated to filling a gap in the material published to date promoting ahead markets, by explaining the NEM’s existing arrangements. The NEM has an elegantly simple scheduling and settlement process:
For those only experienced in the NEM, this scheduling and dispatch process is natural and intuitive. However many other electricity markets, particularly in the US, employ much more complex approaches using multiple settlement prices and a more direct role for the market operator in generators’ operations. Academics refer to the NEM as a “Single-pass/self-commitment” market, and its alternatives as forms of “Day-ahead and balancing/central-commitment” markets.
The NEM’s rather simple decentralised process remarkably produces a near-optimal solution to the incredibly complex problem of scheduling a big power system. The report describes this magic as a “Mega-Algorithm” but might also be considered an example of Adam Smith’s famous “invisible hand”. And, just like many countries in the twentieth century found when they tried it in their economies, attempts to replace the invisible hand with a central one can prove intractable.
Source: Creative Energy Consulting Report
What is it?
The consultants found our request to clarify the proposal perhaps the most challenging part of the project. Many forms of “aheadness” has been discussed and a recent ESB paper presented a continuum of options between what is little more than a small procedural tweak, through to very fundamental changes in the way the market is priced, hedged and dispatched.
At the minimalist end, it involves AEMO simply applying a more methodical approach to the circumstances where it needs to intervene in the market. This is described as “Unit Commitment for Security” (UCS). The report supports this concept and suggests it can be implemented immediately within the existing rules.
Half-way into the spectrum, there are options that are purely financial, and may be optional or mandatory. For example, it could involve a centralised clearing of willing buyers and sellers of short-term derivative instruments a few hours ahead of dispatch.
At the maximalist extreme, two fundamental changes are in play:
Source: Creative Energy Consulting Report
The rationale supporting the need for an ahead market is, as you may have guessed, the dramatically changing power system with its preponderance of variable, asynchronous technologies, and greater demand response and distributed resources. These are dramatic changes indeed, but the report does not find why they should lead to the maximalist options, and indeed the contrary may be the case.
Cited as evidence of a problem in the current approach are the numerous market interventions in the NEM in the last two years, almost all associated with the need to recruit system strength in South Australia. The report finds a very simple explanation – this is a service that the power system needs, but for which there is presently no market reward, so this outcome is entirely predictable. And, the intuitive solution - to create a market for it within the current scheduling arrangements - is also Creative Energy’s recommendation.
As it happens, these recent interventions will soon become moot, as monopoly network equipment is being installed to supply South Australian system strength from non-generation sources.
What would have to change
The implementation challenge is very much dependent on what solution on the minimalist – maximalist spectrum we are contemplating. The UCS described above seems to have near-zero cost and impact. Half-way approaches that are purely financial and optional have some costs, but if optional, participants themselves can decide whether or not they want to invest in order to participate in the new arrangements.
At the maximalist end of mandatory physical participation and multiple settlement passes, the impacts become profound. The dispatch process itself must fundamentally change, and the NEM would need to break new ground, because the US markets apply it with nodal pricing, and their systems are well behind Australia in the technology transition. Their central-commitment mechanisms have proved tractable as they operate largely within traditional, well-understood generation technology, but will soon face great challenges in centrally committing exotic technologies.
The proposal anticipates costs of startup and minimum load being paid at actual costs rather than a competitive bid price, it introduces a quite different paradigm for the way NEM competitive generators reveal costs, and confuses the incentive to maximise flexibility – something very much needed for the transition to renewables.
Retailers will need to provide their own customers’ forecasts to AEMO and will be held responsible for their accuracy with errors charged in the balancing market.
But perhaps the biggest impact is on the way markets are hedged. With multiple settlement passes and new costs associated with startups, participants will need to think differently as to how they structure their forward contracting. Exactly which of these many volatile settlement streams will they want to stabilise, who will be the counterparty, and how will they adapt their existing contract portfolios to the new world? This is an area which has yet to be considered in the ESB’s work.
Creative Energy’s report does not suggest the challenges of the transforming NEM will be easy to navigate with the existing dispatch and settlement processes. It concludes however, that the problems sometimes presented as justification of the more maximalist versions of ahead markets are more appropriately, and at much less risk and cost, resolved in more intuitive ways.
As discussed above, the most obvious solution is to create ancillary services arrangements for all services that the power system is likely to need from the competitive market, but does not presently price. This is thankfully already underway with a series of rule changes being worked on by the AEMC.[vi]
Where, despite these markets, AEMO still needs to intervene, then the UCS process would provide a more mechanical and repeatable way for this to occur at least cost than the current manual approach. Also, by incorporating the intervention itself into pre-dispatch (the report cheekily describes this as “applying good faith bidding rules to AEMO”) then the market will be able to optimise its decentralised commitment process around it.
The report provides a table of its main findings in the executive summary which is reproduced below.
[i] Creative Energy Consulting, Scheduling and Ahead Markets Design Options for post-2025 NEM, June 2020
[iii] See for example AEMO’s Renewable Integration Study: Stage 1 Report, April 2020 Action 2.3 p.8
[iv] Communique of 23rd COAG Energy Council Meeting, available at http://www.coagenergycouncil.gov.au/publications/23rd-energy-council-meeting-communiqu%C3%A9
[v] 80 half hour plus 12 5 minute predispatch forecasts are published ahead of a real-time dispatch interval.
AEMO received headlines with its goal of delivering capability for the grid to manage 100 per cent renewables at any moment by 2025. The announcement was generally well-received but, unfortunately, misreported as targeting the running of the NEM entirely on renewables. Industry participants may ask why AEMO would be indicating a 100 per cent renewables capability by 2025, which is not to say that progress towards a lower emissions market isn’t welcome.
If there's one thing the Australian Energy Regulator’s State of the Energy Market report has reminded us, it’s that the National Electricity Market can be a wild ride at times. After the NEM achieved the lowest first quarter for wholesale prices since 2012, the last few months have seen an increase in price volatility.
The first quarter of 2021 for the NEM's spot market was notable for its lack of notability. But in a measure of the how the NEM can turn, the Q2 Queensland average price to date is $135.83/MWh, versus $42.64/MWh over Q1. We look at how the market almost always successfully responds to the NEM’s variations, and its reliance on hydro and gas-fired generation to keep the lights on.
Send an email with your question or comment, and include your name and a short message and we'll get back to you shortly.