A hackers guide to AEMO & NEM data
This is a short guide to Australian electricity grid data supplied by AEMO (the market operator) for the Australian National Electricity Market (NEM) (the grid in Queensland, New South Wales, Victoria, South Australia, and Tasmania).
Data
Information about the participants in the NEM is given in the NEM Registration and Exemption List. The carbon intensities for generators are given in the Available Generators CDEII file.
Data about the NEM is provided in two sources:
The NEMDE provides infomation about the price setter (aka the marginal generator) in the NemPriceSetter XML files.
The MMSDM provides both actual data and forecasts for a range of variables - including prices, demand and electricity flows. Data in the MMSDM is supplied from three different, overlapping sources:
Some report names can be different across sources - for example DISPATCH_SCADA versus UNIT_SCADA.
Price structure
The wholesale electricity price is known as the trading price - a half hourly price for electricity. The trading price is the average of the six dispatch prices that occur within a half hour - the dispatch price is a 5 minute price for electricity.
The trading price is used for settlement - it is the price that matters. AEMO plan to move to 5 minute pricing in the future.
AEMO timestamping
AEMO timestamp with the time at the end of the interval. This means that 01/01/2018 14:00 refers to the time period 01/01/2018 13:30 - 01/01/2018 14:00.
Personally I shift the AEMO time stamp backwards by one step of the index frequency (i.e. 5 minutes). This allows the following to be true
dispatch_prices.loc['01/01/2018 13:30': '01/01/2018 14:00'].mean() == trading_price.loc['01/01/2018 13:30']
The shifting also allows easier alignment with external data sources such as weather, which is usually stamped with the timestamp at the beginning of the interval.
If the AEMO timestamp is not shifted, then the following is true
dispatch_prices.loc['01/01/2018 13:35': '01/01/2018 14:05'].mean() == trading_price.loc['01/01/2018 14:00']
Useful reports
The MMSDM links are for the reports linked below are all for 2018_05.
actual data
- trading price (30 min electricity price) - TRADINGPRICE - MMSDM
- dispatch price (5 min electricity price) - DISPATCHPRICE - MMSDM
- generation of market participants - UNIT_SCADA - MMSDM
- market participant bid volumes - BIDPEROFFER - MMSDM
- market participant bid prices - BIDAYOFFER - MMSDM
- demand - DISPATCHREGIONSUM - MMSDM
- interconnectors - INTERCONNECTORRES - MMSDM
forecasts
Ecosystem
A major benefit of the large AEMO dataset is the ecosystem of third parties who can build useful (and often open source) tools on top of it.
AEMO dashboard - interative map
electricityMap
AREMI
nemlog
opennem - github
nemsight
gas & coal watch
Further reading
- NEM on the AEMO website
- Winds of change: An analysis of recent changes in the South Australian electricity market - University of Melbourne
- Li, Zili (2016) Topics in deregulated electricity markets. PhD thesis, Queensland University of Technology
- Dungey et. al (2018) Strategic Bidding of Electric Power Generating Companies: Evidence from the Australian National Energy Market
Thanks for reading!