A hackers guide to AEMO & NEM data

2 minute read

This is a short guide to Australian electricity grid data supplied by AEMO (the market operator) for the Australian National Electricity Market (NEM) (the grid in Queensland, New South Wales, Victoria, South Australia, and Tasmania).

Data

Information about the participants in the NEM is given in the NEM Registration and Exemption List. The carbon intensities for generators are given in the Available Generators CDEII file.

Data about the NEM is provided in two sources:

The NEMDE provides infomation about the price setter (aka the marginal generator) in the NemPriceSetter XML files.

The MMSDM provides both actual data and forecasts for a range of variables - including prices, demand and electricity flows. Data in the MMSDM is supplied from three different, overlapping sources:

  • CURRENT - last 24 hours
  • ARCHIVE - last 13 months
  • MMSDM - from 2009 until the end of last month

Some report names can be different across sources - for example DISPATCH_SCADA versus UNIT_SCADA.

Price structure

The wholesale electricity price is known as the trading price - a half hourly price for electricity. The trading price is the average of the six dispatch prices that occur within a half hour - the dispatch price is a 5 minute price for electricity.

The trading price is used for settlement - it is the price that matters. AEMO plan to move to 5 minute pricing in the future.

AEMO timestamping

AEMO timestamp with the time at the end of the interval. This means that 01/01/2018 14:00 refers to the time period 01/01/2018 13:30 - 01/01/2018 14:00.

Personally I shift the AEMO time stamp backwards by one step of the index frequency (i.e. 5 minutes). This allows the following to be true

dispatch_prices.loc['01/01/2018 13:30': '01/01/2018 14:00'].mean() == trading_price.loc['01/01/2018 13:30']

The shifting also allows easier alignment with external data sources such as weather, which is usually stamped with the timestamp at the beginning of the interval.

If the AEMO timestamp is not shifted, then the following is true

dispatch_prices.loc['01/01/2018 13:35': '01/01/2018 14:05'].mean() == trading_price.loc['01/01/2018 14:00']

Useful reports

The MMSDM links are for the reports linked below are all for 2018_05.

actual data

  • trading price (30 min electricity price) - TRADINGPRICE - MMSDM
  • dispatch price (5 min electricity price) - DISPATCHPRICE - MMSDM
  • generation of market participants - UNIT_SCADA - MMSDM
  • market participant bid volumes - BIDPEROFFER - MMSDM
  • market participant bid prices - BIDAYOFFER - MMSDM
  • demand - DISPATCHREGIONSUM - MMSDM
  • interconnectors - INTERCONNECTORRES - MMSDM

forecasts

  • trading price forecast - MMSDM
  • dispatch price forecast - MMSDM

Ecosystem

A major benefit of the large AEMO dataset is the ecosystem of third parties who can build useful (and often open source) tools on top of it.

AEMO dashboard - interative map

electricityMap

AREMI

nemlog

opennem - github

nemsight

gas & coal watch

Further reading

Thanks for reading!

Updated: