Making the economics of worker displacement work for everyone

Below I model the economics of a worker displacement project that works for everyone. To combat inequality, the business and employee both share the project saving. I will show that by sharing the project saving both the business and former employee can end up with acceptable outcomes.

By worker displacement projects I mean any project where the employee loses his job. Automation and artificial intelligence projects are both worker displacement projects.

The model below is simple – this is a good thing.  A model like this is designed to provoke thought and hope in the reader.

I first assume an annual savings breakdown of the worker displacement project. We are displacing an employee that costs the business $40k. This includes tax that is not paid to the employee and any marginal expenses associated with employment.

I then assume a small maintenance cost increase for the business and a saving from efficiency improvements. All three of these net out at an annual saving of $50k for the business from this project.

If we decide to share this saving 50/50, the business ends up only saving $25k. This will double the project payback period.  As we expect automation or AI projects to have decent paybacks (i.e. 2 years or less) we would expect the new payback period to be at most 4 years.  This is still likely to be an acceptable use of capital. It depends on variables such as interest rates and alternative projects the company could finance.

The net financial impact for the employee is more complex than just the lost wages. We would also expect a small decrease in expenses that occur related to work. Our employee also receives a share of the saving from his old employer.

The net result is no financial impact for the employee from being displaced by a machine. The business is left with a project that while not as attractive as it could be, is still acceptable for many business as a use of capital.  Both sides end up with acceptable outcomes.

A key assumption here is the breakdown of the project savings. Technology will improve the ratio of maintenance costs to efficiency improvements. Efficiency improvements should increase as the projects enable more machine intelligence (rather than pure automation based on human heuristics).

We could also see reductions in machine maintenance costs. Alpha Go Zero showed an impressive decrease in computation costs over it’s previous iteration. It would be reasonable to expect that machine O&M costs will decrease over time.

The point of this analysis is not to show exactly zero net impact. It could be possible that the employee would need to accept a small decrease in net income.  Any impact needs to be offset against the non-financially quantifiable benefits and drawbacks that also occur when a worker is displaced.

It’s not clear whether the non-financial impacts would be be net positive or negative. Having more choice over how you spend your time might be offset by the lack of intellectual or social stimulation we get from our work today.

The specific mechanism for value sharing requires though.  The real mechanism for sharing the saving will be complex to implement in the real world. One mechanism would be a universal basic income funded by taxes on projects that displace workers.  This would most likely be a tax on the capital cost, as quantifying savings would be more challenging.

What I am trying to show is that it is possible to share value, rather than default to the business taking all of the value of the project and leaving the employee without any significant source of income.

The capitalist default of today is not acceptable due to the inequality it creates.

We must share the benefit of automation and machine intelligence throughout society.  The key to doing this is to balance between an acceptable return on capital on the business side with quality of life of society.

You can download the Excel spreadsheet here.

Demonetizing Everything: A Post Capitalism World | Peter Diamandis – Energy Insights

Energy Insights highlights interesting energy content from around the web.

Previous posts include Getting Wind and Sun onto the Grid and The Complexity of a Zero Carbon Grid.

This post looks at Peter Diamandis’ talk Demonetizing Everything: A Post Capitalism World. The central premise of the talk is demonitization – technology is making utility cheaper.

Diamandis’ highlights demateralization as one driving force behind demonitization. Put simply – technology allows us to use less stuff to deliver more utility, making that utility cheaper.

Diamandis gives a great example of this demateralization leading to demonitization trend using the smart phone. Diamandis estimates that the functionality of a $50 smart phone of today would have cost millions 20 years ago. This is a direct result of the demateralization of functionality from hardware to software.

Yet when Diamandis got to his section on energy I was left quite frustrated. It’s not that I don’t agree with the central premise that demateralization doesn’t lead to demonitization. It does. I’m disagreeing that demateralization is occurring in our transition to renewables.

Because the energy density of renewable resources are so much less than fossil fuels, we actually require more steel, concrete and plastic per unit energy generated from wind & solar.

Table 1 – Range of materials requirements (fuel excluded) for various electricity generation technologies (DoE Quadrennial Technology Review 2015)

I also found talk too positive – Diamandis makes it sound like everything is OK, coal has been defeated and it’s all smooth sailing from here to a clean & decarbonized world. While we are making progress it is too slow – and carbon emissions are still rising.

Below I have a look at a few of the points Diamandis raised in more detail.

2016: Renewables Cheaper Than Coal

World Economic Forum Reports: Solar and wind now the same price or cheaper than new fossil fuel capacity in over 30 countries. Energy experts think coal will not recover.

I find a statement like this frustrating – it makes it seem as if the fight against coal is already won. The report that Diamandis refers to is the World Economic Forum’s Renewable Infrastructure Investment Handbook: A Guide for Institutional Investors.

You only need to look at one of the figures from that exact report (Figure 1 below) to see that it’s still happy days for coal. Coal is still by far the dominant fuel globally – does it really look like wind & solar have dealt a killing blow from which coal will never recover?

Figure 1 – World Energy Matrix (Renewable Infrastructure Investment Handbook)

It also doesn’t matter if wind & solar are cheap if we aren’t installing more capacity. The same WEF report shows that total investment ($USD billion per annum) in renewables has levelled off since 2011.

Due to the price decrease we still will be installing more renewables at the same level of investment, but it’s the combination of price and investment that gives us what we really care about – annual capacity installed.

Figure 2 – Investments in Clean Energy (USD bn) (Renewable Infrastructure Investment Handbook)

I also find the use of ’30 countries’ a misleading use of statistics. Are these 30 small, sunny countries which are perfect for solar? If the lessons learnt in these 30 countries don’t transfer to China, India and the USA then it doesn’t really matter in terms of fighting climate change.

It would be more relevant to look at in how many countries solar was cheaper than coal, then to weight each country by population or total energy consumption. This would give a more accurate picture of how solar is doing in displacing coal.  More accurate still would be to look at Figure 1.

The Global Status Report for Renewables states that renewable energy now accounts for 25% of the world’s power

The problem here is including all renewables together. The distortion comes from including hydropower with all other renewables.

While the statement Diamandis’ makes is true, it’s misleading to put this fact on the image of a solar panel. BP estimate that non-hydro renewables make up around 8% of global electricity generation. Indeed Ramez Naam (who spoke at the same Exponential Finance conference) estimates that wind & solar are around 7% of global electricity generation. This is a far cry from 25%!

The exact numbers here aren’t important – what’s important is that the viewer of the presentation is left thinking that renewables are doing fantastic when in fact wind & solar still make up a very small portion of global generation. The fight is not over – in fact we aren’t even winning.

Costa Rica operating on 100% renewables for over 300 days

I actually already addressed this misconception in an earlier post (Composition, not consumption). Costa Rica is lucky to have a very high penetration of hydroelectricity (around 80%). Hydroelectric dams have free energy storage built in – this allows the grid to easily deal with the intermittency of renewables.

Most countries do not have the luxury of a large hydro resource, so using Costa Rica as an example of how close we are to going 100% renewable globally is misleading.  We require different techniques and technologies to decarbonize the rest of the world.

Thanks for reading!

Jevon’s paradox – Energy Basics

Efficient use of energy must be the logical first step for anyone trying to slow carbon change. The benefits of not wasting energy are so evident that it should be a high priority for our civilization. Unfortunately it’s not quite that simple.

1865’s The Coal Question introduced what we now call Jevon’s paradox – that technological progress in the efficiency of using a resource leads to increases in resource consumption.

Jevon’s paradox is an inconvenient truth for energy efficiency. It’s not that efficiency doesn’t work – we do use less primary energy per unit of utility. It’s what happens afterwards where the gains in efficiency are cancelled out by more global effects.

Lets look at some of the possible first, second and third order effects (thanks Ray Dalio for this mental model). We will use gas fired heating as the example.

The first order effect of improving heating efficiency is that less gas is required to supply the same amount of heat. This effect is positive – we don’t burn as much gas to provide the same utility.

A secondary effect of improving heating efficiency could be that we now get more heat for the same amount of money. We spend the same amount, we get more heat – but no carbon saving. We can afford to heat bigger homes for the same amount of gas.

A third order effect could be that increased efficiency leads to less gas consumption – meaning saved carbon and money.  The question is then what does the economy do with the saved money?

If the saving is spent on taking a long haul holiday, we could actually see an increase in global carbon emissions. We improve the efficiency of supplying heat but overall as a civilization we burn more carbon. Alternatively if the saving is spent on building cleaner energy generation then even increases in utility could lead to a carbon saving.

It’s very difficult to generalize on what effect Jevon’s paradox has across different consumers, economics and technologies. Measuring the first order effects of energy efficiency projects is notoriously difficult – let alone any second or third order effects.

It’s important to note that energy efficiency is still worthwhile. It allows economic progress – this alone is worth doing. Yet for someone purely concerned with decarbonization, energy efficiency may not be the correct first option.

Jevon’s paradox is not guaranteed to occur. Any negative second or third order effects of energy efficiency can be smaller than the efficiency saving. It perhaps suggests that focusing on making sure any energy we use comes from as clean a primary source as possible is a safer bet than trying to use less dirty energy.

Further reading

Inertia in electricity systems – Energy Basics

Energy Basics is a series covering fundamental energy concepts.

Perhaps you’ve had critics of the energy transition shout “inertia” at you. Perhaps it’s keeping you up at night. Is our dream of a clean energy future impossible? This article will reassure you that losing inertia is something clean energy technologies can deal with.

Large, fossil fuel and synchronous generators have historically dominated our electricity system. Fossil fuels are burnt to force high temperature and pressure gases through turbines. These turbines rapidly spin shafts connected to alternators that generate AC electricity.

We are transitioning to a very different electricity system. We are building small-scale, clean and asynchronous generators. Wind turbines that spin at variable speeds, much slower than synchronous generators. Photovoltaic solar panels and batteries have no moving parts at all.

A key difference between these two systems is the inertia of the generators. Fossil fuel generators posses a lot of inertia due to the rapidly spinning & heavy turbine connected to the alternator. Once the turbine is spinning it’s hard to get it to stop – in the same way that it’s hard to stop a truck traveling at speed.

The speed at which the shaft & alternator needs to spin at is directly proportional to the desired grid frequency. In fact the grid frequency is the result of the speed that all these synchronous generators spin at. The frequency of electricity generated by a synchronous generator is given by

Poles refer to poles of the alternator.  120 is used to convert minutes to seconds and poles to pairs of poles.

The grid is an interconnected system – changing grid frequency requires changing the speed of every generator connected to the grid. This interrelationship becomes useful during times of supply & demand mismatches. Any imbalance needs to work to change the speed at which every generator on the grid spins. If these generators posses a lot of inertia, then the imbalance needs to work harder to change the grid frequency.

This is the value of inertia to the grid – it buys the grid operator time to take other actions such as load shedding or calling upon backup plant. These other actions are still needed – inertia won’t save the grid, just buy time for other actions to save the grid.

So now we understand that fossil fuel generators have inertia and how it is valuable to the grid (it buys the system operator time during emergency events). What does this mean for our energy transition? Do we need to keep around some fossil fuel generators to provide inertia in case something goes wrong? The answer is no.

Modern wind turbines can draw upon kinetic energy stored in the generator and blades to provide a boost during a grid stress. This ‘synthetic inertia’ has been used successfully in Canada, where wind turbines were able to supply a similar level of inertia to conventional synchronous generators.

Figure 1  –  Conceptual fast frequency response from a wind turbine

Photovoltaic solar and batteries also have a role to play. Both operate with inverters that convert DC into AC electricity. The solid-state nature of the devices means that they operate without any inertia. Yet this solid-state nature allows inverters the ability to quickly change operation in a highly controllable way. Inverters can quickly react to deliver whatever kind of support the grid needs during stress events.

Clean technologies are ready to create a new electricity system. Now we need to make sure we incentivize the technology that our grid needs. Market incentives should support technologies that can supply inertia on our cleaner grid. The level of support could be logically set so that the level of inertia on the grid will remain at the same level as our old fossil fuel based grid. That way, no one can complain.

Thanks for reading!

11 tips from 11 months of learning Python

I’ve been learning Python for around 11 months. It’s been a wonderful journey! This post is a list of 11 things that I’ve learned along the way.

1 – Setup

The hardest thing about learning Python can be getting it setup in the first place! I recommend using the Anaconda distribution of Python.

Regarding Python 2 vs Python 3 – if you are starting out now it makes sense to learn Python 3. It’s worth knowing what the differences are between the two – once you’ve made some progress with Python 3.

The installation process is pretty straight forward – you can check that Anaconda installed correctly by typing ‘python’ into Terminal or Command Prompt. You should get something like the following:


2 – pip

pip is a way to manage packages in Python. pip is run from a Terminal. Below are the pip commands I use the most.

To install a package (Note that the -U argument forces pip to install the upgraded version of the package)

pip install pandas -U

To remove a package

pip remove pandas

To print all installed packages

pip freeze

3 – Virtual environments

Virtual environments are best practice for managing Python on your machine. Ideally you should have one virtual environment for each project you work on.

This gives you the ability to work with different versions of packages in different projects and to understand the package dependencies of your project.

There are two main packages for managing virtual environments. Personally I use conda (as I always use the Anaconda distribution of Python).

One cool trick is that once you activate your environment, you can start programs such as Atom or Jupyter and they will use your environment.

For example if you use a terminal plugin within Atom, starting Atom this way will mean the terminal uses your environment Python – not your system Python.

4 – Running Python scripts interactively

Running a script interactively can be very useful when you are learning Python – both for debugging and getting and understanding of what is going on!

cd folder_where_script_lives
python -i

After the script has run you will be left with an interactive console. If Python encounters an error in the script then you will still end up in interactive mode (at the point where the script broke).

5 – enumerate

Often you want to loop over a list and keep information about the index of the current item in the list.

This can naively be done by

idx = 0
for item in a_list:
    other_list[idx] = item
    idx += idx

Python offers a cleaner way to implement this

for idx, item in enumerate(a_list):
    other_list[idx] = item

We can also start the index at a value other than zero

for idx, item in enumerate(a_list, 2):
    other_list[idx] = item

6 – zip

Often we want to iterate over two lists together. A naive approach would be to

for idx, item_1 in enumerate(first_list):
    item_2 = second_list[idx]
    result = item_1 * item_2

A better approach is to make use of zip – part of the Python standard library

for item_1, item_2 in zip(first_list, second_list):
    result = item_1 * item_2

We can even combine zip with enumerate

for idx, (item_1, item_2) in zip(first_list, second_list):
    other_list[idx] = item_1 * item_2

7 – List comprehensions

List comprehensions are baffling at first. They offer a much cleaner way to implement list creation.

A naive approach to making a list would be

new_list = []
for item in old_list:
    new_list.append(2 * item)

List comprehensions offers a way to do this in a single line

new_list = [item * 2 for item in old_list]

You can also create other iterables such as tuples or dictionaries using similar notation.

8 – Default values for functions

Often we create a function with inputs that only need to be changed rarely. We can set a default value for a function by

def my_function(input_1, input_2=10):
    return input_1 * input_2

We can run this function using

result = my_function(input_1=5)

Which will return result = 50.

If we wanted to change the value of the second input we could

result_2 = my_function(input_1=5, input_2=5)

Which will return result = 25.

9 – git

Git is a fantastic tool that I highly recommend using. As with Python I’m no expert! A full write up of how to use git is outside the scope of this article – these commands are useful to get started. Note that all of these commands should be entered in a Terminal that is inside the git repo.

To check the status of the repo

git status

To add files to a commit and push to your master branch

git add file_name
git commit -m 'commit message'
git push origin master

Note that you can do multiple commits in a single push.

We can also add multiple files at once. To add all files that are already tracked (i.e. part of the repo)

git add -u

To add all files (tracked & untracked)

git add *

Another useful command is

git reset HEAD~

What this command allows you to do is to undo local commits. Sometimes you will add files to your commit you didn’t mean to – this allows you to undo them one by one (ie commit by commit).

10 – Text editors

There are a range of text editors you can use to write Python
– Atom
– vi
– vim
– sypder (comes with anaconda)
– Sublime Text
– Pycharm
– notepad ++

All have their positives and negatives. When you are starting out I reccomend using whatever feels the most comfortable.

Personally I started out using notepad ++, then went to spyder, then to Atom and vim.

It’s important to not focus too much on what editor you are using – more important to just write code.

11 – Books & resources

I can recommend the following resources for Python:
Python Reddit
The Hitchhiker’s Guide to Python

Python 3 Object Oriented Programming
Effective Python: 59 Specific Ways to Write Better Python
Python for Data Analysis: Data Wrangling with Pandas, NumPy, and IPython
Python Machine Learning
Automate the Boring Stuf with Python

I can also recommend the following for getting an understanding of git:
Github For The Rest Of Us
Understanding Git Conceptually

Thanks for reading!

Getting Wind and Sun onto the Grid – Energy Insights

Energy Insights highlights interesting energy content from around the web.

Previous posts include Elon Musk on autonomous cars and the CAISO Stage 1 Grid Emergency.

This post highlights three interesting insights from the 2017 IEA report Getting Wind & Sun onto the Grid.

1 – decorrelation of risks to grid stability

An electricity system operator faces different sources of uncertainty when managing a grid. Three of the major sources of uncertainty are:

1 – risk of losing generators or transmission infrastructure – either a large power plant tripping out or loss of part of the transmission system

2 – demand forecasting – electricity supply must be balanced with demand in real time. Backup plant will balance the grid in case demand is higher than forecast

3 – intermittent renewables – the weather dependent nature of wind & solar means that future generation is uncertain. Sudden changes in the weather can lead to dramatic changes in power generated by wind & solar plants. Backup plant must be kept spinning to make up for the lost renewables generation.

The IEA make the point that these three sources of uncertainty are not correlated. I would argue there is a degree of correlation between demand forecasting and the intermittent nature of wind & solar. Generation by low voltage plant like residential solar appears to the system operator as a reduction in demand.

I love the idea of optimizing over all three sources of uncertainty together rather than in isolation. By being smarter about how de-risk these uncertainties we can reduce the grid’s reliance on backup plant.

End consumers of electricity ultimately pay for all grid costs. Avoiding the installation, operation & maintenance of backup plant will mean cheaper & cleaner electricity for us all.

2 – dilemma of resource strength, co-location with grid and geographical smoothing

Optimizing the location of wind & solar plants requires balancing three factors:

1 – renewable resource strength

2 – cost of connecting to the grid

3 – geographical smoothing

Optimizing for resource strength means placing renewables where the wind is strongest or the sun shines brightest. A stronger or more consistent resource availability improves economics by increasing electricity generated. Unfortunately the best wind & solar resources often in remote areas. The cost of connecting to remote resources to the grid is higher.

Co-location with the grid minimizes the investment cost for renewables projects. The IEA quote a study that the cost of expanding the grid had a median value of around 15% of the cost of the generation capacity. Co-location with the grid will keep this cost to a minimum. It will also reduce electricity lost as heat during transmission.

Geographical smoothing is about spreading renewable generation over different areas. Short term variations in renewable generation in different locations tend to cancel out. Concentrating wind turbines in one place means that when the wind stops blowing there it has a major effect on the grid. By spreading the generation geographically, the output is more consistent.

Another concept related to geographical smoothing is technological diversification. This is the smoothing effect gained by diversifying between wind and solar. The IEA make the point that generation from wind & solar is often negatively correlated. This means that changes in solar generation are often balanced by changes in wind generation.

Optimizing the location of a renewables plant requires optimizing all three of these factors together. At first glance we should put wind turbines where it’s windiest – but perhaps a less windy location closer to the grid would be a better use of capital.

3 – technical tools to strengthen the grid

One of the major costs of integrating renewables is grid infrastructure. Renewables plants are often located where the resource is strongest – which can be far away from the transmission system.

The IEA highlight a few techniques to strengthen the grid at minimum investment cost. I’m going to detail two of them.

1 – dynamic line ratings

The capacity of cables to carry electricity is limited by temperature – if the line gets too hot you’re gonna have a bad time. This capacity is also known as the line rating.

Currently the rating of cables are often based on a fixed temperature. Fixing the temperature limits the capacity of the line to carry electricity. A dynamic line rating takes into account the changing ambient temperature. When it gets cold a dynamic line rating would increase the capacity of the cable to carry electricity.

The IEA quote a Swedish project where dynamic line ratings were used to identify a 60% increase in colder periods. These colder periods correlate with windier periods – meaning the increased capacity often coincides with lots of available wind power. I’m not sure if ‘identify’ means these savings were actually realized or just identified as potential savings. Nevertheless – 60% is a significant increase.

Interestingly enough it seems that forecasting for dynamic line rating is an active area of research. It’s another potential application of machine learning – similar to the forecasting of electricity demands or prices I discuss here.

2 – flexible AC transmission systems

This is the use of power electronic devices to improve grid power factor.

Power factor is the ratio of active to reactive power. Active power is useful – reactive power is not useful. But the cables in our electricity systems need to carry both simultaneously.

Reducing reactive power means more space for active power in the cables that make up the grid. The investment required is the solenoids or condensers that are installed midway on the line to absorb or inject reactive power. This is a lot cheaper than building new transmission lines.

Cheat sheet for gas engine & gas turbine CHP – Energy Basics

In my previous life at ENGIE I specialized in the technical modeling of combined heat & power (CHP) plants.

I developed models to support sales projects and to optimize operation of existing sites – such as the district heating scheme at the Olympic Park in London.

CHP is an attractive technology for maximizing the recovery of heat & electricity from fuel. The technologies are mature and will deliver carbon benefits in most electricity grids.

This post aims to give concise practical details about the two most commons forms of gas based CHP. Together the gas engine and gas turbine are around 90 % of installed capacity of CHP in the UK.

Table 1 – CHP in the UK (from DECC)

This post focuses on the facts that matter in the day to day world of energy management.

common for both gas engines & gas turbines
  • Both have total efficiencies (electric + thermal) roughly around 80 % HHV.
  • Both operate with a maximum electric efficiency at full load.
  • In part load operation total efficiency remains around 80 % HHV – reductions in electric efficiency are counteracted by increases in thermal efficiency.
gas engines

Figure 1 – a simple gas engine schematic
Key practical advantages
  • High electric efficiency.
  • Cheap maintenance cost.
  • Cheap capital cost.
Key practical disadvantages
  • Half of the recoverable heat is generated as low quality (<100 °C).
  • Usually only economic at sizes below 5 MWe.

A gas engine has a high electric efficiency (30-38 % HHV).

A gas engine generates roughly the same amount of electricity and heat – i.e. the heat to power ratio is around 1:1.

The recoverable heat generated in a gas engine is split roughly half high grade (>500 °C), half low grade (<100 °C).

Gas engines are typically economic up until 5-6 MWe.  Beyond that size gas turbines become competitive.

Gas engine CHP is low maintenance (0.6 – 1.2 p/kWh @ 8,000 hours/yr) and captial (500 – 1,500 /kWe total project) cost.

Having half of the heat generated as low quality means that a low-grade heat sink is required.

Many industrial processes only require high-temperature heat (typically served using steam). Without a low-grade heat sink for the low-grade heat the economics of a gas engine will suffer.

Typical low-grade heat sinks include space heating, boiler feedwater heating on sites with low condensate return rates and low-temperature process heating. This makes district heating and hospitals good applications of gas engine CHP.

gas turbines
Figure 2 – a simple gas turbine schematic
Key practical advantages
  • All of the heat generated is high quality.
  • Supplementary firing can be used to generate more heat at high efficiency.
  • Potential to combine with steam turbines to generate more power.
Key practical disadvantages
  • Lower electric efficiency.
  • Complex emissions control systems.
  • Usually limited to sizes above 5 MWe.

A gas turbine operates with a lower electric efficiency (25-35% HHV) than a gas engine.

A gas turbine generates roughly twice as much heat as power – ie the heat to power ratio is around 2:1.

Unlike a gas engine, all of the heat generated by a gas turbine is high grade (>500 C).  This makes gas turbines ideal for industrial sites that need high-temperature steam to run their processes.

This also allows gas turbines to be used in combined cycle mode (steam is generated off the exhaust and used to drive a steam turbine).  More gas can be fired into the exhaust to further increase steam generation (known as supplementary firing).

This can be a key advantage of gas turbines, as the marginal efficiency of supplementary firing is higher than generating heat in a shell or water tube boiler.

blockchain in energy – part two – blockchain and the grid

This is the second of a two part series on blockchain in the energy system.

blockchain in energy – part one – what is blockchain
introduction, advantages, description of the system, proof of stake vs. proof of work, smart contracts, bitcoin & Ethereum

blockchain in energy – part two – blockchain and the grid
how far can we go with blockchain and the grid, perspectives of the system operator, consumers, generators, machines & regulators

The first part of this series introduced blockchain and concepts such as smart contracts and proof of work.  In this second post I will theorize on how far we could go with blockchain and the electricity grid.

Today the most successful applications of blockchain are in cryptocurrencies such as bitcoin. Using blockchain to support the grid will offer additional challenges – some of which may necessitate keeping third parties in place.

It’s unclear how far we can go with blockchain and the grid.  Currently applications of blockchain in the energy industry are limited to small pilot scale projects.  Even in more widespread cryptocurrency applications the technology is still not considered mature.

Our transition towards a physically decentralized, small scale and clean electricity system is well underway. Blockchain is not necessary for this transition to happen. Yet it could be a useful tool in developing a more secure, efficient and open electricity grid.

The transmission & distribution system operator

It would be a waste to just use the blockchain to just do meter billing – Joi Ito

I’m excited about the potential for blockchain to be the transaction and control layer for the grid. Blockchain could enable a living, self-organizing grid powered by smart contracts. Smart contracts would automatically match generators with consumers, balancing the grid in near real time. Virtual power plants and storage could also be dispatched using smart contracts.

A blockchain could make smaller scale (in both time and volume) transactions more viable. This would make a more diverse range of resources available to the system operator for real time balancing. It would also reduce the length of the settlement process – no need for complex and costly processes such as profiling.

This blockchain would provide an excellent database for managing grid operations. Data would be available for every transaction – price, quantity and time all down to a fine scale. This dataset would also allow more accurate forecasts of generation & demand. More accurate forecasts will reduce spinning reserve and imbalance costs.

Blockchain is the first technology that can identify of the origin of electricity. This would simplify certification of renewable generation or tracking carbon emissions.

It would also allow a system operator to see which parts of the grid were used to supply electricity. This would allow charging based on how the transmission and distribution system was actually used.

Theoretically a blockchain allows a fully decentralized system. In the context of an electricity grid it’s hard to see how the centralized role of transmission and distribution system operators disappearing.

Electricity consumers

A blockchain could create a more dynamic and competitive marketplace. A smart contract could purchase electricity from many different sources within the same day. An eaiser ability to switch should mean more competition and lower electricity prices. Combine this with the efficiency improvements of removing third parties and we would expect that electricity would get cheaper for consumers.

Smart contracts would also give consumers more power over where their electricity comes from. Environmentally conscious consumers could purchase 100% renewable electricity. Community focused consumers could prefer locally generated electricity. Price concious consumers could minimize costs by always purchasing the cheapest electricity available.

Allowing consumers more control over their electricity mix could increase customer engagement. Getting consumers engaged with where their energy comes from would be a powerful ally in speeding up our energy transition.  It would also be fantastic for utilities looking for loyal customers.

A blockchain could also be powerful in developing communities. Imagine knowing all of the electricity you used was generated on your street. Money would be kept in the local economy, improving the business case for local generation. It’s a virtuous cycle that could accelerate our energy transition.

Using our energy system to develop a community spirit is potentially one of the most powerful impacts a blockchain could have. Technology often leads to more isolated communities – it would be great for technology to bring communities together.

It’s not clear exactly how far a blockchain system could go in completely removing third parties from the grid. A number of duties currently fulfilled by suppliers would need to be done by consumers themselves.

Take forecasting for example. Today electricity suppliers submit forecasts and get charged based on their imbalance. These forecasts are crucial for the system operator to schedule generation. How would this forecasting process be done when there is no supplier responsible for consumers?

Blockchain could be a major positive for electricity consumers in third world countries. Third world economies are often high inflation and low trust – a blockchain using prepaid smart meters could help with both these issues.  Investements in solar panels can be made using cryptocurrencies with the generation supported by a grid blockchainm

Establishing blockchains in a third world countries could allow that economy to skip over the dirty & centralized systems historically used in developed countries.

Electricity generators

A blockchain system could be very powerful for small scale generators. Removing third parties and their inefficiencies should improve the economics of all generators. We are already seeing impressive reductions in the cost of renewables and storage – an increased share of the value of electricity generated kept by investors will only make these technologies more lucrative.

A blockchain would democratize access to the marketplace, removing barriers that currently hurt small scale generators, flexibility, and storage. Today small scale generators often get a very unattractive fixed export rate. A blockchain would allow small scale generators to compete on equal footing in the same marketplace as a 2 GW power station.

Other technologies such as flexibility and storage are often locked out of the market through high minimum capacities or out of market subsidies. A blockchain is the antithesis of this.

The blockchain would also provide an excellent data source for investors in electricity generation. An investor could accurately understand the historical value of electricity generated through the transaction history of the blockchain.


The first role for machines in a blockchain system is to put the smart into smart contracts. The basis for this intelligence could be either a set of human designed heuristics or a narrow machine learning system. Reinforcement learning agents could be deployed as smart contracts, built to provide useful functions such as controlling voltage or frequency.

Blockchain also allows machines to do something quite interesting – own property. A machine with access to the private key of a blockchain account essentially owns any property associated with that private key. A blockchain could lead to the rise of a machine to machine economy, with machines trading and building assets within their blockchain accounts.

Regulators & the legal system

Regulators have a lot of work to do to prepare for a blockchain marketplace. Three broad areas require attention:

  1. protecting consumers,
  2. the physical nature of the electricity system,
  3. nurturing a technology to maturity.

Our current legal framework protects customers in centralized marketplaces. Additional protection is needed for a decentralized system.

The major challenge is the lack of a central authority.  Today’s legal system involves a clear allocation of organizational & legal responsibility – this doesn’t really exist in a system where consensus is reached by the majority. It’s one reason why an electricity grid would likely be supported by a private blockchain.

As discussed in Part One the optimal mechanism for verifying blockchain truth is not clear.  Mechanisms such as proof of work or proof of stake all have drawbacks.  Another implication of verification mechanism design is the balance of power between miners and other nodes.  Paradigms such as the tangle have evolved to adjust this balance.

Another issue is the boundary between the blockchain and the grid (the smart meter). Today suppliers appoint meter operators to install and maintain electricity meters. In a fully decentralized blockchain system it’s unclear who would be responsible for calibrating meters. A blockchain is excellent at ensuring only valid transactions are accepted, but there is still potential for fraud at the interface between the real and blockchain world.

The second area relates to the physical nature of the electricity system. The electricity grid is critical infrastructure that has additional layers of electricity supply security and physical safety.

Using blockchain in an electricity system requires an additional level of care over pure data systems such as cryptocurrencies. Managing the electricity grid involves not only data but the physical delivery of electricity.

Finally regulators will need to be careful in dealing with a technology that is not fully mature. Any immature technology can give surprising results as it develops and deployed at scale.

Thanks for reading!

Sources and further reading

The tangle whitepaper

energy_py update – July 2017

energy_py is a collection of reinforcement learning agents and environments for energy systems. You can read the introductory blog post for the project and check out the repo on GitHub.

Saving of memory and value function after each episode

This quality of life improvement has a major impact on the effectiveness of training agents using energy_py. It means an agent can keep learning from experience that occurred during a different training session.

As I train models on my local machine I often can only dedicate enough time for 10 episodes of training. Saving the memory & value functions an agent to learn from hundreds of episodes without training every episode in one run.

Running each episode on a different time series

Training agents with randomly selected weeks in the year. It’s much more useful for an agent to experience two different weeks of CHP operation than to experience the same week over and over again. It also should help the agent to generalize to operating data sets it hasn’t seen before.

Double Q-Learning

Building another agent has been a todo for energy_py for a long time. I’ve built a Double Q-Leaner – based on the algorithm given in Sutton & Barto. The key extension in Double Q-Learning is to maintain two value functions.

The policy is generated using the average of the estimate for both Q networks. One network is then randomly selected for training using a target created by the other network.

The thinking behind Double Q-Learning is that we can avoid the maximization bias of Q-Learning. A positive bias is caused by the use of maximization operations for estimating the value of states. The maximization functions lead to overoptimistic estimates of the value of state actions.

Next major tasks are:
1 – build a policy gradient method – most likely a Monte Carlo policy gradient algorithm,
2 – build a demand side response environment.

Thanks for reading!

blockchain in energy – part one – what is blockchain

This is the first of a two part series on blockchain in the energy system.

blockchain in energy – part one – what is blockchain
introduction, advantages, description of the system, proof of stake vs. proof of work, smart contracts, bitcoin & Ethereum

blockchain in energy – part two – blockchain and the grid
how far can we go with blockchain and the grid, perspectives of the system operator, consumers, generators, machines & regulators

Decentralization is the future of our physical energy system. Should we manage this decentralized system using large scale, centralized third parties?

Blockchain is a technology that enables decentralization. It allows a true peer to peer economy – no third party needed. Blockchain can offer progress in three dimensions – security, access and efficiency.  Blockchain aligns with our transition towards a decentralized and clean energy future.

Our current energy transition is moving us away from dispatchable, centralized and large-scale generation towards intermittent, distributed and small scale generation.

Figure 1 – Our current energy transition is moving us away from dispatchable, centralized and large-scale generation towards intermittent, distributed and small scale generation.

This post will give an introduction to the blockchain technology – what it is, how it works, the advantages and the challenges.   We will also take a look at two of the largest implementations of blockchain, bitcoin and Ethereum.

Part Two theorizes how far we might be able to go with blockchain and the grid,  We look what a smart contract enabled blockchain might mean for system operators, electricity consumers & generators, machines and regulators.

The Times They Are a-Chanin’

Large scale third parties provide the trust in today’s transaction systems and markets. They perform vital market roles such as identification, clearing, settling and keeping records.

But large scale third parties have downsides that blockchains can address. Blockchain offers advantages in three dimensions – security, access and efficiency.

A centralized system offers a single point of failure for attack – reducing security. Third parties can limit access to markets and data – reducing access. They introduce temporal & monetary costs by increasing the complexity of transactions – reducing efficiency.

Blockchain stores data on every member node – increasing security. Public blockchains are open to all, with individuals able to transact on an equal footing with large corporations – increasing access. Peer to peer transactions are made quickly and at low cost – improving efficiency.

Blockchain can also enable new kinds of transactions and business models. There are exciting applications in the energy industry beyond managing financial transactions – which I expand upon in Part Two.

The future of blockchain is not clear. There are significant technical, legal and regulatory challenges to overcome. Yet these challenges could be far smaller than the potential benefit blockchains could bring to our distributed and clean energy future.

Queen Chain Approximately

Two key technical innovations in blockchain are the decentralization of data storage and decentralization of verification. They both contribute towards the increased security, access and efficiency of the blockchain.

Decentralization of data storage improves security and access. Each member stores a copy of the entire blockchain. This removes the possibility losing data from a central database – improving security.

As each member stores the entire blockchain, she can access infomation about any transaction – democratizing data access. A public blockchain is open for all to join – democratizing market access.

Decentralization of verification improves access and efficiency. By allowing any node to verify transactions, no central authority can limit access to the market – improving access. Rewarding the fastest verification incentivizes reduced transaction time and cost – improving efficiency.

Verification of decentralization is what allows peer to peer transactions. The ability of the collective to verify transactions means you don’t need to wait for a central authority to authorize your transaction.

So decentralization of data and verification leads to improved security, access and efficiency. But what actually is allowing these improvements?

The magic of blockchain is how it maintains the true blockchain state. Blockchain truth is measured by the amount of a scarce resource attached to that blockchain. The scarce resource is supplied by blockchain members.

By making the correct blockchain require resource behind it, it makes proof of the blockchain rely on something that can’t be faked. This is the magic of the blockchain – making digital infomation behave like a scarce physical asset.

Falisfying the blockchain requires attaching more resource to a competitor blockchain. This means a blockchain is not immutable. A transaction can be modified by altering all subsequent blocks with the collusion of the majortiy of the blockchain (a 51% attack).

This fake blockchain could outcompete the true blockchain if enough resource was put behind it. However, once a true blockchain gets far enough ahead this task becomes very difficult.

Jenny From The Block

A key challenge in blockchain is what mechanism to use for validating transactions.  A simplifed description of the mechanics of a proof of work and a proof of stake blockchains are given below.  Currently there is no clear consensus on which mechanism is optimal – so there is significant innovation occuring in developing new mechanisms.

Proof of work

Private and public keys ensure the security of each member. A private key can access an account and execute a transaction. A public key can view the transaction history of the blockchain.

Although this public key is visible to all the identity of the member is not revealed. In this way a blockchain paradoxically provides both complete anonymity but no privacy about transaction history.

Blockchain members (known as nodes) run the blockchain. Each node can generate & digitally sign transactions using their private key. Transactions are added to a list held locally by the node and rapidly forwarded onto other nodes.

Figure 2 – the blockchain process (source = PWC)

It’s then the role of ‘miners’ (verification nodes) to create blocks from these lists of transactions. Miners first check the balances of both parties – available through the distributed blockchain. Miners then compete to solve a mathematical problem that allows transactions to be chained together. Solving the problem requires scarce real world resources (hardware and energy) meaning there is no way to cheat.

Figure 3 – the verification process (source = PWC)

Solving this problem is not computing anything useful – but requires computational resource to be solved correctly. This is how proof of work attaches a scarce resource (hardware and energy) to digital infomation (the true blockchain).

After the new block is added to the blockchain, it is propagated around the network in a similar fashion to transactions. Any conflicting transactions on nodes are discarded. It makes sense for other miners to add that validated block to their own blockchain and to work on the next block. This alignment of incentives allows the true blockchain to outgrow the others as more and more miners get rewarded to extend it.

The disadvantage of the proof of work consensus process is that all this computation requires a lot of electricity. This cost is also paid even if no one is trying to disrupt the blockchain.

In 2015 the electricity consumed in a single Bitcoin transaction was enough to power 1.57 American households for one day. If we assume an annual average consumption of 10.8 MWh and an electricity cost of £30/MWh, this equates to a cost of £1.60 per transaction. There is also the carbon impact of this electricity consumption.

Proof of stake

Proof of stake is a solution to the electricity consumption problem of proof of work. In a proof of stake system validators must place a deposit of cryptocurrency in their block. Blockchain truth is deterministically determined by the amount of currency attached.

In both systems the influence of a a user depends on the amount of scarce resource they put behind a blockchain. In a proof of work system the resource is computational; in a proof of stake system the resource is financial.

Smart contracts

The ability to deploy smart contracts is one of the blockchains key strengths. It’s how we can infuse a blockchain system with intelligence and movement. It enables new types of transactions, business models and markets on the blockchain.

A smart contract is an application that runs on the blockchain. Smart contracts improve the efficiency of the blockchain by automating manual processes. Smart contracts can allow new kinds of business models on the blockchain.

Smart contracts allow innovation to occur on a small scale. No central third party can limit the experimentation of smart contracts. A blockchain can support diverse ecosystem of smart contracts.

Bitcoin and Ethereum

Today the most largest and most visible use of the blockchain technology are in cryptocurrencies like bitcoin and Ethereum.

Bitcoin is both a cryptocurrency and a digital payment system. The main blockchain of bitcoin is public – anyone can mine, buy and trade bitcoin. The technology behind bitcoin is open source. Yet bitcoin is not an ideal blockchain implementation – on key KPIs such as throughput speed, latency bitcoin lags well behind VISA.

Storage of the blockchain is also an issue – in April 2017 the size of the bitcoin ledger was around 200 GB. If VISA were implemented using a blockchain it would take up around 200 PB per year. Electricity consumption is also a problem – in April 2017 bitcoin mining consumed around 300 MW of electricity.

Ethereum is a bitcoin competitor. Ethereum is also a public, open source blockchain with it’s own currency known as ‘ether’. The total value of both bitcoin and Ethererum in circulation is over $80B USD.

The key advantage of Ethereum over bitcoin is that it allows more flexible smart contracts. Ethereum has an integrated, Turing complete and all purpose programming language. Bitcoin’s language is not Turing complete – meaning there are solvable problems that bitcoin’s language cannot solve.

This limits the flexbility of smart contracts on the bitcoin implementation of blockchain. Bitcoin is limited to a currency and payment system. Ethererum could allow many different kinds of assets to be traded for many different business purposes.

Another difference between bitcoin and Ethereum is the algorithm used for reaching a consensus on the true state of the blockchain. Both historically use proof of work to reach a consensus on blockchain truth. Recently Ethereum is phasing in proof of stake as a lower cost way of reaching consensus on blockchain truth.

Thanks for reading!

Sources and further reading
White papers
Talks & Podcasts
Reports & Articles