machine learning in energy – part one

This is the first of a two part series on the intersection of machine learning and energy.

machine learning in energy – part one
Introduction, why it’s so exciting, challenges.

machine learning in energy – part two
Time series forecasting, energy disaggregation, reinforcement learning, Google data
centre optimization.


Technological innovation, environmental politics and international relations all influence the development of our global energy system.  There is one less visible trend that will be one of the most important.  Machine learning is blowing past previous barriers for a wide range of problems.  Computer vision, language processing and decision making have all been revolutionized by machine learning.

I see machine learning as fundamentally new.  Mankind developed using only the intelligence in his own brain until we learned to communicate.  Since then the development of technologies such as the printing press or the internet now allow us to access the intelligence across the entire human species.  But machine learning is something different.  We can now access the intelligence of another species – machines.

Part One of this series will introduce what machine learning is, why it’s so exciting and some of the challenges of modern machine learning. Part Two goes into detail on applications of machine learning in the energy industry such as forecasting or energy disaggregation.

What is machine learning

Machine learning gives computers the ability to learn without being explicitly programmed. Computers use this ability to learn patterns in large, high-dimensionality datasets. Seeing these patterns allows computers to achieve results at superhuman levels – literally better than what a human expert can achieve.  This ability has now made machine learning the state of the art for a wide range of problems.

To demonstrate what is different about machine learning, we can compare two landmark achievements in computing & artificial intelligence.

In 1996 IBM’s Deep Blue defeated World Chess Champion Gary Kasparov. IBMs Deep Blue ‘derived it’s playing strength mainly from brute force computing power’. But all of Deep Blue’s intelligence originated in the brains of a team of programmers and chess Grandmasters.

In 2016 Alphabet’s Alpha Go defeated Go legend Lee Sedol 4-1. AlphaGo also made use of a massive amount of computing power. But the key difference is that AlphaGo was not given any information about the game of Go from its programmers. Alpha Go used reinforcement learning to give Alpha Go the ability to learn from its own experience of the game.

Both of these achievements are important landmarks in computing and artificial intelligence. Yet they are also fundamentally different because machine learning allowed AlphaGo to learn on it’s own.

Why now

Three broad trends have led to machine learning being the powerful force it is today.

One – Data

It’s hard to overestimate the importance of data to modern machine learning. Larger data sets tend to make machine learning models more powerful. A weaker algorithm with more data can outperform a stronger algorithm with less data.

The internet has brought about a massive increase in the growth rate of data. This volume of data is enabling machine learning models to achieve superhuman performance.

For many large technology companies such as Alphabet or Facebook their data has become a major source of the value of their businesses. A lot of this value comes from the insights that machines can learn from such large data sets.

Two – Hardware

There are two distinct trends in hardware that have been fundamental to moving modern machine learning forward.

The first is the use of graphics processing units (GPUs) and the second is the increased availability of computing power.

In the early 2000’s computer scientists innovated the use of graphics cards originally designed for gamers for machine learning. They discovered massive increases in training times – reducing them from months to weeks or even days.

This speed up is important. Most of our understanding of machine learning is empirical . This knowledge is built up a lot faster by reducing the iteration time for training machine learning models.

The second trend is the availability of computing power. Platforms such as Amazon Web Services or Google Cloud allow on-demand access to a large amount of GPU-enabled computing power.

Access to computing power on demand allows more companies to build machine learning products. It enables companies to shift a capital expense (building data centres) into an operating expense, with all the balance sheet benefits that brings.

Three – Algorithms & tools

I debated whether to include this third trend. It’s really the first two trends (data & hardware) that have unlocked the latent power of machine learning algorithms, many of which are decades old. Yet I still think it’s worth touching on algorithms and tools.

Neural networks form the basis of many state of the art machine learning applications. Neural networks with multiple layers of non-linear processing units (known as deep learning) that forms the backbone of the most impressive applications of machine learning today. These artificial neural networks are inspired by the biological neural networks inside our brains.

Convolutional neural networks have revolutionised computer vision through a design based on the structure of our own visual cortex. Recurrent neural networks (specifically the LSTM implementation) have transformed sequence & natural language processing by allowing the network to hold state and ‘remember’.

Another key trend in machine learning algorithms is the availability of open source tools. Companies such as Alphabet or Facebook make many of their machine learning tools all open source and available.

It’s important to note that while these technology companies share their tools, they don’t share their data. This is because data is the crucial element in producing value from machine learning. World-class tools and computing power are not enough to deliver value from machine learning – you need data to make the magic happen.

Challenges

Any powerful technology has downsides and drawbacks.

By this point in the article the importance of data to modern machine learning is clear. In fact large datasets are so important for supervised machine learning algorithms used today that it is a weakness. Many techniques don’t work on small datasets.

Human beings are able to learn from small amounts of training data – burning yourself once on the oven is enough to learn not to touch it again. Many machine learning algorithms are not able to learn in this way.

Another problem in machine learning is interpretability. A model such as a neural network doesn’t immediately lend itself to explanation. The high dimensionality of the input and parameter space means that it’s hard to pin down cause to effect. This can be difficult when considering using a machine learner in a real world system. It’s a challenge the financial industry is struggling with at the moment.

Related to this is the challenge of a solid theoretical understanding. Many academics and computer scientists are uncomfortable with machine learning. We can empirically test if machine learning is working, but we don’t really know why it is working.

Worker displacement from the automation of jobs is a key challenge for humanity in the 21st century. Machine learning is not required for automation, but it will magnify the impact of automation. Political innovations (such as the universal basic income) are needed to fight the inequality that could emerge from the power of machine learning.

I believe it is possible for us to deploy automation and machine learning while increasing the quality of life for all of society. The move towards a machine intelligent world will be a positive one if we share the value created.

In the context of the energy industry, the major challenge is digitization.  The energy system is notoriously poor at managing data, so full digitalization is still needed.  By full digitalization I mean a system where everything from sensor level data to prices are accessible to employees & machines, worldwide in near real time.

It’s not about having a local site plant control system and historian setup. The 21st-century energy company should have all data available in the cloud in real time. This will allow machine learning models deployed to the cloud to help improve the performance of our energy system. It’s easier to deploy a virtual machine in the cloud than to install & maintain a dedicated system on site.

Data is one of the most strategic assets a company can own. It’s valuable not only because of the insights it can generate today, but also the value that will be created in the future. Data is an investment that will pay off.

Part Two of this series goes into detail on specific applications of machine learning in the energy industry – forecasting, energy disaggregation and reinforcement learning.  We also take a look at one of the most famous applications of machine learning in an energy system – Google’s work in their own data centers.

Thanks for reading!

Sources and further reading

CHP Feasibility & Optimization Model v0.3

See the introductory post for this model here.  


This is v0.3 of the open source CHP feasibility and optimization model I am developing.  The model is setup with some dummy data.  This model is in beta – it is a work in progress!

If you want to get it working for your project all you need to do is change:

  • heat & power demands (Model : Column E-G)
  • prices (Model : Column BD-BF)
  • CHP engine (Input : Engine Library).

You can also optimize the operation of the CHP using a parametric optimization VBA script (Model : Column BQ).

You can download the latest version of the CHP scoping model here.

If you would like to get involved in working on this project please get in touch.

Thanks for reading!

 

Oil Reserves Growth – Energy Basics

Energy Basics is a series covering fundamental energy concepts.


As we consume non-renewable resources, the amount of that resource depletes. Makes sense right?

Yet when it comes to oil reserves we find that oil reserves actually grow over time! This is known as ‘oil reserves growth’. Why does this phenomenon occur?

First, let’s start by defining some relevant terms.

Oil reserves are the amount of oil that can be technically recovered at the current price of oil.

Oil resources are all oil that can be technically recovered at any price.

Oil in place is all the oil in a reservoir (both technically recoverable & unrecoverable oil).

oil reserves growth

Figure 1 – Proved oil reserves and Brent crude oil price (BP Statistical Review 2016)

So why do reserves grow over time? There are three reasons.

One – Geological estimates

Initial estimates of the oil resource are often low. It’s very difficult to estimate the amount of oil in a reservoir as you can’t directly measure it. Often a lot of computing power is thrown at trying to figure out how much oil is underground.

It’s also good engineering practice to stay on the low side when estimating for any project. I expect geologists intentionally do the same for geological estimates.

Two – Oil prices

Oil reserves are a direct function of the current oil price. Increasing oil prices means that more of the oil resource can be classed as an oil reserve.

Historically we have seen oil prices increase – leading to growth in oil reserves (even with the oil resource being depleted at the same time).

But increasing prices can also have secondary effects. A higher price might incentivise an oil company to invest more into an existing field – leading to an increase in oil recovery.

The reserves growth of existing fields can actually be responsible for the majority of additions to reserves.  Between 1977 to 1995 approximately 89% of the additions to US proved reserves of crude oil were due to oil reserves growth rather than the discovery of new fields.

Three – Technology

Improvements in technology have two effects. The first is to make more of the oil in place technically recoverable at any price (ie to increase the oil resource). Hydraulic fracturing (fracking) and horizontal drilling now allow

The first is to make more of the oil in place technically recoverable at any price (ie to increase the oil resource). Hydraulic fracturing (fracking) and horizontal drilling now allow access to oil that previously was technically unrecoverable.

The second is that as technology improves it also gets cheaper. This improvement in economics means that more of the oil resource can be classed as an oil reserve (even at constant or falling prices).


Thanks for reading!

The Complexity of a Zero Carbon Grid – Energy Insights

Energy Insights is a series highlighting interesting energy content from around the web.

Previous posts in this series include Automated Cars and How to save the energy system.


I’m excited to present this Energy Insights post. I’m highlighting a few interesting insights from the ‘The Complexity of a Zero Carbon Grid’ show.

This is very special as The Interchange podcast has only been publically relaunched recently.

The show considers what may be necessary to get to levels of 80-100% renewables. Stephen Lacey and Shayle Kann host the show with Jesse Jenkins as the guest.

The concept of flexibility

Jenkins observes that the concept of flexibility of electrical capacity appearing in literature. Flexibility means how quickly an asset is able to respond to change.

A combined cycle gas turbine plant is usually more flexible than a coal or nuclear generator. One reason for this is the ability to control plant electric output by modulating the supplementary burner gas consumption.


We will need flexibility on a second, minute, hourly or seasonal basis.

This concept of flexibility was also recently touched on by the excellent Energy Analyst blog. Patrick Avis notes that we need both flexibility (kW or kW/min) and capacity (kWh) for a high renewables scenario.

The post ‘Flexibility in Europe’s power sector’ could easily be enough material for a few Energy Insights posts. Well worth a read.

One investment cycle away

Jenkins observes that the investment decisions we make today will affect how we decarbonise in the future. Considering the lifetime of many electricity generation assets, we find that we are only a single investment cycle away from building plants that will be operating in 2050.

Most deep decarbonisation roadmaps include essentially zero carbon electricity by 2050. We need to ensure that when the next investment cycle begins we are not installing carbon intense generation as it would still be operating in 2050.

For both gas and coal the implied cutoff date for plant operation to begin is between 2010 – 2020.

Increasing marginal challenge of renewables deployment

The inverse relationship between the level of deployment of renewables and the marginal value added is well known. Jenkins notes that this relationship also applies to the deployment of storage and demand side response.

As renewable deployment increases the challenges for both storage and demand side response also increase.

Seasonal storage technologies

1 – Power to gas

Electricity -> hydrogen -> synthetic methane.

Figure 3 – Apros Power to Gas

Intermittency of the supply of excess renewable generation means that power to gas asset wouldn’t be fully utilized.

Didn’t cover the possibility of storage of electricity to allow a constant supply of electricity to the power to gas asset.

2 – Underground thermal

Limited to demonstration scale.

Didn’t cover the feasibility of generating electricity from the stored heat.

I would expect that the temperature of the stored heat is low.  Perhaps the temperature could be increased with renewable powered heat pumps.


Thanks for reading!

 

energy_py – reinforcement learning in energy systems

energy_py is reinforcement learning in energy systems.  It’s reinforcement learning agents and environments built in Python.

I have a vision of using reinforcement learners to optimally operate energy systems.  energy_py is a step towards this vision.  I’ve built this because I’m so excited about the potential of reinforcement learning in the energy industry.

Reinforcement learning in energy systems requires first proving the concepts in a virtual environment.  This project demonstrates the ability of reinforcement learning to control a virtual energy environment.

What is reinforcement learning

supervised vs unsupervised vs reinforcement

Reinforcement learning is the branch of machine learning where learning occurs through action.  Reinforcement learning will give us the tools to operate our energy systems at superhuman levels of performance.

It’s quite different from supervised learning. In supervised learning we start out with a big data set of features and our target. We train a model to replicate this target from patterns in the data.

In reinforcement learning we start out with no data. The agent generates data by interacting with the environment. The agent then learns patterns in this data. These patterns help the agent to choose actions that maximize total reward.

Why do we need reinforcement learning in energy systems

Optimal operation of energy assets is already very challenging. Our current energy transition is making this difficult problem even harder. The rise of intermittent and distributed generation is introducing volatility and increasing the number of actions available to operators.

For a wide range of problems machine learning results are both state of the art and better than human experts. We can get this level of performance using reinforcement learning in our energy systems.

Today many operators use rules or abstract models to dispatch assets. A set of rules is not able to guarantee optimal operation in many energy systems.

Optimal operating strategies can be developed from abstract models. Yet abstract models (such as linear programming) are often constrained. These models are limited to approximations of the actual plant.  Reinforcement learners are able to learn directly from their experience of the actual plant.

Reinforcement learning can also deal with non-linearity. Most energy systems exhibit non-linear behavior (in fact an energy balance is bi-linear!). Reinforcement learning can model non-linearity using neural networks. It is also able to deal with the non-stationary and hidden environment in many energy systems.

beautiful wind turbines

There are challenges to be overcome. The first and most important is safety. Safety is the number one concern in any engineering discipline. What is important to understand is we limit the actions available to the agent. All lower levels or systems of controls would remain exactly the same.

There is also the possibility to design the reward function to incentivize safety. A well-designed reinforcement learner could actually reduce hazards to operators.

A final challenge worth addressing is the impact such a learner could have on employment. Machine learning is not a replacement for human operators. A reinforcement learner would not need a reduction in employees to be a good investment.

The value of using a reinforcement learner is to let operations teams do their jobs better. It will allow them to spend more time and improve performance for their remaining responsibilities such as maintaining the plant.  The value created here is a better-maintained plant and a happier workforce – in a plant that is operating with superhuman levels of economic and environmental performance.

Any machine requires downtime – a reinforcement learner is no different. There will still be time periods where the plant will operate in manual or semi-automatic modes with human guidance.

energy_py is one step on a long journey of getting reinforcement learners helping us in the energy industry. The fight against climate change is the greatest that humanity faces. Reinforcement learning will be a key ally in fighting it.

 

energy_py is built using Python.  You can checkout the repository on GitHub here.

Initial Results

I’m really looking forward to getting to know energy_py.  There are a number of parameters to tune. For example the structure of the environment or the design of the reward function can be modified to make the reinforcement learning problem more challenging.

One key design choice is the number of assets the agent has to control.  The more choices available to the agent the more complex the shape of the value function becomes.  To approximate a more complex value function we may need a more complex neural network.

There is also a computational cost incurred with increasing the number of actions.  More actions means more [state, actions] to consider during action selection and training (both of which require value function predictions).

So far I’ve been experimenting with an environment based on two assets – a 7.5 MWe gas turbine and a 7.5 MWe gas engine.  The episode length is set to 336 steps (one week).  I run a single naive episode, 30 ε-greedy episodes and a single greedy episode.
Figure 1 below shows the total reward per episode increasing as the agent improves it’s estimate of the value function and spends less time exploring.
Figure 1 – Epsilon decay and total reward per episode

Figure 2 shows the Q-test and the network training history.  Q-test is the average of three random [state, actions] evaluated by the value function.  It shows how the value function approximation changes over time.

Figure 2 – Q-Test and the network training history

Figure 3 shows some energy engineering outputs.  I’m pretty happy with this operating regieme – the model is roughly following both the electricity price and the heat demand which is expected behaviour.

Figure 3 – Energy engineering outputs for the final greedy run

 One interesting decision to make is how often to improve the approximation of the value function.  David Silver makes the point that you should make use of the freshest data when training – hence training after each step through the environment.  He also makes the point that you don’t need to fully train the network – just train a ‘little bit’.

This makes sense as the distribution of the data (the replay memory) will change as the learner trains it’s value function.  I train on a 64 sample batch of the replay memory.  I perform 100 passes over the entire data set (i.e. 100 epochs).   Both values can be optimized.  It could make sense to train for more epochs in later episodes as we want to fit the data more than in earlier episodes.

Another challenge in energy_py is balancing exploration versus exploitation.  The Q_learner algorithm handles this dilemma using ε-greedy action selection.  I decay epsilon at a fixed rate – the optimal selection of this parameter is something I’ll take a look at in the future.

There are many exciting innovations developed recently in reinforcement learning that I’m keen to add to energy_py.  One example is the idea of Prioritized Experience Replay – where the batch is not taken randomly from the replay memory but instead prioritizes some samples over others.

It’s unlikely that I’ll ever catch up to the state of the art in reinforcement learning – what I hope of find is that we don’t need state of the art techniques to get superhuman performance from energy systems!

My reinforcement learning journey

I’m a chemical engineer by training (B.Eng, MSc) and an energy engineer by profession. I’m really excited about the potential of machine learning in the energy industry – in fact that’s what this blog is about!

My understanding of reinforcement learning has come from a variety of resources. I’d like to give credit to all of the wonderful resources I’ve used to understand reinforcement learning.

Sutton & Barto – Reinforcement Learning: An Introduction – the bible of reinforcement learning and a classic machine learning text.

Playing Blackjack with Monte Carlo Methods – I built my first reinforcement learning model to operate a battery using this post as a guide. This post is part two of an excellent three part series. Many thanks to Brandon of Δ ℚuantitative √ourney.

RL Course by David Silver – over 15 hours of lectures from Google DeepMind’s lead programmer – David Silver. Amazing resource from a brilliant mind and brillaint teacher.

Deep Q-Learning with Keras and gym – great blog post that showcases code for a reinforcement learning agent to control a Open AI Gym environment. Useful both for the gym integration and using Keras to build a non-linear value function approximation. Many thanks to Keon Kim – check out his blog here.

Artificial Intelligence and the Future – Demis Hassabis is the co-founder and CEO of Google DeepMind.  In this talk he gives some great insight into the AlphaGo project.

Minh et. al (2013) Playing Atari with Deep Reinforcement Learning – to give you an idea of the importance of this paper – Google purchased DeepMind after this paper was published.  DeepMind was a company with no revenue, no customers and no product – valued by Google at $500M!  This is a landmark paper in reinforcement learning.

Minh et. al (2015) Human-level control through deep reinforcement learning – an update to the 2013 paper published in Nature.

I would also like to thank Data Science Retreat.  I’m just finishing up the three month immersive program – energy_py is my project for the course.  Data Science Retreat has been a fantastic experience and I would highly recommend it.  The course is a great way to invest in yourself, develop professionally and meet amazing people.

That’s it from me – thanks for reading!

Energy Basics – Capacity Factor

All men & women are created equal. Unfortunately the same is not true for electricity generating capacity.
 
Capacity on it’s own is worthless – what counts the electricity generated (kWh) from that capacity (kW). If the distinction between kW and kWh is not clear this previous post will be useful.
 

Capacity factor is one way to quantify the value of capacity. It’s the actual electricity (kWh) generated as a percentage of the theoretical maximum (operation at maximum kW).

For example to calculate the capacity factor on an annual basis:
 
 

There are many reasons why capacity will not generate as much as it could.

Three major reasons are maintenance, unavailability of fuel and economics.

 
Maintenance
 
Burning fossil fuels creates a challenging engineering environment. The core of a gas turbine is high pressure & temperature gases rapidly rotating blazing hot metal. Coal power stations generate electricity by high pressure steam forcing a steam turbine to spin incredibly fast.
 
These high challenges mean that fossil fuel plants need a lot of maintenance. The time when the plant is being maintained is time the capacity isn’t generating electricity.
 
Renewables plants need a lot less maintenance than a fossil fuel generator. No combustion means there is a lot less stress on equipment.
 
Availability of fuel
 
Yet while renewables come ahead in terms of maintenance, they fall behind due to a constraint that fossil fuel generation usually doesn’t suffer from – unavailability of fuel.
 
This is why renewables like wind & solar are classed as intermittent. Fuel is often not available meaning generation is often not possible.
 
Solar panels can’t generate at night. Wind turbines need wind speeds to be within a certain range – not too low, not too high – just right.
 
This means that wind & solar plants are often not able to generate at full capacity – or even to generate at all. This problem isn’t common for fossil fuel generation. Fossil fuels are almost always available through natural gas grids or on site coal storage.
 
Economics
 
The final reason for capacity to not generate is economics.
 
The relative price of energy and regulations change how fossil fuel capacity is dispatched. Today’s low natural gas price environment is the reason why coal capacity factors have been dropping.
 
Here renewables come out way ahead of fossil fuels. As the fuel is free renewables can generate electricity at much lower marginal cost than fossil fuels. Wind & solar almost always take priority over fossil fuel generation.
 
Typical capacity factors
 
The capacity factor wraps up and quantifies all of the factors discussed above.
 
Table 1 – Annual capacity factors (2014-2016 US average)

CoalCCGTWindSolar PVNuclear
Annual Capacity Factor56.13%53.40%33.63%26.30%92.17%

 

Table 1 gives us quite a bit of insight into the relative value of different electricity generating technologies. The capacity factor for natural gas is roughly twice as high as solar PV.

 

We could conclude that 1 MW of natural gas capacity is worth around twice as much as 1 MW of solar PV.

How useful is the capacity factor?

Yet the capacity factor is not a perfect measure of how valuable capacity is. Taking the average of anything loses infomation – capacity factor is no different.
 
Two plants operating in quite different ways can have the same capacity factor. A plant that operated 50% for the entire year and a plant that generated for half of the year at full capacity will both have an identical capacity factor.
 
The capacity factor loses infomation about the time of energy generation. The time of generation & demand is a crucial element in almost every energy system.
 
Generation during a peak can be a lot more valuable to the world than generation at other times. Because of the nature of dispatchable generation it is more likely to be running during a peak.
 
This leads us to conclude that low capacity factor generation could be more valuable than higher capacity factor generation.  This is especially true for solar in many countries as a) the peak often occurs when the sun is down and b) all solar generation is coincident.
 
The solution to the intermittency problem of renewables is storage. Storage will allow intermittent generation to be used when it’s most valuable – not just whenever it happens to be windy or sunny.
 
Thanks for reading!

Energy Insights – How to save the energy system – André Bardow

Energy Insights is a series where we highlight interesting energy content from around the web.

The previous post in this series was about the 2017 BP Annual Energy Outlook.


These three Energy Insights come from a TED talk titled ‘How to save the energy system’ given by André Bardow.   André is a fantastic speaker and his talk is well worth a watch.

Below are three of the many interesting things André discusses. I really enjoyed writing this – I find all of this fascinating.

Why peak oil won’t save the planet

As humanity burns more oil the amount of oil left to recover should decrease. This is logical – right?

Figure 1 below shows the opposite is true. Estimated global oil reserves have actually increased over time.  The key is to understand the definition of oil reserves.

Figure 1 – Estiamted global oil reserves Estimated global oil reserves (1980 – 2014)

Oil reserves are defined
 as the amount of oil that can be technically recovered at a cost that is financially feasible at the present price of oil.

Oil reserves are therefore a function of a number of variables that change over time:

  • Total physical oil in place – physical production of oil reduces the physical amount of oil in the ground.
  • Total estimated oil in place – the initial estimates are low and increased over time.
  • Oil prices – historically oil prices have trended upwards (Figure 2). Oil reserves defined as a direct function of the present oil price.
  • Technology – the oil & gas industry has benefited more than any other energy sector from improved technology.  Improved technology reduces the cost of producing oil.  This makes more oil production viable at a given price.
Figure 2 – Crude oil price (1950 – 2010)

Only the physcial production of oil has had a negative effect on oil reserves.

The other three (total oil estimate, technology & oil price) have all caused oil reserve estimates to increase.

We are not going to run out of oil any time soon. The limit on oil   production is not set by physcial reserves but by climate change.  I find this worrying – it would be much better if humanity was forced to swtich to renewables!

Wind & solar lack an inherent economy of scale

 A lot of the advantages in systems are from economies of scale – energy systems are no different.  Larger plants are more energy efficient and have lower specific capital & maintenance costs.

Energy efficiency improves with size as the ratio of fixed energy losses to energy produced improves.   Figure 3 shows an example of this for spark ignition gas engines.
Figure 3 – Effect of gas engine size [kWe] on gross electric efficiency [% HHV]

This is also why part load efficiency is worse than full load efficiency.  Energy production reduces but fixed  energy losses remain constant.

Specific capital & operating costs also improve with size.  For example, a 10 MW and 100 MW plant may need the same land area at a cost of £10,000.  The specific capital cost of land for both projects is £1,000/MW versus £100/MW respectively.

Fossil fuel plants use their economy of scale to generate large amounts of electricity from a small number of prime movers.

Wind & solar plants are not able to do this. The problem is the prime movers in both wind & solar plants.

The maximum size of a wind or solar prime movers (wind turbines or solar panels) is small comapred with fossil fuel prime movers.  For example GE offer a 519 MWe gas turbine – the world’s largest wind turbine is the 8 MWe Vestas V164.

Figure 4 – The Vestas V164

A single gas turbine in CCGT mode is more than enough to generate 500 MWe.  A wind farm needs 63 wind turbines to generate the same amount.

The reason for the difference is fundamental to the technologies – the energy density of fuel.  Fossil fuels offer fantastic energy densities – meaning we can do a lot with less fuel (and less equipment).  Transportation favours liquid fossil fuels for this reason.

Wind & solar radiation have low energy densities. To capture more energy we need lots more blade or panel surface area.  This physical constraint means that scaling the prime movers in wind & solar plants is difficult. The physical size increases very fast as we increase electricity generation.

This means that wind turbines & solar panels need to very cheap at small scales. As wind & solar technologies improve there will be improvements in both the economy of scale & maximum size of a single prime mover.

But to offer a similar economy of scale as fossil fuels is difficult due to low energy density fuel.  It’s not that wind & solar don’t benefit from any economy of scale – for example grid connection costs can be shared. It’s the fact that fossil fuels:

  • share most of these economies of scale.
  • use high energy density fuels, which gives a fundamental advantage in the form of large maximum prime mover sizes.

We need to decarbonise the supply of heat & power as rapidly as possible.  Renewables are going to be a big part of that.  The great thing is that even with this disadvantage wind & solar plants are being built around the world!

Average German capacity factors
Andre gives reference capacity factors for the German grid of:

  • Solar = 10 %.
  • Wind = 20 %.
  • Coal = 80 %.

This data is on an average basis.  The average capacity factor across the fleet is usually more relevant than the capacity factor of a state of the art plant.

It is always good to have some rough estimates in the back if your mind.  A large part of engineering is using your own estimates based on experience with the inputs or outputs of models.

Thanks for reading!

CHP Scoping Model v0.2

See the introductory post for this model here.  

This is v0.2 of the CHP scoping model I am developing.  The model is setup with some dummy data.

If you want to get it working for your project all you need to do is change:

  • heat & power demands (Model : Column F-H)
  • prices (Model : Column BF-BH)
  • CHP engine (Input : Engine Library).

You can also optimize the operation of the CHP using a parametric optimization VBA script (Model : Column BW).

You can download the latest version of the CHP scoping model here.

Thanks for reading!

Tuning Model Structure – Number of Layers & Number of Nodes

Imbalance Price Forecasting is a series applying machine learning to forecasting the UK Imbalance Price.  

Last post I introduced a new version of the neural network I am building.  This new version is a feedforward fully connected neural network written in Python built using Keras.

I’m now working on tuning model hyperparameters and structure. Previously I setup two experiments looking at:

  1. Activation functions
    • concluded rectified linear (relu) is superior to tanh, sigmoid & linear.
  2. Data sources
    • concluded more data the better.

In this post I detail two more experiments:

  1. Number of layers
  2. Number of nodes per layer

Python improvements

I’ve made two improvements to my implementation of Keras.  An updated script is available on my GitHub.

I often saw during training that the model trained on the last epoch was not necessarily the best model. I have made use of a ModelCheckpoint that saves the weights of the best model trained.

The second change I have made is to include dropout layers after the input layer and each hidden layer.  This is a better implementation of dropout!

Experiment one – number of layers

Model parameters were:

  • 15,000 epochs. Trained in three batches. 10 folds cross-validation.
  • 2016 Imbalance Price & Imbalance Volume data scraped from Elexon.
  • Feature matrix of lag 48 of Price & Volume & Sparse matrix of Settlement Period, Day of the week and Month.
  • Feed-forward neural network:
    • Input layer with 1000 nodes, fully connected.
    • 0-5 hidden layers with 1000 nodes, fully connected.
    • 1-6 dropout layers. One under input & each hidden layer.  30% dropout.
    • Output layer with 1000 nodes, single output node.
    • Loss function = mean squared error.
    • Optimizer = adam (default parameters).

Results of the experiments are shown below in Fig. 1 – 3.

Figure 1 – number of layers vs final training loss
Figure 2 – number of layers vs MASE

Figure 1 shows two layers with the smallest training loss.

Figure 2 shows that two layers also has the lowest CV MASE (although has a high training MASE).

Figure 3 – number of layers vs overfit. Absolute overfit = Test-Training. Relative = Absolute / Test.

In terms of overfitting two layers shows reasonable absolute & relative overfit.  The low relative overfit is due to a high training MASE (which minimizes the overfit for a constant CV MASE).

My conclusion from this set of experiments is to go forward with a model of two layers.  Increasing the number of layers beyond this doesn’t seem to improve performance.

It is possible that training for more epochs may improve the performance of the more complex networks which will be harder to train.  For the scope of this project I am happy to settle on two layers.

Experiment two – number of nodes

For the second set of experiments all model parameters were all as above except for:

  • 2 hidden layers with 50-1000 nodes.
  • 5 fold cross validation.
Figure 4 – number of layers vs final training loss
Figure 5 – number of layers vs MASE
Figure 6 – number of layers vs overfit.  Absolute overfit = Test-Training.  Relative = Absolute / Test

My conclusion from looking at the number of nodes is that 500 nodes per layer is the optimum result.

Conclusions

Both parameters can be further optimized using the same parametric optimization.  For the scope of this work I am happy to work with the results of these experiments.

I trained a final model using the optimal parameters.  A two layer & 500 node network achieved a test MASE of 0.455 (versus the previous best of 0.477).

Table 1 – results of the final model fitted (two layers, 500 nodes per layer)

The next post in this series will look at controlling overfitting via dropout.

CHP Scoping Model v0.1

The most recent version of this model can be found here.

My motivation for producing this model is to give engineers something to dig their teeth into.

My first few months as an energy engineering graduate were spent pulling apart the standard CHP model used by my company.  A few years later I was training other technical teams how to use my own model.

I learnt a huge amount through deconstructing other peoples models and iterating through versions of my own.

So this model is mostly about education. I would love it to be used in a professional setting – but we may need a couple of releases to iron out the bugs!

In this post I present the beta version of a CHP feasibility model.  The model takes as inputs (all on a half hourly basis):

  • High grade heat demand (Model : Column F).
  • Low grade heat demand (Model : Column G).
  • Electricity demand (Model : Column H).
  • Gas, import & export electricity price (Model : Column BF-BH).

Features of the model:

  • CHP is modeled as a linear function of load. Load can be varied from 50-100 %.
  • Can model either gas turbines or gas engines. No ability to model supplementary firing.  Engine library needs work.
  • The CHP operating profile can be optimized using a parametric optimization written in VBA.
    • Iteratively increases engine load from 50% – 100% (single HH period),
    • Keeps value that increased annual saving the most (the optimum),
    • Moves to next HH period,
    • Optimization can be started using button on (Model : Column BV). I reccomend watching it run to try understand it.  The VBA routine is called parametric.
  • Availability is modeled using a randomly generated column of binary variables (Model : Column C).
You can download the latest version of the CHP scoping model here.

Thanks for reading!