A Glance at Reinforcement Learning

One of my professional highlights of 2017 has been teaching an introductory reinforcement learning course – A Glance at Reinforcement LearningYou can find the course materials on GitHub.

This one day course is aimed at data scientists with a grasp of supervised machine learning but no prior understanding of reinforcement learning.

Course scope
– introduction to the fundamental concepts of reinforcement learning
– value function methods
dynamic programming, Monte Carlo, temporal difference, Q-Learning, DQN
– policy gradient methods
score function, REINFORCE, advantage actor-critic, AC3
– AlphaGo
– practical concerns
reward scaling, mistakes I’ve made, advice from Vlad Mnih & John Schulman
– literature highlights
distributional perspective, auxiliary loss functions, inverse RL

I’ve given this course to three batches at Data Science Retreat in Berlin and once to a group of startups from Entrepreneur First in London.  Each time I’ve had great questions, kind feedback and improved my own understanding.

I also meet great people – it’s the kind of high-quality networking that is making a difference in my career. I struggle with ‘cold networking’ (i.e. drinks after a Meetup). Teaching and blogging are much better at creating meaningful professional connections.

I’m not an expert in reinforcement learning – I’ve only been studying the topic for a year. I try to use this to my advantage – I can remember what I struggled to understand, which helps design the course to get others up to speed quicker.

If you are looking to develop your understanding of reinforcement learning, the two best places to start are Reinforcement Learning: An Introduction (Sutton & Barto) and David Silver’s lecture series on YouTube.

The course compliments the development of energy_py – an energy-focused reinforcement learning library.

I’d like to thank Jose Quesada and Chris Armbruster for the opportunity to teach at Data Science Retreat.  I’d also like to thank Alex Appelbe and Bashir Beikzadeh of Metis Labs for the opportunity to teach at Entrepreneur First.