CS802 - Deep Learning and Neural Nets

Aims and Objectives

This course will introduce students to the techniques involved in Deep Learning and Neural Nets.

• an understanding of the problem of agents that learn from experience.
• an understanding of the key ideas of reinforcement learning.
• an understanding of the key ideas of convolutional neural networks and deep learning algorithms.
• an understanding of deep reinforcement learning, which is used in modern AI systems such as Google Deepmind’s Alpha Go program.

Learning Outcomes

On completing of the class students will be able to:

• Define and understand the problem of agents that learn in Artificial Intelligence and how this can be done including the ideas of techniques of reinforcement learning;
• Understand how to build Deep Neural Networks in Python using packages such as Tensorflow and implement such networks.
• Understand how Deep Reinforcement Learning uses Deep Neural Networks together with Reinforcement Learning in, for example, the Atari Games work of Google Deepmind.
• Understand how the techniques of Deep Reinforcement Learning are combined with Monte Carlo Tree Search in Google Deepmind’s Alpha Go program.

Syllabus

1. Introduction: what is reinforcement learning?
2. Exact solution problems: Multi-armed bandits, finite Markov Decision Processes.
3. Exact solution methods: dynamic programing, Monte Carolo methods.
4. Temporal difference learning (with applications in games).
5. Planning and learning using exact solution methods.
6. Approximate solution methods for large search spaces.
7. Policy gradient methods and neural networks.
8. Deep neural networks as function approximation methods.
9. Reinforcement learning and deep neural networks: Atari games problem.
10. Deep reinforcement learning and search: Deepmind’s Alpha Go and Alpha Zero.

Recommended Reading

This list is indicative only – the class lecturer may recommend alternative reading material. Please do not purchase any of the reading material listed below until you have confirmed with the class lecturer that it will be used for this class.

Reinforcement Learning: An Introduction. Sutton, R. and Barto, A. 2nd Edition, MIT Press. 2018.

Mastering the Game of Go with Deep Neural Networks and Tree Search. Silver, D, Huang, A., Maddison, C., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvanm, V., Lanctot, M., Dieleman S., Grewe, D., Nham, J., Kalchbrenner, N., Sutskever, I., Lillicrap, T., Leach, M., Kavukcuoglu, K., Graepel, T. and Hassabis, D., Nature 529 (7587): 484–489 (January 2016).

Last updated: 2024-09-05 17:28:53