Advanced Machine Learning (NWI-NM048B)

• Master course - 6 EC
• Teaching Assistants: Mauricio Diaz-Ortiz

This course provides advanced topics in machine learning, with a particular focus on how the statistical physics of disordered systems helps in understanding learning and generalization abilities of neural networks. The course is intended for Master’s students in physics and mathematics. This course is the follow-up of CDS: Machine Learning.

Main topics

• Monte Carlo methods and variational approximation for inference and learning
• Message passing algorithms
• Statistical physics of machine learning: replica method, phase transitions in learning and generalization
• Boltzmann Machines and Deep Belief Networks
• Infinite width limits in deep networks and theory of kernels
• Modern Hopfield Networks, Attention, Transformers

Course Material

The slides used in the course will be available in pdf format.

Books

• Information Theory, Inference and Learning Algorithms by David MacKay.
• Bayesian Reasoning and Machine Learning by David Barber.

Schedule

TBA

Exam

There will be no final examination. The grade will be based on the exercises and a final presentation of a research paper. You are expected to work in groups of 3 persons and you will be graded as a group. The final grade for each student is his group grade.