Dr Ferenc Huszár

I'm interested in the principles of machine learning, with a particular focus on modern deep learning methods. My research falls into the following themes:

• Unsupervised Representation Learning: What are the hallmarks of good neural representations of data, and how do we discover these from data without labels? How can we formalize the goals and principles of unsupervised representation learning, of self-supervised learning, of transfer learning? Can we understand why self-supervised learning works so well in vision and NLP? 
• Optimization and Generalization in Deep Learning: Why do deep networks generalise? What is the role of stochastic gradient descent? What is the role of the (over)parametrization of neural networks? Are natural gradient descent methods a good idea for neural network training, from the perspective of generalization?
• Probabilistic foundations: How should we represent uncertainty about models in deep learning? Is Bayesian inference a good principle for deep learning? Can we develop alternative ways of quantifying uncertainty that line up better with our goal? Can we develop practical inference algorithms in prediction-oriented, loss-calibrated situations?
• Causally Robust Deep Learning: How can we devise principled methods that learn causally meaningful representations from observed data, which robustly generarlise under distribution shift or changing policies? Can we identify invariant causal mechanisms using deep learning?