Deep Learning – Recent Advances in Kernel Methods for Neural Networks
5. Oktober 2023 10:00 Uhr – 6. Oktober 2023 18:00 Uhr
Sparked by the success of deep neural networks in many practical applications, there has been an increasing interest in analyzing these methods from a statistical perspective. In recent years, advances in the theory of neural networks and kernels have brought both fields closer together.
This not only sparked new interest and with that fresh ideas in the field of kernels, it also enabled research to explain phenomena occurring in neural networks. Double descent [Belkin et al., 2018], the usefulness of overparametrization (for example in Autoencoder architectures) [Radhakrishnan et al., 2018] or the Neural Tangent Kernel [Jacot et al., 2018] are among those. Notably, Radhakrishnan and co-authors [Radhakrishnan et al., 2022] showed that neural networks merely learn something called the ‘Expected Gradient Outer Product’ (EGOP). They showed that using the EGOP within a simple kernel framework outperforms not only neural networks but also methods such as Gradient Boosting and Random Forests in most cases.
In this workshop Adit Radhakrishnan will teach the mathematical reasoning behind the above-mentioned methods, but also how they are applied in practical applications (mainly from biology). In interactive Python coding sessions, participants will have the chance to develop these methods themselves. The overarching goal of the workshop therefore is to learn about a mathematically founded way of applying deep learning in practice and to disentangle myth and actual capabilities of deep learning and kernels. The power of these methods is that they make use of mechanisms in neural networks without the overhead of training them. These methods are hence applicable to many practical problems, with or without a plethora of available data.
More information can be found here.
About Adit Radhakrishnan
Adit is a George F. Carrier postdoctoral fellow at Harvard. He completed his Ph.D. in electrical engineering and computer science (EECS) at MIT advised by Caroline Uhler and was a Ph.D. fellow at the Eric and Wendy Schmidt Center at the Broad Institute of MIT and Harvard. He received his M.Eng. in EECS and his Bachelor’s of Science in Math and EECS from MIT. His research focuses on advancing theoretical foundations of machine learning in order to develop new methods for tackling biomedical problems. In his talk at KIT, he will present some of his recent projects, which shed light on some of the mysteries of deep learning that are of interest to both theorists and practitioners.
