Alice in the autodiff wonderland

This seminar takes place within the lecture of Advanced Topics in Machine Learning

Speaker: Simone Scadapane, Sapienza University, Rome

Abstract: Automatic differentiation (autodiff) is at the heart of the deep learning "magic", and it is also powering advances in multiple fields ranging from visual rendering to quantum chemistry. In the first part of this practical tutorial, we show some fundamental ideas from the autodiff field, and how they are implemented in several common frameworks, including TensorFlow, PyTorch, and JAX. In the second part, we show instead how to implement a custom PyTorch-like autodiff library from scratch. We conclude with some trends and advanced tools from the autodiff world, e.g., stateless models in PyTorch.

Biography: Simone Scardapane is a tenure-track assistant professor at Sapienza University of Rome. His research is focused on graph neural networks, explainability, continual learning and, more recently, modular and efficient deep networks. He has published more than 100 papers on these topics in top-tier journals and conferences. Currently, he is an associate editor for the IEEE Transactions on Neural Networks and Learning Systems (IEEE), Neural Networks (Elsevier), Industrial Artificial Intelligence (Springer), and Cognitive Computation (Springer). He is a member of multiple groups and societies, including the ELLIS society, the IEEE Task Force on Reservoir Computing, the “Machine learning in geodesy” joint study group of the International Association of Geodesy, and the Statistical Pattern Recognition Techniques TC of the International Association for Pattern Recognition."

Host: Prof. Cesare Alippi