Elena and Decebal Mocanu - Scalable Deep Learning Tutorial
A fundamental task for artificial intelligence is learning. Deep Neural Networks have proven to cope perfectly with all learning paradigms, i.e. supervised, unsupervised, and reinforcement learning. Nevertheless, traditional deep learning approaches make use of cloud computing facilities and do not scale well to autonomous agents with low computational resources. Even in the cloud, they suffer from computational and memory limitations. They cannot be used to model adequately large physical worlds for agents which assume networks with billions of neurons. These issues are addressed in the last few years by the emerging topic of scalable deep learning which makes use of static and adaptive sparse connectivity in neural networks before and throughout training (or, on short, sparse training). The tutorial covers these research directions focusing on theoretical advancements, practical applications, and hands-on experience.
Elena Mocanu is Assistant Professor in Machine Learning and Autonomous Agents at University of Twente, Netherlands. She received her B.Sc. degree in Mathematics and Physics from Transilvania University of Brasov, Romania, in 2004. After four years as a mathematics and physics teacher at the high-school level, Elena moved to the university. She has been an Assistant Lecturer within the Department of Information Technology, University of Bucharest, Romania from September 2008 to January 2011. In parallel, in 2009, she started a master program in Theoretical Physics. In 2011 she obtained the M.Sc. degree with a specialization in Quantum Transport from University of Bucharest, Romania. In January 2011, Elena moved from Romania to the Netherlands. She obtained the M.Sc. degree in Operations Research from Maastricht University, The Netherlands, in 2013. In her master thesis, she has investigated deep learning methods for "People detection for building automation" at NXP Semiconductors. In October 2013, Elena started her PhD research in Machine Learning and Smart Grids at TU Eindhoven. In January 2015 she performed a short research visit at the Technical University of Denmark and, from January to April 2016 she was a visiting researcher at University of Texas at Austin, USA. In 2017, Elena received her Doctor of Philosophy degree in Machine learning and Smart Grids from TU/e.