=====  Reading Group on Deep Neural Nets =====
^^^
|organizer|** Jean Barbier**|
|office|[[http://plan.epfl.ch/?room=INR139]]|
|phone|**+41 21 6938111**|
|email|**jean.barbier@epfl.ch**|
|||
|room|**INR-113**|
|time|**3:00pm**|
==== Special Announcements ====
\\
==== Objectives ====
\\
After a quick introduction in order to familiarise ourselves with some of basic notions of deep neural nets, we will read and discuss a sequence of fairly recent papers. Our emphasis will be on papers that explain why deep neural networks work rather than on papers that apply neural nets to various problems. If you have suggestions what to read, please mail Jean. Also, if you are willing to present one of the papers, we are always looking for volunteers.
\\

==== Detailed Schedule ====
^ Date ^ Topics Covered  ^ Presenter ^
| April 7 | Introduction | Marco Mondelli ({{:en:courses:2015-2016:notes_intro_to_nn.pdf|Notes}})|
| April 13 | Introduction (bis)  | Marco Mondelli ({{:en:courses:2015-2016:notes_neuralnets_marco.pdf|Notes}})| 
| April 20 | Representation Power of Neural Nets | Olivier Leveque ([[http://ipg.epfl.ch/~leveque/barron_part1.pdf|Notes]]) |
| April 27 | Representation Power of Neural Nets (bis) | Olivier Leveque |
| May 4 | Ascension | |
| May 11 | Iterative Optimization of Neural Nets (ter) | Olivier Leveque |
| May 18 | Break | |
| May 25 | Train Faster, Generalize Better | Ruediger Urbanke |
| June 2 | Train Faster, Generalize Better (bis) | Ruediger Urbanke |
\\
==== Resources ==== 
\\
[[http://neuralnetworksanddeeplearning.com|Michael Nielsen online tutorial on neural nets]] \\

[[https://www.dartmouth.edu/~gvc/Cybenko_MCSS.pdf|"Approximation by Superpositions of a Sigmoidal Function" by G. Cybenko]] \\

[[http://www.stat.yale.edu/~arb4/publications_files/UniversalApproximationBoundsForSuperpositionsOfASigmoidalFunction.pdf|"Universal Approximation Bounds for Superpositions of a Sigmoidal Function" by A. Barron]] \\

[[http://arxiv.org/abs/1310.6343#|"Provable bounds for learning some deep representations" by S. Arora, A. Bhaskara, R. Ge, T. Ma]] \\

[[http://arxiv.org/pdf/1412.0233.pdf|"The Loss Surfaces of Multilayer Networks" by A. Choromanska, M. Henaf, M. Mathieux, G. Ben Arous, and Y. LeCun]] \\

[[http://www.jmlr.org/papers/volume2/bousquet02a/bousquet02a.pdf|"Stability and generalisation" by O. Bousquet and A. Elisseeff]]  \\

[[http://arxiv.org/abs/1509.01240|"Train faster, generalize better: Stability of stochastic gradient descent" by M. Hardt, B. Recht, Y. Singer]]  \\
\\