### Basic Information

- Course: Mathematics of Deep Learning
- Lecture: Monday/Wednesday/Friday, 1:50pm – 2:40pm,
~~Engineering Building 1234~~online via Slack/Zoom - Office hours: Monday – Thursday, 4:00pm – 5:00pm,
~~Engineering Building 2507F~~online via Slack/Zoom - Syllabus

### Reports

**The second report is now optional. Please see my email (sent on April 8) for more details.**

#### Due dates

- Topic idea + 1/2 page abstract + preliminary references: February 14, 2020 (1st report) /
~~April 10, 2020 (2nd report)~~[optional] -
Complete reports: February 28, 2020 (1st report) [required] / April 24, 2020 (2nd report) [optional, unless you did not turn in the first report, in which case the 2nd report is required]

#### Formatting and Length

4 typed pages, using LaTeX, and using the NeurIPS style files with the preprint setting. The style files can be downloaded here.

#### Content

Topic of your choice, must be related to deep learning in some way. Must use at least two sources, one of which must be a recent research paper in the field. For more details, see the syllabus (linked to above).

### Announcements

- [May 08] Handwritten notes for lectures 38-40 posted (sorry for the delay!).
- [Apr 24] Handwritten notes for lectures 35-37 posted.
- [Apr 10] Handwritten notes for lectures 32-34 posted.
- Handwritten notes for lectures 25-31 posted.
- I have resumed posting lecture notes, however starting with lecture 25 they will be handwritten. When I have more time over the summer, I hope to go back and type them up and post typeset versions of the notes.
- Lecture 24 notes are posted. Sorry for the long delay!
**[Mar 16] No lecture on Monday (March 16) and no lecture on Wednesday (March 18). Office hours are cancelled Monday – Wednesday (March 16-18). Office hours (online) will resume Thursday (March 19), and class (via Zoom) will resume on Friday (March 20).**- [Mar 11] Lecture 23 notes posted.
**[Mar 11] There is now a Slack for the class. The link to join has been sent by email. If you did not receive that email, please let me know.****[Mar 11] Class will done remotely via Zoom. More details will be posted here and by email. If you are not officially registered for the class, please email me ASAP so I can add your email to the list.**- [Mar 10] Lectures 21 and 22 notes posted.
- [Feb 26]
**Reminder, the first report is due Friday, February 28! Please email it to me by 11:59pm, AoE (anywhere on earth).** - [Feb 26] All papers referenced in the course can be found here.
- [Feb 26] Lectures 19 and 20 notes posted.
- [Feb 24] Lectures 17 and 18 notes posted and the link to Benjamin Fish’s talk posted.
- [Feb 17] Lecture 16 notes posted.
- [Feb 15] Lecture 15 notes posted.
- [Feb 13] Lecture 14 notes posted and the link to Shahar Kovalsky’s talk posted.
- [Feb 10] Lecture 13 notes posted.
- [Feb 08] Lecture 12 notes and the link to Karianne Bergen’s talk posted.
- [Feb 05] I’ve started the process of grouping the lecture notes together, see the section “Lecture notes grouped by topic!”
- [Feb 05] Lecture 11 notes posted.
- [Feb 03] Lecture 10 notes and the link to Qing Qu’s talk posted!
- [Feb 03] Corrected lecture 09 notes are posted. Thanks to everyone who helped with these!
- [Jan 30] Lecture 08 notes and the link to Murat Kocaoglu’s talk posted!
- [Jan 27] Lecture 07 notes posted.
- [Jan 27] Small update to lecture 06 notes.
- [Jan 26] Lecture 06 notes posted.
- [Jan 23] Link to a video recording of Jose Bento’s talk posted!
- [Jan 23] Lecture 05 notes posted.
- [Jan 23] Small update to lecture 04 notes.
- [Jan 17] No class on Monday, January 20 for Martin Luther King Jr. Day!
- [Jan 15] Lecture 03 posted! It has the complete notes on functional models that we did not finish in class.
- [Jan 15] Small updates to lecture 02 notes. If you’re “???” in footnote 2, let me know!
- [Jan 13] Lecture 01 notes and slides posted.
- [Jan 13] Seminars of interest for report topics posted below!
- [Jan 03] Class will start Friday, January 10. There are no lectures on January 6 and 8!

### Lecture Notes

All lecture notes in one pdf: here

- lecture 01 + slides
- lecture 02 + slides
- lecture 03
- lecture 04
- lecture 05
- lecture 06
- lecture 07
- lecture 08
- lecture 09
- lecture 10
- lecture 11
- lecture 12
- lecture 13
- lecture 14
- lecture 15
- lecture 16
- lecture 17
- lecture 18
- lecture 19
- lecture 20
- lecture 21
- lecture 22
- lecture 23
- lecture 24
- lecture 25
- lectures 26-28
- lectures 29-31
- lectures 32-34
- lectures 35-37
- lectures 38-40

### Lecture Notes Grouped by Topic:

- Prologue: Course introduction (lecture 01)
- Part 1: Background on machine learning and learning theory (lectures 02 – 11)
- Part 2: Artificial neural networks
- Introduction and basics (lectures 11 – 14)
- Classical approximation theory, pre year 2000 (lectures 15 – 20)
- Modern approximation theory, post year 2010 (lectures 21 – 28)

- Part 3: Convolutional neural networks (lectures 29 – 40)
- Introduction and overview of mathematical properties (lectures 29-34)
- Scattering transform models for CNNs (35 – 40)

~~Part 4: Geometric deep learning (TBD)~~(I hope to cover some geometric deep learning in my graph theory course, which will run spring 2021!)~~Part 5: Generative models (TBD)~~

### Papers Referenced in this Course

All papers referenced in this course can be found here.

### Seminars of Interest

#### Deep Learning/Machine Learning Seminars

All deep learning/machine learning seminars are at 10:00am and in the CMSE conference room (1502/1503 EB). The talks will be recorded (with the speaker’s permission) and the links will be posted here once they are available.

- January 23: Jose Bento
- Video of Jose’s talk here.

- January 30: Murat Kocaoglu
- Video of Murat’s talk here.

- February 3: Qing Qu
- Video of Qing’s talk here.

- February 6: Karianne Bergen
- Video of Karianne’s talk here.

- February 10: Shahar Kovalsky
- Video of Shahar’s talk here.

- February 17: Benjamin Fish
- Video of Benjamin’s talk here.

#### CMSE Colloquium

The CMSE colloquium will start February 24, and will be held every Monday at 4:00pm in the CMSE conference room (1502/1503 EB). Not every talk will be on deep learning, but some may be! The schedule is here.