Spring 2020 CMSE 890-002

Basic Information

  • Course: Mathematics of Deep Learning
  • Lecture: Monday/Wednesday/Friday, 1:50pm – 2:40pm, Engineering Building 1234 online via Slack/Zoom
  • Office hours: Monday – Thursday, 4:00pm – 5:00pm, Engineering Building 2507F online via Slack/Zoom
  • Syllabus

Reports

The second report is now optional. Please see my email (sent on April 8) for more details.

Due dates

  • Topic idea + 1/2 page abstract + preliminary references: February 14, 2020 (1st report) / April 10, 2020 (2nd report) [optional]

  • Complete reports: February 28, 2020 (1st report) [required] / April 24, 2020 (2nd report) [optional, unless you did not turn in the first report, in which case the 2nd report is required]

Formatting and Length

4 typed pages, using LaTeX, and using the NeurIPS style files with the preprint setting. The style files can be downloaded here.

Content

Topic of your choice, must be related to deep learning in some way. Must use at least two sources, one of which must be a recent research paper in the field. For more details, see the syllabus (linked to above).

Announcements

  • [May 08] Handwritten notes for lectures 38-40 posted (sorry for the delay!).
  • [Apr 24] Handwritten notes for lectures 35-37 posted.
  • [Apr 10] Handwritten notes for lectures 32-34 posted.
  • Handwritten notes for lectures 25-31 posted.
  • I have resumed posting lecture notes, however starting with lecture 25 they will be handwritten. When I have more time over the summer, I hope to go back and type them up and post typeset versions of the notes.
  • Lecture 24 notes are posted. Sorry for the long delay!
  • [Mar 16] No lecture on Monday (March 16) and no lecture on Wednesday (March 18). Office hours are cancelled Monday – Wednesday (March 16-18). Office hours (online) will resume Thursday (March 19), and class (via Zoom) will resume on Friday (March 20).
  • [Mar 11] Lecture 23 notes posted.
  • [Mar 11] There is now a Slack for the class. The link to join has been sent by email. If you did not receive that email, please let me know.
  • [Mar 11] Class will done remotely via Zoom. More details will be posted here and by email. If you are not officially registered for the class, please email me ASAP so I can add your email to the list.
  • [Mar 10] Lectures 21 and 22 notes posted.
  • [Feb 26] Reminder, the first report is due Friday, February 28! Please email it to me by 11:59pm, AoE (anywhere on earth). 
  • [Feb 26] All papers referenced in the course can be found here.
  • [Feb 26] Lectures 19 and 20 notes posted.
  • [Feb 24] Lectures 17 and 18 notes posted and the link to Benjamin Fish’s talk posted.
  • [Feb 17] Lecture 16 notes posted.
  • [Feb 15] Lecture 15 notes posted.
  • [Feb 13] Lecture 14 notes posted and the link to Shahar Kovalsky’s talk posted.
  • [Feb 10] Lecture 13 notes posted.
  • [Feb 08] Lecture 12 notes and the link to Karianne Bergen’s talk posted.
  • [Feb 05] I’ve started the process of grouping the lecture notes together, see the section “Lecture notes grouped by topic!”
  • [Feb 05] Lecture 11 notes posted.
  • [Feb 03] Lecture 10 notes and the link to Qing Qu’s talk posted!
  • [Feb 03] Corrected lecture 09 notes are posted. Thanks to everyone who helped with these!
  • [Jan 30] Lecture 08 notes and the link to Murat Kocaoglu’s talk posted!
  • [Jan 27] Lecture 07 notes posted.
  • [Jan 27] Small update to lecture 06 notes.
  • [Jan 26] Lecture 06 notes posted.
  • [Jan 23] Link to a video recording of Jose Bento’s talk posted!
  • [Jan 23] Lecture 05 notes posted.
  • [Jan 23] Small update to lecture 04 notes.
  • [Jan 17] No class on Monday, January 20 for Martin Luther King Jr. Day!
  • [Jan 15] Lecture 03 posted! It has the complete notes on functional models that we did not finish in class.
  • [Jan 15] Small updates to lecture 02 notes. If you’re “???” in footnote 2, let me know!
  • [Jan 13] Lecture 01 notes and slides posted.
  • [Jan 13] Seminars of interest for report topics posted below!
  • [Jan 03] Class will start Friday, January 10. There are no lectures on January 6 and 8!

Lecture Notes

All lecture notes in one pdf: here

Lecture Notes Grouped by Topic:

  • Prologue: Course introduction (lecture 01)
  • Part 1: Background on machine learning and learning theory (lectures 02 – 11)
  • Part 2: Artificial neural networks
    • Introduction and basics (lectures 11 – 14)
    • Classical approximation theory, pre year 2000 (lectures 15 – 20)
    • Modern approximation theory, post year 2010 (lectures 21 – 28)
  • Part 3: Convolutional neural networks (lectures 29 – 40)
    • Introduction and overview of mathematical properties (lectures 29-34)
    • Scattering transform models for CNNs (35 – 40)
  • Part 4: Geometric deep learning (TBD) (I hope to cover some geometric deep learning in my graph theory course, which will run spring 2021!)
  • Part 5: Generative models (TBD)

Papers Referenced in this Course

All papers referenced in this course can be found here.

Seminars of Interest

Deep Learning/Machine Learning Seminars

All deep learning/machine learning seminars are at 10:00am and in the CMSE conference room (1502/1503 EB). The talks will be recorded (with the speaker’s permission) and the links will be posted here once they are available.

CMSE Colloquium

The CMSE colloquium will start February 24, and will be held every Monday at 4:00pm in the CMSE conference room (1502/1503 EB). Not every talk will be on deep learning, but some may be! The schedule is here.