Machine Learning Systems (Spring 2022)
- When: Mondays from 1:00 to 4:00
- Where: Soda 405 (and on zoom with link posted in Piazza).
- Instructor: Joseph E. Gonzalez
- Co-Instructor: Amir Gholami
- Office Hours:: Arrange by email.
- Announcements: Piazza
- Sign-up to Present: Every student should sign-up to present in at least three rows and as different roles each time. Note that the Backup/Scribe presenter may be asked to fill in for one of the other roles with little notice.
- If you have reading suggestions please send a pull request to this course website on Github by modifying the index.md file.
The recent success of AI has been in large part due in part to advances in hardware and software systems. These systems have enabled training increasingly complex models on ever larger datasets. In the process, these systems have also simplified model development, enabling the rapid growth in the machine learning community. These new hardware and software systems include a new generation of GPUs and hardware accelerators (e.g., TPU and Nervana), open source frameworks such as Theano, TensorFlow, PyTorch, MXNet, Apache Spark, Clipper, Horovod, and Ray, and a myriad of systems deployed internally at companies just to name a few. At the same time, we are witnessing a flurry of ML/RL applications to improve hardware and system designs, job scheduling, program synthesis, and circuit layouts.
In this course, we will describe the latest trends in systems designs to better support the next generation of AI applications, and applications of AI to optimize the architecture and the performance of systems. The format of this course will be a mix of lectures, seminar-style discussions, and student presentations. Students will be responsible for paper readings, and completing a hands-on project. For projects, we will strongly encourage teams that contains both AI and systems students.
New Course Format
Two previous versions of this course were offered in Spring 2019, and Fall 2019. The format of this third offering is slightly different. Each week will cover a different research area in AI-Systems. The lecture will be organized around a mini program committee meeting for the weeks readings. Students will be required to submit detailed reviews for a subset of the papers and lead the paper review discussions. For some of the topics, we have also invited prominent researchers for each area and they will present an overview of the field, followed by discussions raised during the “committee meeting”. The goal of this new format is to both build a mastery of the material and also to develop a deeper understanding of how to evaluate and review research and hopefully provide insight into how to write better papers.
This is a tentative schedule. Specific readings are subject to change as new material is published.
( 1 )
Introduction and Course Overview
This lecture will be an overview of the class, requirements, and an introduction to the history of machine learning and systems research.
Detailed candidate project descriptions will be posted shortly. However, students are encourage to find projects that relate to their ongoing research.
Grades will be largely based on class participation and projects. In addition, we will require weekly paper summaries submitted before class.
- Projects: 60%
- Weekly Summaries: 20%
- Class Participation: 20%