AI-Sys Machine Learning Systems

The recent success of AI has been in large part due in part to advances in hardware and software systems. These systems have enabled training increasingly complex models on ever larger datasets. In the process, these systems have also simplified model development, enabling the rapid growth in the machine learning community. These new hardware and software systems include a new generation of GPUs and hardware accelerators (e.g., TPU and Nervana), open source frameworks such as Theano, TensorFlow, PyTorch, MXNet, Apache Spark, Clipper, Horovod, and Ray, and a myriad of systems deployed internally at companies just to name a few. At the same time, we are witnessing a flurry of ML/RL applications to improve hardware and system designs, job scheduling, program synthesis, and circuit layouts.

In this course, we will describe the latest trends in systems designs to better support the next generation of AI applications, and applications of AI to optimize the architecture and the performance of systems. The format of this course will be a mix of lectures, seminar-style discussions, and student presentations. Students will be responsible for paper readings, and completing a hands-on project. For projects, we will strongly encourage teams that contains both AI and systems students.

• You have taken previous courses in either
  • Systems (for example CS162) → comfortable building systems and familiar with big ideas in system design
  • Machine Learning (for example CS189) → comfortable training neural networks and familiar with big ideas in ML
• You are actively involved in research in either systems or machine learning (or both)

• Identify and solve impactful problems at the intersection of AI and Systems
• Learn about the big ideas and key results in AI systems
• Learn how to read, evaluate, and peer review papers
• Learn how to write papers that get high review scores

Reasons not to take this class …

• If you want to learn how to train models and use TensorFlow, PyTorch, and SkLearn
  • Take: CS C100, CS182, CS189
• If you want learn how to use big data systems
  • Take: CS162, CS186
• You are not interested in learning how to evaluate, conduct, and communicate research
  • Why not?
• You are unable to attend lecture