AI-Sys

AI-Sys Spring 2019

Course Description

The recent success of AI has been in large part due in part to advances in hardware and software systems. These systems have enabled training increasingly complex models on ever larger datasets. In the process, these systems have also simplified model development, enabling the rapid growth in the machine learning community. These new hardware and software systems include a new generation of GPUs and hardware accelerators (e.g., TPU and Nervana), open source frameworks such as Theano, TensorFlow, PyTorch, MXNet, Apache Spark, Clipper, Horovod, and Ray, and a myriad of systems deployed internally at companies just to name a few. At the same time, we are witnessing a flurry of ML/RL applications to improve hardware and system designs, job scheduling, program synthesis, and circuit layouts.

In this course, we will describe the latest trends in systems designs to better support the next generation of AI applications, and applications of AI to optimize the architecture and the performance of systems. The format of this course will be a mix of lectures, seminar-style discussions, and student presentations. Students will be responsible for paper readings, and completing a hands-on project. Readings will be selected from recent conference proceedings and journals. For projects, we will strongly encourage teams that contains both AI and systems students.

Course Syllabus

This is a tentative schedule. Specific readings are subject to change as new material is published.

Jump to Today

Week  Date (Lec.)  Topic 
1 1/23/19 
( 1 )

Introduction and Course Overview

This lecture will be an overview of the class, requirements, and an introduction to what makes great AI-Systems research.

Slide Links
2 1/28/19 
( 2 )

Convolutional Neural Network Architectures

Minor Update: We have moved the reading on auto-encoders to Wednesday.

Reading notes for the two required readings below must be submitted using this google form by Monday the 28th at 9:30AM. We have asked that for each reading you answer the following questions:

  1. What is the problem that is being solved?
  2. What are the metrics of success?
  3. What are the key innovations over prior work?
  4. What are the key results?
  5. What are some of the limitations and how might this work be improved?
  6. How might this work have long term impact?

If you find some of the reading confusing and want a more gentle introduction, the optional reading contains some useful explanatory blog posts that may help.

Links

Additional Optional Reading
1/30/19 
( 3 )

More Neural Network Architectures

Links

 

Additional Optional Reading
3 2/4/19 
( 4 )

Deep Learning Frameworks

Links

 
2/6/19 
( 5 )

RL Systems & Algorithms

Links

 
4 2/11/19 
( 6 )

Application: Data Structure and Algorithms

Links

 
2/13/19 
( 7 )

Distributed Systems for ML

Links

 
5 2/18/19 
( 8 )

Administrative Holiday (Feb 18th)
2/20/19 
( 9 )

Hyperparameter search

Links

  • Reading Quiz due before class. There was a mix-up in updating the reading and the wrong paper was swapped. You may either read the Hyperband paper (preferred) or the Vizer paper (see optional reading) for the second reading.
  • A Generalized Framework for Population Based Training [pdf]
 
6 2/25/19 
( 10 )

Auto ML & Neural Architecture Search (1/2)

Links

 
2/27/19 
( 11 )

Auto ML & Neural Architecture Search (2/2)

Links

 
7 3/4/19 
( 12 )

Autonomous Vehicles

Links

  • Reading Quiz due before class.
  • Autonomous Vehicles Overview [pdf, pptx]
  • Presentation: The Architectural Implications of Autonomous Driving[pdf]
 
3/6/19 
( 13 )

Deep Learning Compilers

Links

 
8 3/11/19 
( 14 )

Project Presentation Checkpoints
3/13/19 
( 15 )

Application: Program synthesis

Links

 
9 3/18/19 
( 16 )

Distributed Deep Learning (Part 1)

Links

 
3/20/19 
( 17 )

Distributed Deep Learning (Part 2)

Links

 
10 3/25/19 
( 18 )

Spring Break (March 25th)
3/27/19 
( 19 )

Spring Break (March 27th)
11 4/1/19 
( 20 )

Application: Networking

Links

 
4/3/19 
( 21 )

Dynamic Neural Networks

Links

 
12 4/8/19 
( 22 )

Model Compression

Links

 
4/10/19 
( 23 )

Applications: Security

Links

 
13 4/15/19 
( 24 )

Application: Prediction Serving

Links

 
4/17/19 
( 25 )

Natural Language Processing Systems

Links

 
14 4/22/19 
( 26 )

Explanability & Interpretability

Links

 
4/24/19 
( 27 )

Scheduling for DL Workloads

Links

  • Reading Quiz due before class.
  • DL Scheduling slides [pdf]
  • Dominant Resource Fairness (DRF) slides [pdf]
 
15 4/29/19 
( 28 )

Cortical Learning and Stoica Course Summary

Links

 

Additional Optional Reading
5/1/19 
( 29 )

Neural Modular Networks and Gonzalez Course Summary

Links

  • Reading Quiz due before class.
  • Neural Modular Networks Slides [pdf, pptx]
  • Gonzalez Course Summary (Reflections on the Field of AI-Systems) [pdf, pptx]
 
16 5/6/19 
( 30 )

RRR Week (May 6th)
5/8/19 
( 31 )

Poster Session from 9:00 to 11:00
17 5/13/19 
( 32 )

Final Reports Due

  • Due at 11:59 PM
  • Format: 8 pages (Google Doc)
  • Email link to jegonzal@berkeley.edu and istoica@berkeley.edu

Projects

Detailed candidate project descriptions will be posted shortly. However, students are encourage to find projects that relate to their ongoing research.

Grading

Grades will be largely based on class participation and projects. In addition, we will require weekly paper summaries submitted before class.

  • Projects: 60%
  • Weekly Summaries: 20%
  • Class Participation: 20%

Back to top

© 2017-2018 UC Berkeley · Privacy · Terms

最新文章

  1. 【极力分享】[C#/.NET]Entity Framework(EF) Code First 多对多关系的实体增，删，改，查操作全程详细示例【转载自https://segmentfault.com/a/1190000004152660】
  2. html学习第三天—— 第13,14章
  3. opecv获取图像轮廓
  4. Linq 101 工具和源码
  5. 迅为4412开发板支持AVIN视频输入/AV监控摄像头输入模块
  6. i++为什么没有自增探析——JVM中i++的实现(转)
  7. UML系列02之UML类图(1)
  8. leetcode-173：Binary Search Tree Iterator（Java）
  9. java.lang.IllegalArgumentException: Timestamp format must be yyyy-mm-dd hh:mm:ss[.fffffffff]
  10. C语言实现约瑟夫环讨论
  11. 初识Selenium（三）
  12. Android之本地相冊图片选取和拍照以及图片剪辑
  13. 【一天一道LeetCode】#66. Plus One
  14. python爬虫入门（四）利用多线程爬虫
  15. 【PgSQL安装（含配置）】PostgreSQL简称PgSQL，是1980以加利福尼亚大学开发的DBMS，严格遵守标准SQL。
  16. boost.python入门教程 ----python 嵌入c++
  17. 菜鸟手下的iOS开发笔记(swift)
  18. linux常用命令：cp 命令
  19. hibernate 和mybatis
  20. XML 解析的两种方法

热门文章

  1. 【day07】php
  2. SVN版本更新自动通知提醒
  3. Unity 2018 Artificial Intelligence Cookbook Second Edition (Jorge Palacios 著)
  4. QMap::remove操作，并不会调用值的析构，跟QTreeWidget同类，需要主动去释放
  5. 海边拾贝-C-面试篇
  6. LeetCode 142：环形链表 II Linked List Cycle II
  7. LeetCode 189：旋转数组 Rotate Array
  8. C++：Name Lookup & Best Match
  9. Spring-AOP源码分析随手记(一)
  10. My time is limited