I think this course cs294-ai-sys-sp22 is a very good introductory course on ML systems. I will read the papers in it and write detailed reviews and comments.

Papers

Week 1

SysML: The New Frontier of Machine Learning Systems

• Designing software systems to support the full machine learning lifecycle:
  • Enabling quick “programming” of ML models through emerging interfaces.
  • Supporting development, monitoring, debugging etc. of production ML applications.
  • Defining ML models and systems in higher-level ways.
• Designing hardware systems optimized for machine learning:
  • Developing specialized hardware for ML workloads.
  • Taking advantage of properties like stochasticity for new trade-offs.
  • Distributed systems design for ML.
• Designing ML systems for metrics beyond accuracy:
  • Power efficiency, accessibility, cost, latency, privacy, security, fairness, and interpretability.
  • Techniques like federated learning for privacy/security.
  • Making ML more accessible to non-experts.
  • Optimizations for on-device deployment.

Principles of Computer System Design

1. Introduction:
   • Defines the concept of “system”.
   • Provides a technical definition.
2. Viewpoint Analysis:
   • Purpose, granularity of components.
   • System boundary and interfaces of interest.
3. Five Signs of Complexity:
   • Large number of components and interconnections.
   • Many irregularities.
   • Long description.
   • Team of designers needed.
4. Sources of Complexity:
   • Cascading and interacting requirements.
   • Maintaining high utilization.
5. Coping Techniques:
   • Modularity, abstraction, layering, hierarchy.
6. Computer Systems Distinctions:
   • No physical limits on composition.
   • Unprecedented rate of technology change.
7. Additional Techniques:
   • Iteration and “keep it simple” principle.
8. Themes of the Book:
   • Modularity.
   • Principle-based design.
   • Building robust and resilient systems.