Schedule
The schedule is still being finalized and is subject to major changes.
Given the pace of innovations in this area, the following list is subject to change.
Color Legend: Presenter Reviewer Scriber
Introduction
- Aug 18
- Course Introduction
- Anand
- Paper Presentation Preferences Fill out the form here
- Aug 20
- Topics, Challenges & Tips
- Anand
- 📖 How to Read a Paper
- 📖 How to Give a Bad Talk
- 📖 Writing Reviews for Systems Conferences
- 📖 Challenges and Applications of Large Language Models
- 📖 An Open Source Stack for AI Compute
- Aug 22
- Paper Presentation Preferences Due
Basics & Project
- Aug 25
- Background
- Vima
- 📖 Attention Is All You Need
- 📖 The Illustrated Transformer
- 📖 The Illustrated Stable Diffusion
- 📖 Multimodality and Large Multimodal Models
- 📖 Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
- Aug 27
- Project Logistics
- Anand
- 📖 How to Write a Great Research Paper
- 📖 Hints and Principles for Computer System Design
- Aug 29
- Project Groups Formation Due
- Sep 1
- No class Labor day
Pre-training
- Sep 3
- Parallelism Strategies
- 📖 Efficient large-scale language model training on GPU clusters using megatron-LM Required
- 📖 WLB-LLM: Workload-Balanced 4D Parallelism for Large Language Model Training Required
- 📖 Zero Bubble (Almost) Pipeline Parallelism
- Sep 5
- Project Proposal Due
- Sep 8
- Dynamic Switching
- 📖 Enabling Parallelism Hot Switching for Efficient Training of Large Language Models Required
- 📖 Tenplex: Dynamic Parallelism for Deep Learning using Parallelizable Tensor Collections Required
- Sep 10
- Dealing with Issues
- 📖 Understanding Stragglers in Large Model Training Using What-if Analysis
- 📖 Training with Confidence: Catching Silent Errors in Deep Learning Training with Automated Proactive Checks Required
- 📖 SuperBench: Improving Cloud AI Infrastructure Reliability with Proactive Validation Required
- Sep 15
- Resiliency
- 📖 Oobleck: Resilient Distributed Training of Large Models Using Pipeline Templates Required
- 📖 ReCycle: Resilient Training of Large DNNs using Pipeline Adaptation Required
Post-Training
- Sep 17
- Parameter Efficient Fine Tuning
- 📖 LoRA: Low-Rank Adaptation of Large Language Models Required
- 📖 QLoRA: Efficient Finetuning of Quantized LLMs Required
- Sep 22
- Test-time Scaling
- 📖 DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning Required
- 📖 Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters Required
- 📖 Efficient Reasoning Models: A Survey
- 📖 A Survey of Reasoning with Foundation Models
- Sep 24
- RL
- 📖 HybridFlow: A Flexible and Efficient RLHF Framework Required
- 📖 Optimizing RLHF Training for Large Language Models with Stage Fusion Required
- 📖 AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning
- Sep 29
- Adaptive Representation
- 📖 Flextron: Many-in-One Flexible Large Language Model Required
- 📖 Matryoshka Representation Learning Required
- 📖 Confident Adaptive Language Modeling
- 📖 Apparate: Rethinking Early Exits to Tame Latency-Throughput Tensions in ML Serving
- Oct 1
- Mid-Semester Project Presentations
- Oct 6
- No class Recess
- Oct 8
- Mid-Semester Project Presentations
Inference
- Oct 13
- Single Instance Serving
- 📖 NanoFlow: Towards Optimal Large Language Model Serving Throughput Required
- 📖 DistServe: Disaggregating Prefill and Decoding for Goodput-Optimized Large Language Model Serving Required
- 📖 Improving DNN Inference Throughput Using Practical, Per-Input Compute Adaptation
- 📖 Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve
- 📖 Orca: A Distributed Serving System for Transformer-Based Generative Models
- 📖 Efficient Memory Management for Large Language Model Serving with PagedAttention
- Oct 15
- Multi Instance Serving
- 📖 Llumnix: Dynamic Scheduling for Large Language Model Serving Required
- 📖 Mooncake: Trading More Storage for Less Computation — A KVCache-centric Architecture for Serving LLM Chatbot Required
- 📖 BlitzScale: Fast and Live Large Model Autoscaling with O(1) Host Caching
- 📖 AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving
- Oct 20
- Diffusion
- 📖 Approximate Caching for Efficiently Serving Text-to-Image Diffusion Models Required
- 📖 DiffServe: Efficiently Serving Text-to-Image Diffusion Models with Query-Aware Model Scaling Required
- Oct 22
- Multimodality
- 📖 ModServe: Scalable and Resource-Efficient Large Multimodal Model Serving Required
- 📖 DREAM: A Dynamic Scheduler for Dynamic Real-Time Multi-Model ML Workloads Required
- Oct 27
- Multimodality - Vision I
- 📖 LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models Required
- 📖 A Stitch in Time Saves Nine: Small VLM is a Precise Guidance for Accelerating Large VLMs
- 📖 An Image is Worth 1/2 Tokens After Layer 2: Plug-and-PLay Acceleration for VLLM Inference Required
- 📖 Beyond Text-Visual Attention: Exploiting Visual Cues for Effective Token Pruning in VLMs
- 📖 [CLS] Token Tells Everything Needed for Training-free Efficient MLLMs
- Oct 29
- Multimodality - Vision II
- 📖 MovieChat: From Dense Token to Sparse Memory for Long Video Understanding Required
- 📖 ViLA: Efficient Video-Language Alignment for Video Question Answering
- 📖 M-LLM Based Video Frame Selection for Efficient Video Understanding Required
Agentic Systems
- Nov 3
- Workflow Optimization I
- 📖 Parrot: Efficient Serving of LLM-based Applications with Semantic Variable Required
- 📖 Cognify: Supercharging Gen-AI Workflows With Hierarchical Autotuning Required
- Nov 5
- Workflow Optimization II
- 📖 DroidSpeak: KV Cache Sharing for Cross-LLM Communication and Multi-LLM Serving Required
- 📖 Towards End-to-End Optimization of LLM-based Applications with Ayo Required
- 📖 Autellix: An Efficient Serving Engine for LLM Agents as General Programs
- Nov 10
- RAGs
- 📖 METIS: Fast Quality-Aware RAG Systems with Configuration Adaptation Required
- 📖 TeleRAG: Efficient Retrieval-Augmented Generation Inference with Lookahead Retrieval
- 📖 RAGO: Systematic Performance Optimization for Retrieval-Augmented Generation Serving Required
- 📖 LEANN: A Low-Storage Vector Index
- Nov 12
- Reasoning
- 📖 Efficiently Serving LLM Reasoning Programs with Certaindex Required
- 📖 ReAct: Synergizing Reasoning and Acting in Language Models Required
- Nov 17
- Applications I
- 📖 Curie: Toward Rigorous and Automated Scientific Experimentation with AI Agents Required
- 📖 Mathematical Discoveries from Program Search with Large Language Models (FunSearch)
- 📖 AlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery Required
- Nov 19
- Applications II
- 📖 NetLLM: Adapting Large Language Models for Networking Required
- 📖 TextGrad: Automatic “Differentiation” via Text Required
- 📖 Building AI Agents for Autonomous Clouds: Challenges and Design Principles
Hardware
- Nov 24
- Infrastructure Considerations for AI
- Vima
- 📖 WaferLLM: Large Language Model Inference at Wafer Scale
- 📖 Insights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architectures
- 📖 USHER: Holistic Interference Avoidance for Resource Optimized ML Inference
- 📖 FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving
- 📖 Meta’s Second Generation AI Chip: Model-Chip Co-Design and Productionization Experiences
Conclusion
- Nov 24
- Course Wrap-Up
- Anand
- Dec 1
- Final Project Poster Presentation
- Dec 8
- Final Project Report + Code Due