Overview

For the first time in computing, it's possible to embed arbitrary intelligent behavior inside software programs, in particular by providing natural language specifications to pre-trained LLMs. Such LLMs supply broad capabilities, but each AI system we want to build must satisfy requirements that depend on its particular application and context. These requirements must therefore be elicited by engineers, who must elicit and express their application's requirements, some of which can only be stated in natural language, while others must be expressed in code or even learned gradually from feedback on specific decisions by the system itself.

This course studies how to build reliable, efficient, and maintainable intelligent software systems around LLMs. We will begin with how we should model the behavior of foundation models and understand the requirements of situated AI systems, then study declarative abstractions for LLM programming and principles of agent design. From there, we will turn to evaluation, retrieval and reasoning over massive collections, and downstream learning in symbolic programs (e.g., prompts and code) and model weights.

Throughout the course, we will consider trade-offs between the reliability, efficiency, and maintainability of AI systems and will seek out and emphasize durable ideas that outlast individual generations of models. The homeworks and project in this course are deliberately designed around the construction of highly practical, open-ended, and competitive systems.

Course information

Meetings: Wednesdays, 11:00am–1:00pm (first meeting: September 9)
Room: 32-144
Instructor: Omar Khattab
Teaching assistants: Jyothish Pari and Zekai Wang
Units: 2-0-10, graduate
Satisfies: AUS, AAGS in Computer Systems and Artificial Intelligence

Prerequisites

Prerequisites are 6.3900 and either 6.1020, 6.1800, or 6.1810; or permission of instructor. We recommend 6.1210 or equivalent algorithmic maturity.

Course structure

We meet once a week for two hours. Students will complete two individual homeworks and a substantial project. Eight short quizzes will be given in class, with the best five counting toward the grade. These will be held in most meetings from September 16 to November 4.

Models and compute

The course will provide the model endpoints and GPU compute that the homeworks and the project require. We will rely on a combination of free Parley credits and generous support per student from Prime Intellect.

Enrollment

Despite a much larger pool of pre-registered students, enrollment is unfortunately limited to 40 students in this first offering of the course for logistical reasons, including the cost of compute which we'll cover. The course is graduate-level, but well-prepared advanced undergraduates are encouraged to apply. Please complete the short enrollment survey by September 4 at 11:59pm ET. Based on responses, we expect to notify students by the end of September 6th, if not sooner.