printer friendly pageCS 25: Transformers United V5
Since their introduction in 2017, Transformers have revolutionized Deep Learning, powering large language models (LLM) like ChatGPT and DeepSeek, image and video generation (e.g. Dall-E and Sora), and much more. In
CS25, one of Stanford's hottest seminar courses, we delve into Transformer architectures and their diverse applications through classroom discussions and instructor and guest lectures. Topics include LLM, creative uses in art and music, healthcare, neuroscience, robotics, and so forth. We host leading researchers, with past speakers like Andrej Karpathy, Geoffrey Hinton, Jim Fan, Ashish Vaswani, and folks from OpenAI, Anthropic, Google DeepMind, NVIDIA, etc. Lectures are livestreamed and recorded, attracting a global audience with over a million total YouTube views. This is a 1-unit S/NC course, where attendance is the only homework! Enroll via Axess or audit through livestreams or in-person, space permitting. Prerequisites: basic Deep Learning and Transformers knowledge (understanding of attention) or completion of
CS224N,
CS231N, or
CS230. Course website:
https://web.stanford.edu/class/cs25/
Terms: Spr
| Units: 1
CS 129X: Human Centered NLP (CS 329X)
Recent advances in natural language processing (NLP), especially around large pretrained models, have enabled extensive successful applications. However, there are growing concerns about the negative aspects of NLP systems, such as biases and a lack of input from users. This course gives an overview of human-centered techniques and applications for NLP, ranging from human-centered design thinking to human-in-the-loop algorithms, fairness, and accessibility. Along the way, we will cover machine-learning techniques which are especially relevant to NLP and to human experiences. Prerequisite: CS224N or
CS224U, or equivalent background in natural language processing. Prerequisite: CS224N or
CS224U, or equivalent background in natural language processing.
CS 224N: Natural Language Processing with Deep Learning (LINGUIST 284, SYMSYS 195N)
Methods for processing human language information and the underlying computational properties of natural languages. Focus on deep learning approaches: understanding, implementing, training, debugging, visualizing, and extending neural network models for a variety of language understanding tasks. Exploration of natural language tasks ranging from simple word level and syntactic processing to coreference, question answering, and machine translation. Examination of representative papers and systems and completion of a final project applying a complex neural network model to a large-scale NLP problem. Prerequisites: calculus and linear algebra;
CS124,
CS221, or
CS229.
Terms: Win
| Units: 3-4
Instructors:
Hashimoto, T. (PI)
;
Yang, D. (PI)
;
Akoush, B. (TA)
...
more instructors for CS 224N »
Instructors:
Hashimoto, T. (PI)
;
Yang, D. (PI)
;
Akoush, B. (TA)
;
Arora, A. (TA)
;
Bunnapradist, E. (TA)
;
Chang, J. (TA)
;
Cheng, M. (TA)
;
Choi, H. (TA)
;
Deepak, A. (TA)
;
Ding, Z. (TA)
;
Fu, Y. (TA)
;
Gu, C. (TA)
;
Huang, J. (TA)
;
Jiang, M. (TA)
;
Lee, A. (TA)
;
Shaikh, O. (TA)
;
Shao, Y. (TA)
;
Singh, J. (TA)
;
Tao, J. (TA)
;
Varshney, P. (TA)
;
Wang, J. (TA)
;
Wei, A. (TA)
;
Wu, F. (TA)
;
Wu, Z. (TA)
;
Xie, L. (TA)
CS 224S: Spoken Language Processing (LINGUIST 285)
Introduction to spoken language technology with an emphasis on dialogue and conversational systems. Deep learning and other methods for automatic speech recognition, speech synthesis, affect detection, dialogue management, and applications to digital assistants and spoken language understanding systems. Prerequisites:
CS124,
CS221,
CS224N, or
CS229.
Terms: Spr
| Units: 2-4
Instructors:
Maas, A. (PI)
;
Abdullah, S. (TA)
;
Chang, J. (TA)
;
Gupta, A. (TA)
;
Khan, A. (TA)
;
Sun, L. (TA)
;
Weissberg, J. (TA)
CS 329A: Self Improving AI Agents
This graduate seminar course covers the latest techniques and applications of AI agents that can continuously improve themselves through interaction with themselves and the environment. The course will start with self-improvement techniques for LLMs, such as constitutional AI, using learned/domain-specific verifiers, scaling test-time compute, and combining search with LLMs. We will then discuss the latest research in augmenting LLMs with tool use and retrieval techniques, and orchestrating AI capabilities with multimodal web interaction. We will next discuss multi-step reasoning and planning problems for agentic workflows, and the challenges in building robust evaluation and orchestration frameworks. Industry applications that will be discussed include coding agents, research assistants in STEM, robotics and more. Students will work on an original research project in this area, discuss the suggested readings in each class, and learn from invited academic and industry speakers. Prerequisites: CS224N or
CS229S; Fluency in Python programming and using large language model APIs. Application required to get permission number, apply at
https://docs.google.com/forms/d/e/1FAIpQLScRNLBJuIR0eOSWhzO0l6tiSVYq-lYj60RPqSEC4DZoGyGtaQ/viewform Website:
cs329a.stanford.edu
Terms: Win
| Units: 3
CS 329S: Machine Learning Systems Design
This project-based course covers the iterative process for designing, developing, and deploying machine learning systems. It focuses on systems that require massive datasets and compute resources, such as large neural networks. Students will learn about data management, data engineering, approaches to model selection, training, scaling, how to continually monitor and deploy changes to ML systems, as well as the human side of ML projects. In the process, students will learn about important issues including privacy, fairness, and security. Pre-requisites: At least one of the following;
CS229,
CS230,
CS231N, CS224N or equivalent. Students should have a good understanding of machine learning algorithms and should be familiar with at least one framework such as TensorFlow, PyTorch, JAX.
Last offered: Winter 2022
CS 329X: Human Centered NLP (CS 129X)
Recent advances in natural language processing (NLP), especially around large pretrained models, have enabled extensive successful applications. However, there are growing concerns about the negative aspects of NLP systems, such as biases and a lack of input from users. This course gives an overview of human-centered techniques and applications for NLP, ranging from human-centered design thinking to human-in-the-loop algorithms, fairness, and accessibility. Along the way, we will cover machine-learning techniques which are especially relevant to NLP and to human experiences. Prerequisite: CS224N or
CS224U, or equivalent background in natural language processing. Prerequisite: CS224N or
CS224U, or equivalent background in natural language processing.
Terms: Aut
| Units: 3-4
LINGUIST 285: Spoken Language Processing (CS 224S)
Introduction to spoken language technology with an emphasis on dialogue and conversational systems. Deep learning and other methods for automatic speech recognition, speech synthesis, affect detection, dialogue management, and applications to digital assistants and spoken language understanding systems. Prerequisites:
CS124,
CS221,
CS224N, or
CS229.
Terms: Spr
| Units: 2-4
Instructors:
Maas, A. (PI)
;
Abdullah, S. (TA)
;
Chang, J. (TA)
...
more instructors for LINGUIST 285 »
Instructors:
Maas, A. (PI)
;
Abdullah, S. (TA)
;
Chang, J. (TA)
;
Gupta, A. (TA)
;
Khan, A. (TA)
;
Sun, L. (TA)
;
Weissberg, J. (TA)
Filter Results: