Results for CS229

As black-box AI models grow increasingly relevant in human-centric applications, explainability and fairness becomes increasingly necessary for trust in adopting AI models. This seminar class introduces students to major problems in AI explainability and fairness, and explores key state-of-theart methods. Key technical topics include surrogate methods, feature visualization, network dissection, adversarial debiasing, and fairness metrics. There will be a survey of recent legal and policy trends. Each week a guest lecturer from AI research, industry, and related policy fields will present an open problem and solution, followed by a roundtable discussion with the class. Students have the opportunity to present a topic of interestnor application to their own projects (solo or in teams) in the final class. Code examples of each topic will be provided for students interested in a particular topic, but there will be no required coding components. Students who will benefit most from this class have exposure to AI, such as through projects and related coursework (e.g. statistics, CS221, CS230, CS229). Students who are pursuing subjects outside of the CS department (e.g. sciences, social sciences, humanities) with sufficient mathematical maturity are welcomed to apply. Enrollment limited to 20.

Methods for processing human language information and the underlying computational properties of natural languages. Focus on deep learning approaches: understanding, implementing, training, debugging, visualizing, and extending neural network models for a variety of language understanding tasks. Exploration of natural language tasks ranging from simple word level and syntactic processing to coreference, question answering, and machine translation. Examination of representative papers and systems and completion of a final project applying a complex neural network model to a large-scale NLP problem. Prerequisites: calculus and linear algebra; CS124, CS221, or CS229.

Introduction to spoken language technology with an emphasis on dialogue and conversational systems. Deep learning and other methods for automatic speech recognition, speech synthesis, affect detection, dialogue management, and applications to digital assistants and spoken language understanding systems. Prerequisites: CS124, CS221, CS224N, or CS229.

Topics: statistical pattern recognition, linear and non-linear regression, non-parametric methods, exponential family, GLMs, support vector machines, kernel methods, deep learning, model/feature selection, learning theory, ML advice, clustering, density estimation, EM, dimensionality reduction, ICA, PCA, reinforcement learning and adaptive control, Markov decision processes, approximate dynamic programming, and policy search. Prerequisites: knowledge of basic computer science principles and skills at a level sufficient to write a reasonably non-trivial computer program in Python/NumPy to the equivalency of CS106A, CS106B, or CS106X, familiarity with probability theory to the equivalency of CS 109, MATH151, or STATS 116, and familiarity with multivariable calculus and linear algebra to the equivalency of MATH51 or CS205.

Sequence data and time series are becoming increasingly ubiquitous in fields as diverse as bioinformatics, neuroscience, health, environmental monitoring, finance, speech recognition/generation, video processing, and natural language processing. Machine learning has become an indispensable tool for analyzing such data; in fact, sequence models lie at the heart of recent progress in AI like GPT3. This class integrates foundational concepts in time series analysis with modern machine learning methods for sequence modeling. Connections and key differences will be highlighted, as well as how grounding modern neural network approaches with traditional interpretations can enable powerful leaps forward. You will learn theoretical fundamentals, but the focus will be on gaining practical, hands-on experience with modern methods through real-world case studies. You will walk away with a broad and deep perspective of sequence modeling and key ways in which such data are not just 1D images.

How do we use mathematical thinking to design better machine learning methods? This course focuses on developing mathematical tools for answering this question. This course will cover fundamental concepts and principled algorithms in machine learning, particularly those that are related to modern large-scale non-linear models. The topics include concentration inequalities, generalization bounds via uniform convergence, non-convex optimization, implicit regularization effect in deep learning, and unsupervised learning and domain adaptations. Prerequisites: linear algebra ( MATH 51 or CS 205), probability theory (STATS 116, MATH 151 or CS 109), and machine learning ( CS 229, STATS 229, or STATS 315A).

Deep learning and neural networks are being increasingly adopted across industries. They are now used to serve billions of users across applications such as search, knowledge discovery, and productivity assistants. As models become more capable and intelligent, this trend of large-scale adoption will continue to grow rapidly. Due to the widespread application, there is an increasing need to achieve high performance for both training and serving deep-learning models. However, performance is hindered by a multitude of infrastructure and lifecycle hurdles - the increasing complexity of the models, massive sizes of training and inference data, heterogeneity of the available accelerators and multi-node platforms, and diverse network properties. The slow adaptation of systems to new algorithms creates a bottleneck for the rapid evolution of deep-learning models and their applications. This course will cover systems approaches for improving the efficiency of machine learning pipelines - comprising data preparation, model training, and model deployment & inference -at each level of the systems stack spanning software and hardware.

Generative Adversarial Networks (GANs) have rapidly emerged as the state-of-the-art technique in realistic image generation. This course presents theoretical intuition and practical knowledge on GANs, from their simplest to their state-of-the-art forms. Their benefits and applications span realistic image editing that is omnipresent in popular app filters, enabling tumor classification under low data schemes in medicine, and visualizing realistic scenarios of climate change destruction. This course also examines key challenges of GANs today, including reliable evaluation, inherent biases, and training stability. After this course, students should be familiar with GANs and the broader generative models and machine learning contexts in which these models are situated. Prerequisites: linear algebra, statistics, CS106B, plus a graduate-level AI course such as: CS230, CS229 (or CS129), or CS221.

Machine learning has become an indispensable tool for creating intelligent applications, accelerating scientific discoveries, and making better data-driven decisions. Yet, the automation and scaling of such tasks can have troubling negative societal impacts. Through practical case studies, you will identify issues of fairness, justice and truth in AI applications. You will then apply recent techniques to detect and mitigate such algorithmic biases, along with methods to provide more transparency and explainability to state-of-the-art ML models. Finally, you will derive fundamental formal results on the limits of such techniques, along with tradeoffs that must be made for their practical application. CS229 or equivalent classes or experience.

The progress of machine learning systems has seemed remarkable and inexorable a wide array of benchmark tasks including image classification, speech recognition, and question answering have seen consistent and substantial accuracy gains year on year. However, these same models are known to fail consistently on atypical examples and domains not contained within the training data. The goal of the course is to introduce the variety of areas in which distributional shifts appear, as well as provide theoretical characterization and learning bounds for distribution shifts. Prerequisites: CS229 or equivalent. Recommended: CS229T (or basic knowledge of learning theory).

This project-based course covers the iterative process for designing, developing, and deploying machine learning systems. It focuses on systems that require massive datasets and compute resources, such as large neural networks. Students will learn about data management, data engineering, approaches to model selection, training, scaling, how to continually monitor and deploy changes to ML systems, as well as the human side of ML projects. In the process, students will learn about important issues including privacy, fairness, and security. Pre-requisites: At least one of the following; CS229, CS230, CS231N, CS224N or equivalent. Students should have a good understanding of machine learning algorithms and should be familiar with at least one framework such as TensorFlow, PyTorch, JAX.

This course will provide an introduction to state-of-the-art ML methods designed to make AI more trustworthy. The course focuses on four concepts: explanations, fairness, privacy, and robustness. We first discuss how to explain and interpret ML model outputs and inner workings. Then, we examine how bias and unfairness can arise in ML models and learn strategies to mitigate this problem. Next, we look at differential privacy and membership inference in the context of models leaking sensitive information when they are not supposed to. Finally, we look at adversarial attacks and methods for imparting robustness against adversarial manipulation.Students will gain understanding of a set of methods and tools for deploying transparent, ethically sound, and robust machine learning solutions. Students will complete labs, homework assignments, and discuss weekly readings. Prerequisites: CS229 or similar introductory Python-based ML class; knowledge of deep learning such as CS230, CS231N; familiarity with ML frameworks in Python (scikit-learn, Keras) assumed.

Deep learning-based AI systems have demonstrated remarkable learning capabilities. A growing field in deep learning research focuses on improving the Fairness, Accountability, and Transparency (FAccT) of a model in addition to its performance. Although FAccT will be difficult to achieve, emerging technical approaches in this topic show promise in making better FAccT AI systems. In this course, we will study the rigorous computer science necessary foundations for FAccT deep learning and dive into the technical underpinnings of topics including fairness, robustness, interpretability, accountability, and privacy. These topics reflect state-of-the-art research in FAccT, are socially important, and they have strong industrial interest due to government and other policy regulation. This course will focus on the algorithmic and statistical methods needed to approach FAccT AI from a deep learning perspective. We will also discuss several application areas where we can apply these techniques. Prerequisites: Intermediate knowledge of statistics, machine learning, and AI. Qualified students will have taken any one of the following, or their advanced equivalents: CS224N, CS230, CS231N, CS236, CS273B. Alternatively, students who have taken CS229 or have equivalent knowledge can be admitted with the permission of the instructors.

Course Description: Generation of high-quality 3D models and scenes by leveraging machine learning tools and approaches. Survey of geometry representations. Public 3D object and scene data sets. Neural architectures for geometry, including deep architectures for point clouds and meshes. Generative models for 3D: autoencoders, GANs, neural implicits, neural ODEs, autoregressive models. Conditional generation based on images or partial geometry. Variation generation. Evaluation metrics for content generation. Use of synthetic data in ML training pipelines. Prerequisites: CS148 and the rudiments of deep learning. Recommended: CS229.

The last ten years has seen a watershed in the development of large-scale neural networks in artificial intelligence. At the same time, computational neuroscientists have discovered a surprisingly robust mapping between the internal components of these networks and real neural structures in the human brain. In this class we will discuss a panoply of examples of such "convergent man-machine evolution", including: feedforward models of sensory systems (vision, audition, somatosensation); recurrent neural networks for dynamics and motor control; integrated models of attention, memory, and navigation; transformer models of language areas; self-supervised models of learning; and deep RL models of decision and planning. We will also delve into the methods and metrics for comparing such models to real-world neural data, and address how unsolved open problems in AI (that you can work on!) will drive forward novel neural models. Some meaningful background in modern neural networks is highly advised (e.g. CS229, CS230, CS231n, CS234, CS236, CS 330), but formal preparation in cognitive science or neuroscience is not needed (we will provide this).

Most successful machine learning algorithms of today use either carefully curated, human-labeled datasets, or large amounts of experience aimed at achieving well-defined goals within specific environments. In contrast, people learn through their agency: they interact with their environments, exploring and building complex mental models of their world so as to be able to flexibly adapt to a wide variety of tasks. One crucial next direction in artificial intelligence is to create artificial agents that learn in this flexible and robust way. Students will read and take turns presenting current works, and they will produce a proposal of a feasible next research direction. Prerequisites: CS229, CS231N, CS234 (or equivalent).

Most successful machine learning algorithms of today use either carefully curated, human-labeled datasets, or large amounts of experience aimed at achieving well-defined goals within specific environments. In contrast, people learn through their agency: they interact with their environments, exploring and building complex mental models of their world so as to be able to flexibly adapt to a wide variety of tasks. One crucial next direction in artificial intelligence is to create artificial agents that learn in this flexible and robust way. Students will read and take turns presenting current works, and they will produce a proposal of a feasible next research direction. Prerequisites: CS229, CS231N, CS234 (or equivalent).

Methods for processing human language information and the underlying computational properties of natural languages. Focus on deep learning approaches: understanding, implementing, training, debugging, visualizing, and extending neural network models for a variety of language understanding tasks. Exploration of natural language tasks ranging from simple word level and syntactic processing to coreference, question answering, and machine translation. Examination of representative papers and systems and completion of a final project applying a complex neural network model to a large-scale NLP problem. Prerequisites: calculus and linear algebra; CS124, CS221, or CS229.

Introduction to spoken language technology with an emphasis on dialogue and conversational systems. Deep learning and other methods for automatic speech recognition, speech synthesis, affect detection, dialogue management, and applications to digital assistants and spoken language understanding systems. Prerequisites: CS124, CS221, CS224N, or CS229.

The last ten years has seen a watershed in the development of large-scale neural networks in artificial intelligence. At the same time, computational neuroscientists have discovered a surprisingly robust mapping between the internal components of these networks and real neural structures in the human brain. In this class we will discuss a panoply of examples of such "convergent man-machine evolution", including: feedforward models of sensory systems (vision, audition, somatosensation); recurrent neural networks for dynamics and motor control; integrated models of attention, memory, and navigation; transformer models of language areas; self-supervised models of learning; and deep RL models of decision and planning. We will also delve into the methods and metrics for comparing such models to real-world neural data, and address how unsolved open problems in AI (that you can work on!) will drive forward novel neural models. Some meaningful background in modern neural networks is highly advised (e.g. CS229, CS230, CS231n, CS234, CS236, CS 330), but formal preparation in cognitive science or neuroscience is not needed (we will provide this).

Methods for processing human language information and the underlying computational properties of natural languages. Focus on deep learning approaches: understanding, implementing, training, debugging, visualizing, and extending neural network models for a variety of language understanding tasks. Exploration of natural language tasks ranging from simple word level and syntactic processing to coreference, question answering, and machine translation. Examination of representative papers and systems and completion of a final project applying a complex neural network model to a large-scale NLP problem. Prerequisites: calculus and linear algebra; CS124, CS221, or CS229.