Logistic Regression

Neural network

Python

NumPy

Batch Normalization

Generative adversarial network (GAN)

Shallow Neural Network

Deep Neural Network

Optimization Algorithms

Initialization

Regularization (mathematics)

Gradient Checking

Optimization

Hyperparameter Tuning

TensorFlow

Deep Convolutional Models

Residual Networks

Detection Algorithms

Face Recognition

Neural Style Transfer

You Only Look Once (YOLO)

ConvNets

Image Segmentation

U-Net

Recurrent neural network (RNN)

Long Short-Term Memory (LSTM)

Natural Language Processing (NLP)

Word Embeddings

Attention (machine learning)

Transformer (machine learning model)

Deep Reinforcement Learning

Neural Machine Translation

Trigger Word Detection

Transfer learning

Sequence-to-sequence (Seq2Seq)

CS 230 Deep Learning Stanford University Fall 2022 An in-depth course focused on building neural networks and leading successful machine learning projects. It covers Convolutional Networks, RNNs, LSTM, Adam, Dropout, BatchNorm, Xavier/He initialization, and more. Students are expected to have basic computer science skills, probability theory knowledge, and linear algebra familiarity.  Deep Learning is one of the most highly sought after skills in AI. In this course, you will learn the foundations of Deep Learning, understand how to build neural networks, and learn how to lead successful machine learning projects. You will learn about Convolutional networks, RNNs, LSTM, Adam, Dropout, BatchNorm, Xavier/He initialization, and more.  Students are expected to have the following background, and are invited to take the [Workera](http://www.workera.ai/) technical assessments prior to the class to self-assess themselves prior to taking the class:

- Knowledge of basic computer science principles and skills, at a level sufficient to write a reasonably non-trivial computer program. This corresponds to a Developing level (or badge) in the “Algorithmic Coding” section on [Workera](http://www.workera.ai/).
- Familiarity with the probability theory (CS 109 or STATS 116), which students can assess by taking the “Data Science” section on [Workera](http://www.workera.ai/).
- Familiarity with linear algebra (MATH 51), which students can assess by taking the “Mathematics” section on [Workera](http://www.workera.ai/). 

CS 230 Deep Learning

Deep Learning

Regular Expressions

Minimum Edit Distance

Naive Bayes

Sentiment Analysis

Information Retrieval

Vector Semantics and Embeddings

Sequence Labelling

Chatbots

Web graphs, Links, and PageRank

Social Networks

Language Modeling

Text classification

CS 124: From Languages to Information Stanford University Winter 2023 This course is centered on extracting information from unstructured data in language and social networks using machine learning tools. It covers techniques like sentiment analysis, chatbot development, and social network analysis. The online world has a vast array of unstructured information in the form of language and social networks. Learn how to make sense of it using neural networks and other machine learning tools, and how to interact with humans via language, from answering questions to giving advice! Extracting meaning, information, and structure from human language text, speech, web pages, social networks. Introducing methods (string algorithms, edit distance, language modeling, machine learning, logistic regression, neural networks, neural embeddings, inverted indices, collaborative filtering, PageRank), applications (chatbots, sentiment analysis, information retrieval, text classification, social networks, recommender systems), and ethical issues. CS106B. CS 107 can be helpful, but is fine if you haven't had it, we'll cover the required UNIX material. Math 51 can also be helpful, but isn't required, since we will introduce the basic vectors knowledge we need in the class. #### Textbook

There is no required textbook, but I'll expect you to know the textbook/reading material listed above, and will test it on the midterms.
- Online new chapters from [Jurafsky and Martin. third edition in progress. *Speech and Language Processing*](https://web.stanford.edu/~jurafsky/slp3/).
- Chapters from Manning, Raghavan, and Schutze. 2008. Introduction to Information Retrieval. Cambridge University Press. You can buy the book, get it from the library, or it's also available online [*HERE*](https://site.ebrary.com/lib/stanford/docDetail.action?docID=10240274).

**Is 106B the only prereq? Do I need 109 or 221 before I take CS124?** : 106B is the only prereq. Taking the course as a sophomore is recommended, but we also get lots of juniors and a reasonable number of frosh; the course is designed to be taken early in your Stanford career. It will help if you have at least done some programming beyond 106B, and is also useful to have had 107 or Math 51, but not required; we'll try to give you pointers to places to make up missing background.

**Can I take this course as a non-CS grad student?**: Yes, although this course is not appropriate for CS grad students (there are graduate versions of all the material in this course), it's very commonly taken by PhD students in the social sciences or humanities who plan to use text processing methods in their research.

CS 124: From Languages to Information

Natural Language Processing

Linear Algebra

Gaussian Discriminant Analysis

Support Vector Machine (SVM)

Expectation Maximization

Unsupervised learning

Prompting

Exponential family

Weighted Least Squares

Generalized Linear Models

Kernel (operating system)

Feature / Model Selection

Factor Analysis

Self-supervised Learning

Instruct Tuning

Supervised learning

Newton's Method

Probability

Bias - Variance

Gaussian Mixture Models (GMM)

Independent Component Analysis (ICA)

In-context Learning

Multi-class Classification

Backpropagation

K-means clustering

Principal Component Analysis (PCA)

Large Language Model (LLM)

Reinforcement Learning from Human Feedback (RLHF)

CS 229: Machine Learning Stanford University Winter 2023 This comprehensive course covers various machine learning principles from supervised, unsupervised to reinforcement learning. Topics also touch on neural networks, support vector machines, bias-variance tradeoffs, and many real-world applications. It requires a background in computer science, probability, multivariable calculus, and linear algebra.  This course provides a broad introduction to machine learning and statistical pattern recognition. Topics include: supervised learning (generative/discriminative learning, parametric/non-parametric learning, neural networks, support vector machines); unsupervised learning (clustering, dimensionality reduction, kernel methods); learning theory (bias/variance tradeoffs, practical advice); reinforcement learning and adaptive control. The course will also discuss recent applications of machine learning, such as to robotic control, data mining, autonomous navigation, bioinformatics, speech recognition, and text and web data processing.  Students are expected to have the following background:

- Knowledge of basic computer science principles and skills, at a level sufficient to write a reasonably non-trivial computer program in Python/NumPy. (CS106A or CS106B, CS106X.)
- Familiarity with probability theory. (CS 109, MATH151, or STATS 116) 
- Familiarity with multivariable calculus and linear algebra (relevant classes include, but not limited to MATH 51, MATH 104, MATH 113, CS 205, CME 100.) Stanford Math 51 course text can be found [here](https://web.stanford.edu/class/math51/stanford/math51book.pdf). 

CS 229: Machine Learning

Machine Learning

Informed Search

Decision Networks

Perceptron

Search with Other Agents

Hidden Markov Model (HMM)

Bayes' Nets: Independence

Constraint Satisfaction Problem (CSP)

Value of Information (VPI)

Bayes' Nets: Representation

Markov Decision Process (MDP)

Particle Filtering

Bayes' Nets: Inference

Uninformed Search

Reinforcement learning (RL)

Bayes' Nets: Sampling

CS 188 Introduction to Artificial Intelligence UC Berkeley Fall 2022 UC Berkeley's CS 188 course covers the basic ideas and techniques for designing intelligent computer systems, emphasizing statistical and decision-theoretic modeling. By the course's end, students will have built autonomous agents that can make efficient decisions in a variety of settings. This course will introduce the basic ideas and techniques underlying the design of intelligent computer systems. A specific emphasis will be on the statistical and decision-theoretic modeling paradigm.

By the end of this course, you will have built autonomous agents that efficiently make decisions in fully informed, partially observable and adversarial settings. Your agents will draw inferences in uncertain environments and optimize actions for arbitrary reward structures. Your machine learning algorithms will classify handwritten digits and photographs. The techniques you learn in this course apply to a wide variety of artificial intelligence problems and will serve as the foundation for further study in any application area you choose to pursue.

See the syllabus for slides, deadlines, and the lecture schedule. Readings refer to [fourth edition of AIMA](http://aima.cs.berkeley.edu/) unless otherwise specified.  - **CS 61A or 61B**: Prior computer programming experience is expected (see below)
- **CS 70 or Math 55**: Familiarity with basic concepts of propositional logic and probability are expected (see below)

*CS61A AND CS61B AND CS70 is the recommended background.*

The required math background in the second half of the course will be significantly greater than the first half. The self-diagnostic assignment Homework 0 will help check your preparation.

Course programming assignments will be in Python. We do not assume that students have previous experience with the language, but we do expect you to learn the basics very rapidly. Project 0 is designed to teach you the basics of Python and how the CS 188 submission autograder works. Project 1 is a good representation of the programming level that will be required for subsequent projects in this class. 

CS 188 Introduction to Artificial Intelligence

Artificial Intelligence

Decision Tree Learning

Margins

Maximizing Conditional Likelihood

Kernelizing Algorithms

Primal and Dual Forms

Finite Hypothesis Classes

k-fold cross-validation

Convolutional neural network (CNN)

Objective-Based Clustering

Semi-Supervised Learning

Learning Linear Separators

Maximum Likelihood Estimation (MLE)

Bag of Words

Computer Vision

Generalization and Overfitting

Model Selection and Regularization

Minimizing Squared Error

Deep Networks

Adaboost

Learning Representations

Active Learning

Co-training

Q-learning

Overfitting

Conditional Independence

Gradient descent

Kernelizing Perceptron

Kernelizing SVM

VC Dimension Based Bounds

Linear regression

Boosting

Hierarchical Clustering

Interactive Learning

Transductive SVM

Value iteration

Bayes' theorem

Maximum A Posteriori (MAP)

Geometric Margins

Sample Complexity

Structural Risk Minimization

Maximizing Data Likelihood

Dimensionality Reduction

Sampling Bias

Convolution

10-401 Introduction to Machine Learning Carnegie Mellon University Spring 2018 A comprehensive exploration of machine learning theories and practical algorithms. Covers a broad spectrum of topics like decision tree learning, neural networks, statistical learning, and reinforcement learning. Encourages hands-on learning via programming assignments.  Machine Learning is concerned with computer programs that automatically improve their performance through experience (e.g., programs that learn to recognize human faces, recommend music and movies, and drive autonomous robots). This course covers the theory and practical algorithms for machine learning from a variety of perspectives. We cover topics such as decision tree learning, Support Vector Machines, neural networks, boosting, statistical learning methods, unsupervised learning, active leaerning, and reinforcement learning. Short programming assignments include hands-on experiments with various learning algorithms.   - Machine Learning, Tom Mitchell. (optional)
- Pattern Recognition and Machine Learning, Christopher Bishop. (optional)
- Machine Learning: A Probabilistic Perspective, Kevin P. Murphy, [available online](https://ebookcentral.proquest.com/lib/cm/detail.action?docID=3339490), (optional)

10-401 Introduction to Machine Learning

Counting

Combinatorics

Conditional probability

Independence (probability theory)

Random variable

Expectation

Variance

Bernoulli distribution

Binomial distribution

Poisson distribution

Continuous Random Variables

Normal distribution

Joint probability distribution

Statistical inference

Beta distribution

Central Limit Theorem

P-Value

Algorithmic Analysis

Bootstrapping

CS 109 Probability for Computer Scientists Stanford University Spring 2023 This course offers a thorough understanding of probability theory and its applications in data analysis and machine learning. Prerequisites include CS103, CS106B, and Math 51 or equivalent courses. CS109: Probability for Computer Scientists starts by providing a fundamental grounding in combinatorics, and then quickly moves into the basics of probability theory. We will then cover many essential concepts in probability theory, including common probability distributions, properties of probabilities, and mathematical tools for analyzing probabilities. The last third of the class will focus on data analysis and machine learning as direct applications of probability we've learned in the weeks prior. Read more here to learn what CS109 is all about. This is going to be a great quarter and we are looking forward to the chance to teach you!! After you're finished with CS109, we hope you'll have achieved the following learning goals:

- Reason about situations using probabilities, expectation, and variance.
- Feel comfortable learning about new probability concepts beyond the scope of this class (e.g., through one's own research, studies, or interests).
- Write programs to simulate random experiments and to test experiment hypotheses.
- Understand and implement simple machine learning algorithms like Naive Bayes and Logistic Regression.

## Course Topics

Here are the broad strokes of the course (in approximate order). More information is available on our Schedule page. We cover a very broad set of topics so that you are equipped with the probability and statistics you will see in your future CS studies!

- Counting and probability fundamentals
- Single-dimensional random variables
- Probabilistic models
- Uncertainty theory
- Parameter estimation
- Introduction to machine learning The prerequisites for this course are CS103, CS106B, and Math 51 (or equivalent courses). Probability involves a fair bit of mathematics (set theory, calculus, and familiarity with linear algebra), and we'll be considering several applications of probability in CS that require familiarity with algorithms and data structures covered in CS106B. Here is a quick rundown of some of the mathematical tools from CS103 and Math 51 that we'll be using in this class: multivariate calculus (integration and differentiation), linear algebra (basic operations on vectors and matrices), an understanding of the basics of set theory (subsets, complements, unions, intersections, cardinality, etc.), and familiarity with basic proof techniques. We'll also do combinatorics in the class, but we'll be covering a fair bit of that material ourselves in the first week. Past students have managed to take CS106B concurrently with CS109 and have done just fine. CS103 is the pre-requisite that we rely on the least. See our FAQ for more information. ## Optional Textbook

Sheldon Ross, *A First Course in Probability* (10th Ed.), Pearson Prentice Hall, 2018.

This is an optional textbook, meaning that the text is not required material, but students may find Ross offers a different and useful perspective on the important concepts of the class. Suggested, optional reading assignments from the textbook (10th Ed.) are in the schedule on the course website. The 8th, 9th, and 10th editions of the textbook are all fine for this class.

*Borrowing the textbook online*: HathiTrust, a library archive of which Stanford is a member, has granted the university online access to the 8th edition (2010) for the duration of the Fall quarter. The "check out" system works similarly to print reserves: A user can check out the book an hour at a time as long as they are actively using it. Access guidelines are on the [HathiTrust How To Use It webpage](https://www.hathitrust.org/ETAS-How-To). Once you're logged in, the book is at [this link](https://catalog.hathitrust.org/Record/006904347).

All students should retain receipts for books and other course-related expenses, as these may be qualified educational expenses for tax purposes. If you are an undergraduate receiving financial aid, you may be eligible for additional financial aid for required books and course materials if these expenses exceed the aid amount in your award letter. For more information, review your award letter or visit the [Student Budget website](https://financialaid.stanford.edu/undergrad/budget/index.html).

CS 109 Probability for Computer Scientists

Mathematical Foundations