Regularization (mathematics)

Neural network

Python

NumPy

Logistic Regression

Batch Normalization

Generative adversarial network (GAN)

Shallow Neural Network

Deep Neural Network

Optimization Algorithms

Initialization

Gradient Checking

Optimization

Hyperparameter Tuning

TensorFlow

Deep Convolutional Models

Residual Networks

Detection Algorithms

Face Recognition

Neural Style Transfer

You Only Look Once (YOLO)

ConvNets

Image Segmentation

U-Net

Recurrent neural network (RNN)

Long Short-Term Memory (LSTM)

Natural Language Processing (NLP)

Word Embeddings

Attention (machine learning)

Transformer (machine learning model)

Deep Reinforcement Learning

Neural Machine Translation

Trigger Word Detection

Transfer learning

Sequence-to-sequence (Seq2Seq)

CS 230 Deep Learning Stanford University Fall 2022 An in-depth course focused on building neural networks and leading successful machine learning projects. It covers Convolutional Networks, RNNs, LSTM, Adam, Dropout, BatchNorm, Xavier/He initialization, and more. Students are expected to have basic computer science skills, probability theory knowledge, and linear algebra familiarity.  Deep Learning is one of the most highly sought after skills in AI. In this course, you will learn the foundations of Deep Learning, understand how to build neural networks, and learn how to lead successful machine learning projects. You will learn about Convolutional networks, RNNs, LSTM, Adam, Dropout, BatchNorm, Xavier/He initialization, and more.  Students are expected to have the following background, and are invited to take the [Workera](http://www.workera.ai/) technical assessments prior to the class to self-assess themselves prior to taking the class:

- Knowledge of basic computer science principles and skills, at a level sufficient to write a reasonably non-trivial computer program. This corresponds to a Developing level (or badge) in the “Algorithmic Coding” section on [Workera](http://www.workera.ai/).
- Familiarity with the probability theory (CS 109 or STATS 116), which students can assess by taking the “Data Science” section on [Workera](http://www.workera.ai/).
- Familiarity with linear algebra (MATH 51), which students can assess by taking the “Mathematics” section on [Workera](http://www.workera.ai/). 

CS 230 Deep Learning

Deep Learning

Linear Algebra

Gaussian Discriminant Analysis

Support Vector Machine (SVM)

Expectation Maximization

Unsupervised learning

Prompting

Exponential family

Weighted Least Squares

Generalized Linear Models

Kernel (operating system)

Feature / Model Selection

Factor Analysis

Self-supervised Learning

Instruct Tuning

Supervised learning

Newton's Method

Probability

Bias - Variance

Gaussian Mixture Models (GMM)

Independent Component Analysis (ICA)

In-context Learning

Multi-class Classification

Backpropagation

K-means clustering

Principal Component Analysis (PCA)

Large Language Model (LLM)

Reinforcement Learning from Human Feedback (RLHF)

CS 229: Machine Learning Stanford University Winter 2023 This comprehensive course covers various machine learning principles from supervised, unsupervised to reinforcement learning. Topics also touch on neural networks, support vector machines, bias-variance tradeoffs, and many real-world applications. It requires a background in computer science, probability, multivariable calculus, and linear algebra.  This course provides a broad introduction to machine learning and statistical pattern recognition. Topics include: supervised learning (generative/discriminative learning, parametric/non-parametric learning, neural networks, support vector machines); unsupervised learning (clustering, dimensionality reduction, kernel methods); learning theory (bias/variance tradeoffs, practical advice); reinforcement learning and adaptive control. The course will also discuss recent applications of machine learning, such as to robotic control, data mining, autonomous navigation, bioinformatics, speech recognition, and text and web data processing.  Students are expected to have the following background:

- Knowledge of basic computer science principles and skills, at a level sufficient to write a reasonably non-trivial computer program in Python/NumPy. (CS106A or CS106B, CS106X.)
- Familiarity with probability theory. (CS 109, MATH151, or STATS 116) 
- Familiarity with multivariable calculus and linear algebra (relevant classes include, but not limited to MATH 51, MATH 104, MATH 113, CS 205, CME 100.) Stanford Math 51 course text can be found [here](https://web.stanford.edu/class/math51/stanford/math51book.pdf). 

CS 229: Machine Learning

Machine Learning

Latent Factor Models

Policy Iteration

Feature (machine learning)

Linear classification

Hierarchical Clustering

Reinforcement learning (RL)

Linear regression

Cross-validation (statistics)

Planning

Basis Functions

Kernel-based Classification

Principal Component Analysis

Markov Decision Process (MDP)

Overfitting

Singular Value Decomposition (SVD)

Value iteration

COS 324: Introduction to Machine Learning Princeton University Spring 2019 This introductory course focuses on machine learning, probabilistic reasoning, and decision-making in uncertain environments. A blend of theory and practice, the course aims to answer how systems can learn from experience and manage real-world uncertainties.  This course provides a broad introduction to machine learning, probabilistic reasoning and decision making in uncertain environments. The course should be of interest to undergraduate students in computer science, applied mathematics, sciences and engineering, and lower-level graduate students looking to gain an introduction to the tools of machine learning and probabilistic reasoning with applications to data-intensive problems in the applied sciences, natural sciences and social sciences.

For students with interests in the fundamentals of machine learning and probabilistic artificial intelligence, this course will address three central, related questions in the design and engineering of intelligent systems. How can a system process its perceptual inputs in order to obtain a reasonable picture of the world? How can we build programs that learn from experience? How can we design systems to deal with the inherent uncertainty in the real world?

Our approach to these questions will be both theoretical and practical. We will develop a mathematical underpinning for the methods of machine learning and probabilistic reasoning. We will look at a variety of successful algorithms and applications. We will also discuss the motivations behind the algorithms, and the properties that determine whether or not they will work well for a particular task.  Students should be comfortable with writing non-trivial programs in Python. Students should have a background in basic probability theory, and some level of mathematical sophistication, including calculus and linear algebra. There is no required textbook for the course. This course has its own notes that are considered the required reading. Nevertheless, people learn in different ways and seeing the material presented in different formats can be valuable. To that end, additional optional material is linked on the course website and several books provide useful additional reading:

- Kevin Murphy. *Machine Learning: A Probabilistic Perspective*. MIT Press. 2012.
- Christopher M. Bishop. *Pattern Recognition and Machine Learning*. Springer. 2011.
- David J.C. MacKay. *Information Theory, Inference, and Learning Algorithms*. Cambridge University
Press. 2003. Freely available online at http://www.inference.org.uk/itila/book.html.
- Trevor Hastie, Robert Tibshirani, and Jerome Friedman. *The Elements of Statistical Learning*.
Springer. 2001. Freely available online at http://www-stat.stanford.edu/~tibs/ElemStatLearn/
- Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani. *An Introduction to Statistical
Learning*. Springer. 2013. Freely available online at http://www-bcf.usc.edu/~gareth/ISL/
- Richard S. Sutton and Andrew G. Barto. *Reinforcement Learning: An Introduction*. MIT Press. 1998.
Freely available online at http://incompleteideas.net/book/the-book-2nd.html

COS 324: Introduction to Machine Learning

Property-preserving lossy compression

Nearest Neighbor

k-d tree

L1 regularization

Matrix compression

Laplacian of a graph

Reservoir sampling

Markov chain Monte Carlo (MCMC)

Majority elements

Distance-preserving compression

PAC Guarantees

Linear-Algebraic Techniques

De-noising

Eigenvectors/eigenvalues

Markov's inequality

The Fourier Perspective

Linear programming

Dimensionality Reduction

Approximate heavy hitters

Locality-Sensitive Hashing (LSH)

MinHash

Matrix completion

Spectral embeddings

Chebyshev‘s inequality

Convolution

Convex optimization

Empirical risk minimization

Bloom filters

Jaccard Similarity

Polynomial embedding

Maximizing variance

Uniqueness of low-rank tensor factorizations

Graph coloring

Importance Sampling

Sparse Vector/Matrix Recovery

Privacy Preserving Computation

Count-min sketch

Euclidean Distance

Johnson-Lindenstrauss Transform

Random projection

Power iteration algorithm

Jenrich's algorithm

Spectral clustering

Markov Chains

Compressed Sensing

Differential privacy

Similarity Search

Lp distance

Generalization

L2 regularization

Spectral Graph Theory

Interpretations of the second eigenvalue

Stationary distributions

Mathematical Programming

CS 168: The Modern Algorithmic Toolbox Stanford University Spring 2022 CS 168 provides a comprehensive introduction to modern algorithm concepts, covering hashing, dimension reduction, programming, gradient descent, and regression. It emphasizes both theoretical understanding and practical application, with each topic complemented by a mini-project. It's suitable for those who have taken CS107 and CS161.  This course will provide a rigorous and hands-on introduction to the central ideas and algorithms that constitute the core of the modern algorithms toolkit. Emphasis will be on understanding the high-level theoretical intuitions and principles underlying the algorithms we discuss, as well as developing a concrete understanding of when and how to implement and apply the algorithms. The course will be structured as a sequence of one-week investigations; each week will introduce one algorithmic idea, and discuss the motivation, theoretical underpinning, and practical applications of that algorithmic idea. Each topic will be accompanied by a mini-project in which students will be guided through a practical application of the ideas of the week. Topics include modern techniques in hashing, dimension reduction, linear and convex programming, gradient descent and regression, sampling and estimation, compressive sensing, and linear-algebraic techniques (principal components analysis, singular value decomposition, spectral techniques).  CS107 and CS161, or permission from the instructor. 

CS 168: The Modern Algorithmic Toolbox

Theoretical Computer Science

Connectionist Machines

McCullough and Pitt model

Hebb’s learning rule

Rosenblatt’s perceptron

Universal Approximator

Perceptron learning rule

Gradient descent

Back-propagation

Momentum

Nestorov

Convergence

Learning Rates

RMSProp

Stochastic gradient descent (SGD)

Acceleration

Convolutional neural network (CNN)

Translation Invariance

Cascade Correlation Filters

Bidirectional RNNs

Sequence Prediction

Connectionist Temporal Classification (CTC)

Representations

Autoencoders

Hopfield Networks

Boltzmann Machines

Normalizing Flows

Variational autoencoder (VAE)

Multi-layer Perceptron

AdaGrad

11-785 Introduction to Deep Learning Carnegie Mellon University Spring 2020 This course provides a comprehensive introduction to deep learning, starting from foundational concepts and moving towards complex topics such as sequence-to-sequence models. Students gain hands-on experience with PyTorch and can fine-tune models through practical assignments. A basic understanding of calculus, linear algebra, and Python programming is required. “Deep Learning” systems, typified by deep neural networks, are increasingly taking over all AI tasks, ranging from language understanding, and speech and image recognition, to machine translation, planning, and even game playing and autonomous driving. As a result, expertise in deep learning is fast changing from an esoteric desirable to a mandatory prerequisite in many advanced academic settings, and a large advantage in the industrial job market.

In this course we will learn about the basics of deep neural networks, and their applications to various AI tasks. By the end of the course, it is expected that students will have significant familiarity with the subject, and be able to apply Deep Learning to a variety of tasks. They will also be positioned to understand much of the current literature on the topic and extend their knowledge through further study.

If you are only interested in the lectures, you can watch them on the YouTube channel listed below.

### Course description from student point of view

The course is well rounded in terms of concepts. It helps us understand the fundamentals of Deep Learning. The course starts off gradually with MLPs and it progresses into the more complicated concepts such as attention and sequence-to-sequence models. We get a complete hands on with PyTorch which is very important to implement Deep Learning models. As a student, you will learn the tools required for building Deep Learning models. The homeworks usually have 2 components which is Autolab and Kaggle. The Kaggle components allow us to explore multiple architectures and understand how to fine-tune and continuously improve models. The task for all the homeworks were similar and it was interesting to learn how the same task can be solved using multiple Deep Learning approaches. Overall, at the end of this course you will be confident enough to build and tune Deep Learning models.  1. We will be using one of several toolkits (the primary toolkit for recitations/instruction is PyTorch). The toolkits are largely programmed in Python. You will need to be able to program in at least one of these languages. Alternately, you will be responsible for finding and learning a toolkit that requires programming in a language you are comfortable with,
1. You will need familiarity with basic calculus (differentiation, chain rule), linear algebra and basic probability. ### Course description from student point of view

The course is well rounded in terms of concepts. It helps us understand the fundamentals of Deep Learning. The course starts off gradually with MLPs and it progresses into the more complicated concepts such as attention and sequence-to-sequence models. We get a complete hands on with PyTorch which is very important to implement Deep Learning models. As a student, you will learn the tools required for building Deep Learning models. The homeworks usually have 2 components which is Autolab and Kaggle. The Kaggle components allow us to explore multiple architectures and understand how to fine-tune and continuously improve models. The task for all the homeworks were similar and it was interesting to learn how the same task can be solved using multiple Deep Learning approaches. Overall, at the end of this course you will be confident enough to build and tune Deep Learning models.

11-785 Introduction to Deep Learning

Classification

Training/Validation/Testing

Perceptron

MNIST

Loss function

Automatic differentiation

Matrix representation of Neural Networks

Graphics Processor (GPU)

Multilayer perceptron

Activation Functions

Language Models

Feedforward neural network

Gated recurrent unit (GRU)

Machine Translation

Scaling Deep Learning Systems

Interpretation of Neural Networks

Deepfake

Deep Q-learning

Policy Gradient Methods

Actor-Critic Methods

Graph neural network (GNN)

CSCI 1470/2470 Deep Learning Brown University Spring 2022 Brown University's Deep Learning course acquaints students with the transformative capabilities of deep neural networks in computer vision, NLP, and reinforcement learning. Using the TensorFlow framework, topics like CNNs, RNNs, deepfakes, and reinforcement learning are addressed, with an emphasis on ethical applications and potential societal impacts. Over the past few years, Deep Learning has become a popular area, with deep neural network methods obtaining state-of-the-art results on applications in computer vision (Self-Driving Cars), natural language processing (Google Translate), and reinforcement learning (AlphaGo). These technologies are having transformative effects on our society, including some undesirable ones (e.g. deep fakes).

This course is there to give students a practical understanding of how Deep Learning works, how to implement neural networks, and how to apply them ethically. We introduce students to the core concepts of deep neural networks and survey the techniques used to model complex processes within the contexts of computer vision and natural language processing.

Throughout the course, we emphasize and require students to think critically about potential ethical pitfalls that can result from mis-application of these powerful models. The course is taught using the Tensorflow deep learning framework.
 By the end of this course, you will be able to:

- Learn about the fundamental algorithms that underly all modern deep learning models.
- Implement different types of deep learning models in Tensorflow.
- Think critically about using a deep learning model for a task and its potential societal impact.
- Collaborate with classmates on a team project to apply deep learning models to task of your
choice.
- Communicate your findings (both positive and negative results are encouraged) through pre-
sentations. - A basic programming course: (CSCI 0150, 0170 or 0190)
- A linear algebra course: (CSCI 0530, MATH 0520 or 0540)
- A stats / probability course: (CSCI 0220, 1450, 0450, MATH 1610, APMA 1650 or 1655)

Exceptions may be possible for those missing one of these prerequisites if (a) the student has taken
another course which covers similar material, or if (b) the student will be concurrently taking the
prerequisite. If either of these situations applies to you, use the “Request Override” feature in
Courses@Brown to request an override code (and explain why you believe your situation merits
one). ### Textbook

None required. Students are encouraged to refer to the following textbook, which is available online:

- [Deep Learning](https://www.deeplearningbook.org/), by Ian Goodfellow, Yoshua Bengio, and Aaron Courville.

CSCI 1470/2470 Deep Learning