Optimization Algorithms

Neural network

Python

NumPy

Logistic Regression

Batch Normalization

Generative adversarial network (GAN)

Shallow Neural Network

Deep Neural Network

Initialization

Regularization (mathematics)

Gradient Checking

Optimization

Hyperparameter Tuning

TensorFlow

Deep Convolutional Models

Residual Networks

Detection Algorithms

Face Recognition

Neural Style Transfer

You Only Look Once (YOLO)

ConvNets

Image Segmentation

U-Net

Recurrent neural network (RNN)

Long Short-Term Memory (LSTM)

Natural Language Processing (NLP)

Word Embeddings

Attention (machine learning)

Transformer (machine learning model)

Deep Reinforcement Learning

Neural Machine Translation

Trigger Word Detection

Transfer learning

Sequence-to-sequence (Seq2Seq)

CS 230 Deep Learning Stanford University Fall 2022 An in-depth course focused on building neural networks and leading successful machine learning projects. It covers Convolutional Networks, RNNs, LSTM, Adam, Dropout, BatchNorm, Xavier/He initialization, and more. Students are expected to have basic computer science skills, probability theory knowledge, and linear algebra familiarity.  Deep Learning is one of the most highly sought after skills in AI. In this course, you will learn the foundations of Deep Learning, understand how to build neural networks, and learn how to lead successful machine learning projects. You will learn about Convolutional networks, RNNs, LSTM, Adam, Dropout, BatchNorm, Xavier/He initialization, and more.  Students are expected to have the following background, and are invited to take the [Workera](http://www.workera.ai/) technical assessments prior to the class to self-assess themselves prior to taking the class:

- Knowledge of basic computer science principles and skills, at a level sufficient to write a reasonably non-trivial computer program. This corresponds to a Developing level (or badge) in the “Algorithmic Coding” section on [Workera](http://www.workera.ai/).
- Familiarity with the probability theory (CS 109 or STATS 116), which students can assess by taking the “Data Science” section on [Workera](http://www.workera.ai/).
- Familiarity with linear algebra (MATH 51), which students can assess by taking the “Mathematics” section on [Workera](http://www.workera.ai/). 

CS 230 Deep Learning

Deep Learning

Connectionist Machines

McCullough and Pitt model

Hebb’s learning rule

Rosenblatt’s perceptron

Universal Approximator

Perceptron learning rule

Empirical risk minimization

Gradient descent

Back-propagation

Momentum

Nestorov

Convergence

Learning Rates

RMSProp

Stochastic gradient descent (SGD)

Acceleration

Overfitting

Convolutional neural network (CNN)

Translation Invariance

Cascade Correlation Filters

Bidirectional RNNs

Sequence Prediction

Connectionist Temporal Classification (CTC)

Representations

Autoencoders

Hopfield Networks

Boltzmann Machines

Normalizing Flows

Variational autoencoder (VAE)

Reinforcement learning (RL)

Multi-layer Perceptron

AdaGrad

11-785 Introduction to Deep Learning Carnegie Mellon University Spring 2020 This course provides a comprehensive introduction to deep learning, starting from foundational concepts and moving towards complex topics such as sequence-to-sequence models. Students gain hands-on experience with PyTorch and can fine-tune models through practical assignments. A basic understanding of calculus, linear algebra, and Python programming is required. “Deep Learning” systems, typified by deep neural networks, are increasingly taking over all AI tasks, ranging from language understanding, and speech and image recognition, to machine translation, planning, and even game playing and autonomous driving. As a result, expertise in deep learning is fast changing from an esoteric desirable to a mandatory prerequisite in many advanced academic settings, and a large advantage in the industrial job market.

In this course we will learn about the basics of deep neural networks, and their applications to various AI tasks. By the end of the course, it is expected that students will have significant familiarity with the subject, and be able to apply Deep Learning to a variety of tasks. They will also be positioned to understand much of the current literature on the topic and extend their knowledge through further study.

If you are only interested in the lectures, you can watch them on the YouTube channel listed below.

### Course description from student point of view

The course is well rounded in terms of concepts. It helps us understand the fundamentals of Deep Learning. The course starts off gradually with MLPs and it progresses into the more complicated concepts such as attention and sequence-to-sequence models. We get a complete hands on with PyTorch which is very important to implement Deep Learning models. As a student, you will learn the tools required for building Deep Learning models. The homeworks usually have 2 components which is Autolab and Kaggle. The Kaggle components allow us to explore multiple architectures and understand how to fine-tune and continuously improve models. The task for all the homeworks were similar and it was interesting to learn how the same task can be solved using multiple Deep Learning approaches. Overall, at the end of this course you will be confident enough to build and tune Deep Learning models.  1. We will be using one of several toolkits (the primary toolkit for recitations/instruction is PyTorch). The toolkits are largely programmed in Python. You will need to be able to program in at least one of these languages. Alternately, you will be responsible for finding and learning a toolkit that requires programming in a language you are comfortable with,
1. You will need familiarity with basic calculus (differentiation, chain rule), linear algebra and basic probability. ### Course description from student point of view

The course is well rounded in terms of concepts. It helps us understand the fundamentals of Deep Learning. The course starts off gradually with MLPs and it progresses into the more complicated concepts such as attention and sequence-to-sequence models. We get a complete hands on with PyTorch which is very important to implement Deep Learning models. As a student, you will learn the tools required for building Deep Learning models. The homeworks usually have 2 components which is Autolab and Kaggle. The Kaggle components allow us to explore multiple architectures and understand how to fine-tune and continuously improve models. The task for all the homeworks were similar and it was interesting to learn how the same task can be solved using multiple Deep Learning approaches. Overall, at the end of this course you will be confident enough to build and tune Deep Learning models.

11-785 Introduction to Deep Learning

Mathematical optimization

Optimization Algorithms

2 courses cover this concept

CS 230 Deep Learning

11-785 Introduction to Deep Learning