Computer Vision

Convolutional neural network (CNN)

Graph neural network (GNN)

Recurrent neural network (RNN)

Long Short-Term Memory (LSTM)

Sequence-to-sequence (Seq2Seq)

Attention (machine learning)

Self-supervision

Autoencoders

Transformer (machine learning model)

Meta-learning

Fine-tuning

Generative Models

Transfer learning

CS 182/282A: Deep Neural Networks UC Berkeley Fall 2022 An advanced course dealing with deep networks in the fields of computer vision, language technology, robotics, and control. It delves into the themes of deep learning, model families, and real-world applications. A strong mathematical background in calculus, linear algebra, probability, optimization, and statistical learning is necessary. Deep Networks have revolutionized computer vision, language technology, robotics and control. They have a growing impact in many other areas of science and engineering, and increasingly, on commerce and society. They do not however, follow any currently known compact set of theoretical principles. In Yann Lecun's words they require "an interplay between intuitive insights, theoretical modeling, practical implementations, empirical studies, and scientific analyses." This is a fancy way of saying “we don’t understand this stuff nearly well enough, but we have no choice but to muddle through anyway.” This course attempts to cover that ground and show you how to muddle through even as we aspire to do more. The goal is to teach a principled course in Deep Learning that serves the diverse needs of our students while also codifying the present understanding of the field. Topics covered may include, but are not limited to:
- Underlying themes of deep learning, including building beyond underlying machine learning concepts like supervised vs unsupervised learning, regression and classification, training/validation/testing, distribution shifts, regularization, the fundamental underlying tradeoffs;
- Defining and training neural networks: features, computation graphs, backpropagation, iterative optimization (SGD, Newton’s Method, Momentum, RMSProp, AdaGrad, Adam), strategies for training (explicit and implicit regularization, batch and layer normalization, weight initialization, gradient clipping, ensembles, dropout), hyperparameter tuning
- Families of contemporary models: fully connected networks, convolutional nets, graph neural nets, recurrent neural nets, transformers
- Problems that utilize neural networks: computer vision, natural language processing, generative models, and others.
- Conducting experiments in a systematic, repeatable way, leveraging and presenting data from experiments to reason about network behavior. This is a graduate-level/advanced undergraduate course about a particular approach to information processing using (simulated) analog circuits where the desired circuit behavior is tuned via optimization involving data since we have no idea how to do hand-tuning at scale. Probabilistic frames are useful to understand what is going on, as well as how we navigate certain design choices. Overall, we expect students to have a strong mathematical background in calculus, linear algebra, probability, optimization, and statistical learning. Berkeley undergraduate courses that can help build maturity include:
- **Calculus**: Math 53 (note: Math 1B or AP Math is not enough)
- **Linear Algebra and Optimization**: EECS 16B and EECS 127/227A is ideal, but EECS 16B alone might be enough if students have complete mastery of that material. Math 110 is also helpful. (note: Math 54 or EECS 16A is required as a minimum, but are not nearly enough.)
- **Probability**: EECS 126, Stat 134, or Stat 140 (note: CS 70 is required at a minimum, but might not be enough for everyone)
- **Statistical Learning**: CS 189/289A or Stat 154 (note: Data 102 is insufficient, even when combined with Data 100.)

*Math 53 and EECS 126 and EECS 127 and CS 189 is the recommended background.*

Prerequisites are not enforced for enrollment, but we encourage you to consider taking some of the classes listed above and save this course for a future semester if you feel shaky on the fundamentals.

The course assumes familiarity with programming in a high-level language with data structures. Homeworks and projects will typically use Python. We encourage you to check out this [tutorial](https://docs.python.org/3/tutorial/) if you haven’t used it before. Students who have taken Berkeley courses like CS 61A and CS 61B are well-prepared for the programming components of the class.

We do not have the staff bandwidth to help students with material that they should have understood before taking this course. If you choose to proceed with this course, you are accepting full responsibility to teach yourself anything in your background that you are missing. We will not be slowing down to accommodate you, and questions pertaining to background material will always have the lowest priority in all course forums. 

CS 182/282A: Deep Neural Networks

Deep Learning

Decision Tree Learning

Margins

Naive Bayes

Maximizing Conditional Likelihood

Kernelizing Algorithms

Primal and Dual Forms

Finite Hypothesis Classes

k-fold cross-validation

Neural network

Objective-Based Clustering

Principal Component Analysis (PCA)

Semi-Supervised Learning

Learning Linear Separators

Maximum Likelihood Estimation (MLE)

Bag of Words

Generalization and Overfitting

Model Selection and Regularization

Support Vector Machine (SVM)

Markov Decision Process (MDP)

Minimizing Squared Error

Deep Networks

Adaboost

Learning Representations

Active Learning

Co-training

Q-learning

Overfitting

Conditional Independence

Gradient descent

Kernelizing Perceptron

Kernelizing SVM

VC Dimension Based Bounds

Linear regression

Backpropagation

Boosting

Hierarchical Clustering

Interactive Learning

Transductive SVM

Value iteration

Bayes' theorem

Perceptron

Maximum A Posteriori (MAP)

Logistic Regression

Kernel (operating system)

Geometric Margins

Sample Complexity

Structural Risk Minimization

Maximizing Data Likelihood

Unsupervised learning

Dimensionality Reduction

Sampling Bias

Reinforcement learning (RL)

Convolution

10-401 Introduction to Machine Learning Carnegie Mellon University Spring 2018 A comprehensive exploration of machine learning theories and practical algorithms. Covers a broad spectrum of topics like decision tree learning, neural networks, statistical learning, and reinforcement learning. Encourages hands-on learning via programming assignments.  Machine Learning is concerned with computer programs that automatically improve their performance through experience (e.g., programs that learn to recognize human faces, recommend music and movies, and drive autonomous robots). This course covers the theory and practical algorithms for machine learning from a variety of perspectives. We cover topics such as decision tree learning, Support Vector Machines, neural networks, boosting, statistical learning methods, unsupervised learning, active leaerning, and reinforcement learning. Short programming assignments include hands-on experiments with various learning algorithms.   - Machine Learning, Tom Mitchell. (optional)
- Pattern Recognition and Machine Learning, Christopher Bishop. (optional)
- Machine Learning: A Probabilistic Perspective, Kevin P. Murphy, [available online](https://ebookcentral.proquest.com/lib/cm/detail.action?docID=3339490), (optional)

10-401 Introduction to Machine Learning

Machine Learning

Intelligent agent

Search

Game theory

Adversarial Search

Knowledge representation and reasoning

Logic

Uncertain Knowledge

Bayesian network

Hidden Markov Model (HMM)

Robot Motion Planning

Classical Planning

Probabilistic Planning

Supervised learning

Natural Language Processing (NLP)

Robotics

Philosophy of AI

Social and Ethical Issues in AI

Ethical Dilemmas in AI

CS1410 Artificial Intelligence Brown University Fall 2022 CS1410 at Brown University delves into the realm of Artificial Intelligence. Using the 3rd edition of "Artificial Intelligence, A Modern Approach" by Russell & Norvig, students explore intelligent agents, game theory, knowledge representation, logic, probabilistic learning, NLP, robotics, computer vision, and ethical implications of AI.   Prerequisites to CS1410 include (CSCI 0160, 0180, 0190, or 0200) and (CSCI 0220, 1450, 0450, APMA 1650 or 1655) or minimum score of WAIVE in 'Graduate Student PreReq'. ### Required Text

*Artificial Intelligence, A Modern Approach* Russell & Norvig, **3rd** Edition.

### On Lectures

The textbook contains everything you need to know.

Lectures contain everything you need to know.

Lecture notes **do not contain everything you need to
know**.

CS1410 Artificial Intelligence

Artificial Intelligence

Computer vision

Computer Vision

3 courses cover this concept

CS 182/282A: Deep Neural Networks

10-401 Introduction to Machine Learning

CS1410 Artificial Intelligence