PyTorch

Text classification

Language Modeling

Vector Embeddings

Finite-state machine

Sequence Labelling

Machine Translation

Sequence-to-sequence (Seq2Seq)

Morphology

Syntax

Conditional Random Fields (CRF)

Semantics

Linguistic Structure Prediction

Neural Machine Translation

Multinomial Logistic Regression

Weighted Finite-State Transducers

CSE 447 and 517 Natural Language Processing University of Washington Winter 2022 This course provides a comprehensive overview of Natural Language Processing (NLP), including core components like text classification, machine translation, and syntax analysis. It offers two project types: implementation problem-solving for CSE 447, and reproducing experiments from recent NLP papers for CSE 517.  Natural language processing (NLP) seeks to endow computers with the ability to intelligently process human language. NLP components are used in conversational agents and other systems that engage in dialogue with humans, automatic translation between human languages, automatic answering of questions using large text collections, the extraction of structured information from text, tools that help human authors, and many, many more. This course will teach you the fundamental ideas used in key NLP components. It is organized into several “greatest hits” topics, each with a more-or-less self-contained lecture and associated readings, problems, and implementation exercises.  The courses are similar in breadth and use the same lecture content. The projects are quite different; 447’s project is a predefined implementation problem that gives teams freedom in developing a solution. It is designed to encourage iterative improvement and an understanding of inherent tradeoffs in building an NLP system. 517’s project is more research-oriented; it asks teams to reproduce experiments in recently published NLP papers. Teams have great flexibility in the choice of a paper to reproduce.

Additionally, there will be differences in the assignments.

CSE 447 and 517 Natural Language Processing

Natural Language Processing

Exploration / Exploitation

Q-learning

Imitation learning

Tabular MDP planning

Policy Search

Tabular RL Policy Evaluation

Batch Reinforcement Learning

Reinforcement learning (RL)

Function approximation

Value Alignment

CS 234: Reinforcement Learning Stanford University Winter 2023 This course offers a solid introduction to the field of reinforcement learning (RL), covering challenges, approaches, and deep RL. Prerequisites include Python proficiency and foundations of machine learning. Students will be able to implement RL algorithms and evaluate them. To realize the dreams and impact of AI requires autonomous systems that learn to make good decisions. Reinforcement learning is one powerful paradigm for doing so, and it is relevant to an enormous range of tasks, including robotics, game playing, consumer modeling and healthcare. This class will provide a solid introduction to the field of reinforcement learning and students will learn about the core challenges and approaches, including generalization and exploration. Through a combination of lectures, and written and coding assignments, students will become well versed in key ideas and techniques for RL. Assignments will include the basics of reinforcement learning as well as deep reinforcement learning — an extremely promising new area that combines deep learning techniques with reinforcement learning. By the end of the class students should be able to:

- Define the key features of reinforcement learning that distinguishes it from AI and non-interactive machine learning (as assessed by the exam).
- Given an application problem (e.g. from computer vision, robotics, etc), decide if it should be formulated as a RL problem; if yes be able to define it formally (in terms of the state space, action space, dynamics and reward model), state what algorithm (from class) is best suited for addressing it and justify your answer (as assessed by the exam).
- Implement in code common RL algorithms (as assessed by the assignments).
- Describe (list and define) multiple criteria for analyzing RL algorithms and evaluate algorithms on these metrics: e.g. regret, sample complexity, computational complexity, empirical performance, convergence, etc (as assessed by assignments and the exam).
- Describe the exploration vs exploitation challenge and compare and contrast at least two approaches for addressing this challenge (in terms of performance, scalability, complexity of implementation, and theoretical guarantees) (as assessed by an assignment and the exam). - Proficiency in Python  
     All class assignments will be in Python. There is a tutorial [here](http://cs231n.github.io/python-numpy-tutorial/) for those who aren't as familiar with Python. If you have a lot of programming experience but in a different language (e.g. C/ C++/ Matlab/ Javascript) you will probably be fine.
- College Calculus, Linear Algebra (e.g. MATH 51, CME 100)  
     You should be comfortable taking derivatives and understanding matrix vector operations and notation.
- Basic Probability and Statistics (e.g. CS 109 or other stats course)  
     You should know basics of probabilities, Gaussian distributions, mean, standard deviation, etc.
- Foundations of Machine Learning  
     We will be formulating cost functions, taking derivatives and performing optimization with gradient descent. Either CS 221 or CS 229 cover this background. Some optimization tricks will be more intuitive with some knowledge of convex optimization. There is no official textbook for the class but a number of the supporting readings will come from:

- Reinforcement Learning: An Introduction, Sutton and Barto, 2nd Edition. This is available for free [here](http://incompleteideas.net/book/the-book-2nd.html) and references will refer to the final pdf version available [here](http://incompleteideas.net/book/RLbook2018.pdf).

Some other additional references that may be useful are listed below:

- Reinforcement Learning: State-of-the-Art, Marco Wiering and Martijn van Otterlo, Eds. \[[link](https://link.springer.com/book/10.1007%2F978-3-642-27645-3)\]
- Artificial Intelligence: A Modern Approach, Stuart J. Russell and Peter Norvig.\[[link](http://aima.cs.berkeley.edu/)\]
- Deep Learning, Ian Goodfellow, Yoshua Bengio, and Aaron Courville. \[[link](http://www.deeplearningbook.org/)\]
- David Silver's course on Reinforcement Learning \[[link](http://www0.cs.ucl.ac.uk/staff/D.Silver/web/Teaching.html)\]

CS 234: Reinforcement Learning

Artificial Intelligence

Meta-learning

Multi-task Learning

Fine-tuning

Black-box Meta-learning

In-context Learning

Optimization-based Meta-learning

Non-Parametric Few-Shot Learning

Unsupervised pre-training

Task Construction

Large-scale Meta-optimization

Variational Inference

Bayesian Meta-learning

Domain Adaptation

Domain Generalization

Lifelong Learning

Transfer learning

CS 330 Deep Multi-Task and Meta Learning Stanford University Fall 2022 This course emphasizes leveraging shared structures in multiple tasks to enhance learning efficiency in deep learning. It provides a thorough understanding of multi-task and meta-learning algorithms with a focus on topics like self-supervised pre-training, few-shot learning, and lifelong learning. Prerequisites include an introductory machine learning course. The course is designed for graduate-level students. While deep learning has achieved remarkable success in many problems such as image classification, natural language processing, and speech recognition, these models are, to a large degree, specialized for the single task they are trained for. This course will cover the setting where there are multiple tasks to be solved, and study how the structure arising from multiple tasks can be leveraged to learn more efficiently or effectively. This includes:

- self-supervised pre-training for downstream few-shot learning and transfer learning
- meta-learning methods that aim to learn efficient learning algorithms that can learn new tasks quickly
- curriculum and lifelong learning, where the problem requires learning a sequence of tasks, leveraging their shared structure to enable knowledge transfer

This is a graduate-level course. By the end of the course, students will be able to understand and implement the state-of-the-art multi-task learning and meta-learning algorithms and be ready to conduct research on these topics.  CS 229 or an equivalent introductory machine learning course is required. 

CS 330 Deep Multi-Task and Meta Learning

Deep Learning

PyTorch

3 courses cover this concept

CSE 447 and 517 Natural Language Processing

CS 234: Reinforcement Learning

CS 330 Deep Multi-Task and Meta Learning