Constituency Parsing

Word Vectors

Word Window Classification

Language Models

Backpropagation

Neural network

Dependency Parsing

Recurrent neural network (RNN)

Sequence-to-sequence (Seq2Seq)

Subword Models

Self-Attention

Transformer (machine learning model)

Natural Language Generation

Pretraining

Reinforcement Learning from Human Feedback (RLHF)

Question Answering (QA)

Convolutional neural network (CNN)

Tree Recursive Neural Networks

Code Generation

Multimodal Deep Learning

Coreference Resolution

CS 224N: Natural Language Processing with Deep Learning Stanford University Winter 2023 CS 224N provides an in-depth introduction to neural networks for NLP, focusing on end-to-end neural models. The course covers topics such as word vectors, recurrent neural networks, and transformer models, among others. Natural language processing (NLP) is a crucial part of artificial intelligence (AI), modeling how people share information. In recent years, deep learning approaches have obtained very high performance on many NLP tasks. In this course, students gain a thorough introduction to cutting-edge neural networks for NLP. ## What is this course about?

Natural language processing (NLP) or computational linguistics is one of the most important technologies of the information age. Applications of NLP are everywhere because people communicate almost everything in language: web search, advertising, emails, customer service, language translation, virtual agents, medical reports, politics, etc. In the last decade, deep learning (or neural network) approaches have obtained very high performance across many different NLP tasks, using single end-to-end neural models that do not require traditional, task-specific feature engineering. In this course, students will gain a thorough introduction to cutting-edge research in Deep Learning for NLP. Through lectures, assignments and a final project, students will learn the necessary skills to design, implement, and understand their own neural network models, using the Pytorch framework.

>  “Take it. CS221 taught me algorithms. CS229 taught me math. CS224N taught me how to write machine learning models.” – A CS224N student on Carta - **Proficiency in Python**
    All class assignments will be in Python (using NumPy and PyTorch). If you need to remind yourself of Python, or you're not very familiar with NumPy, you can come to the Python review session in week 1 (listed in the schedule). If you have a lot of programming experience but in a different language (e.g. C/C++/Matlab/Java/Javascript), you will probably be fine.
- **College Calculus, Linear Algebra** (e.g. MATH 51, CME 100)
    You should be comfortable taking (multivariable) derivatives and understanding matrix/vector notation and operations.

- **Basic Probability and Statistics** (e.g. CS 109 or equivalent)
    You should know the basics of probabilities, gaussian distributions, mean, standard deviation, etc.

- **Foundations of Machine Learning** (e.g. CS221, CS229, CS230, or CS124)
    We will be formulating cost functions, taking derivatives and performing optimization with gradient descent. If you already have basic machine learning and/or deep learning knowledge, the course will be easier; however it is possible to take CS224n without it. There are many introductions to ML, in webpage, book, and video form. One approachable introduction is Hal Daumé’s in-progress A Course in Machine Learning. Reading the first 5 chapters of that book would be good background. Knowing the first 7 chapters would be even better! ### Reference Texts

The following texts are useful, but none are required. All of them can be read free online.

- Dan Jurafsky and James H. Martin. [Speech and Language Processing (3rd ed. draft)](https://web.stanford.edu/~jurafsky/slp3/)
- Jacob Eisenstein. [Natural Language Processing](https://github.com/jacobeisenstein/gt-nlp-class/blob/master/notes/eisenstein-nlp-notes.pdf)
- Yoav Goldberg. [A Primer on Neural Network Models for Natural Language Processing](http://u.cs.biu.ac.il/~yogo/nnlp.pdf)
- Ian Goodfellow, Yoshua Bengio, and Aaron Courville. [Deep Learning](http://www.deeplearningbook.org/)
- Delip Rao and Brian McMahan. [Natural Language Processing with PyTorch](http://library.stanford.edu/sfx?genre=book&atitle=&title=Natural%20language%20processing%20with%20PyTorch%20:%20build%20intelligent%20language%20applications%20using%20deep%20learning%20/&isbn=9781491978207&volume=&issue=&date=20190101&aulast=Rao,%20Delip,,%20author.&spage=&pages=&sid=EBSCO:VLeBooks:edsvle.AH35866319) (requires Stanford login).
- Lewis Tunstall, Leandro von Werra, and Thomas Wolf. [Natural Language Processing with Transformers](https://transformersbook.com/)

If you have no background in neural networks but would like to take the course anyway, you might well find one of these books helpful to give you more background:

- Michael A. Nielsen. [Neural Networks and Deep Learning](http://neuralnetworksanddeeplearning.com)
- Eugene Charniak. [Introduction to Deep Learning](https://mitpress.mit.edu/books/introduction-deep-learning)

CS 224N: Natural Language Processing with Deep Learning

Natural Language Processing

N-gram Language Models

Text classification

Word Embeddings

Sequence Models

Machine Translation

Contextualized Representations and Pre-training

Large Language Model (LLM)

Long Short-Term Memory (LSTM)

COS 484: Natural Language Processing Princeton University Spring 2023 This course introduces the basics of NLP, including recent deep learning approaches. It covers a wide range of topics, such as language modeling, text classification, machine translation, and question answering. ### What is this course about?

Recent advances have ushered in exciting developments in natural language processing (NLP), resulting in systems that can translate text, answer questions and even hold spoken conversations with us. This course will introduce students to the basics of NLP, covering standard frameworks for dealing with natural language as well as algorithms and techniques to solve various NLP problems, including recent deep learning approaches. Topics covered include language modeling, representation learning, text classification, sequence tagging, syntactic parsing, machine translation, question answering and others.  - Required: [COS 324](https://www.cs.princeton.edu/courses/archive/fall21/cos324/), knowledge of probability, linear algebra, multivariate calculus.
- Proficiency in Python: programming assignments and projects will require use of Python, Numpy and PyTorch. ### Reading:

There is no required textbook for this class, and you should be able to learn everything from the lectures and assignments. However, if you would like to pursue more advanced topics or get another perspective on the same material, here are some books (all of them can be read free online):

- Dan Jurafsky and James H. Martin. [Speech and Language Processing (3rd ed. draft).](https://web.stanford.edu/~jurafsky/slp3/)
- Jacob Eisenstein. [Natural Language Processing ](https://github.com/jacobeisenstein/gt-nlp-class/blob/master/notes/eisenstein-nlp-notes.pdf)
- Christopher Manning and Hinrich Schütze. [Foundations of Statistical Natural Language Processing](https://nlp.stanford.edu/fsnlp/).

COS 484: Natural Language Processing

Constituency Parsing

2 courses cover this concept

CS 224N: Natural Language Processing with Deep Learning

COS 484: Natural Language Processing