Machine Translation

Text classification

Question Answering (QA)

Morphology

Language Modeling

Recurrent neural network (RNN)

Constituency Parsing

Neural network

Natural Language Generation

Dependency Parsing

Sequence Labelling

Word Embeddings

Transformer (machine learning model)

Sequence-to-sequence (Seq2Seq)

N-gram Language Models

Sequence Models

Contextualized Representations and Pre-training

Large Language Model (LLM)

Long Short-Term Memory (LSTM)

COS 484: Natural Language Processing Princeton University Spring 2023 This course introduces the basics of NLP, including recent deep learning approaches. It covers a wide range of topics, such as language modeling, text classification, machine translation, and question answering. ### What is this course about?

Recent advances have ushered in exciting developments in natural language processing (NLP), resulting in systems that can translate text, answer questions and even hold spoken conversations with us. This course will introduce students to the basics of NLP, covering standard frameworks for dealing with natural language as well as algorithms and techniques to solve various NLP problems, including recent deep learning approaches. Topics covered include language modeling, representation learning, text classification, sequence tagging, syntactic parsing, machine translation, question answering and others.  - Required: [COS 324](https://www.cs.princeton.edu/courses/archive/fall21/cos324/), knowledge of probability, linear algebra, multivariate calculus.
- Proficiency in Python: programming assignments and projects will require use of Python, Numpy and PyTorch. ### Reading:

There is no required textbook for this class, and you should be able to learn everything from the lectures and assignments. However, if you would like to pursue more advanced topics or get another perspective on the same material, here are some books (all of them can be read free online):

- Dan Jurafsky and James H. Martin. [Speech and Language Processing (3rd ed. draft).](https://web.stanford.edu/~jurafsky/slp3/)
- Jacob Eisenstein. [Natural Language Processing ](https://github.com/jacobeisenstein/gt-nlp-class/blob/master/notes/eisenstein-nlp-notes.pdf)
- Christopher Manning and Hinrich Schütze. [Foundations of Statistical Natural Language Processing](https://nlp.stanford.edu/fsnlp/).

COS 484: Natural Language Processing

Word Vectors

Word Window Classification

Language Models

Backpropagation

Subword Models

Self-Attention

Pretraining

Reinforcement Learning from Human Feedback (RLHF)

Convolutional neural network (CNN)

Tree Recursive Neural Networks

Code Generation

Multimodal Deep Learning

Coreference Resolution

CS 224N: Natural Language Processing with Deep Learning Stanford University Winter 2023 CS 224N provides an in-depth introduction to neural networks for NLP, focusing on end-to-end neural models. The course covers topics such as word vectors, recurrent neural networks, and transformer models, among others. Natural language processing (NLP) is a crucial part of artificial intelligence (AI), modeling how people share information. In recent years, deep learning approaches have obtained very high performance on many NLP tasks. In this course, students gain a thorough introduction to cutting-edge neural networks for NLP. ## What is this course about?

Natural language processing (NLP) or computational linguistics is one of the most important technologies of the information age. Applications of NLP are everywhere because people communicate almost everything in language: web search, advertising, emails, customer service, language translation, virtual agents, medical reports, politics, etc. In the last decade, deep learning (or neural network) approaches have obtained very high performance across many different NLP tasks, using single end-to-end neural models that do not require traditional, task-specific feature engineering. In this course, students will gain a thorough introduction to cutting-edge research in Deep Learning for NLP. Through lectures, assignments and a final project, students will learn the necessary skills to design, implement, and understand their own neural network models, using the Pytorch framework.

>  “Take it. CS221 taught me algorithms. CS229 taught me math. CS224N taught me how to write machine learning models.” – A CS224N student on Carta - **Proficiency in Python**
    All class assignments will be in Python (using NumPy and PyTorch). If you need to remind yourself of Python, or you're not very familiar with NumPy, you can come to the Python review session in week 1 (listed in the schedule). If you have a lot of programming experience but in a different language (e.g. C/C++/Matlab/Java/Javascript), you will probably be fine.
- **College Calculus, Linear Algebra** (e.g. MATH 51, CME 100)
    You should be comfortable taking (multivariable) derivatives and understanding matrix/vector notation and operations.

- **Basic Probability and Statistics** (e.g. CS 109 or equivalent)
    You should know the basics of probabilities, gaussian distributions, mean, standard deviation, etc.

- **Foundations of Machine Learning** (e.g. CS221, CS229, CS230, or CS124)
    We will be formulating cost functions, taking derivatives and performing optimization with gradient descent. If you already have basic machine learning and/or deep learning knowledge, the course will be easier; however it is possible to take CS224n without it. There are many introductions to ML, in webpage, book, and video form. One approachable introduction is Hal Daumé’s in-progress A Course in Machine Learning. Reading the first 5 chapters of that book would be good background. Knowing the first 7 chapters would be even better! ### Reference Texts

The following texts are useful, but none are required. All of them can be read free online.

- Dan Jurafsky and James H. Martin. [Speech and Language Processing (3rd ed. draft)](https://web.stanford.edu/~jurafsky/slp3/)
- Jacob Eisenstein. [Natural Language Processing](https://github.com/jacobeisenstein/gt-nlp-class/blob/master/notes/eisenstein-nlp-notes.pdf)
- Yoav Goldberg. [A Primer on Neural Network Models for Natural Language Processing](http://u.cs.biu.ac.il/~yogo/nnlp.pdf)
- Ian Goodfellow, Yoshua Bengio, and Aaron Courville. [Deep Learning](http://www.deeplearningbook.org/)
- Delip Rao and Brian McMahan. [Natural Language Processing with PyTorch](http://library.stanford.edu/sfx?genre=book&atitle=&title=Natural%20language%20processing%20with%20PyTorch%20:%20build%20intelligent%20language%20applications%20using%20deep%20learning%20/&isbn=9781491978207&volume=&issue=&date=20190101&aulast=Rao,%20Delip,,%20author.&spage=&pages=&sid=EBSCO:VLeBooks:edsvle.AH35866319) (requires Stanford login).
- Lewis Tunstall, Leandro von Werra, and Thomas Wolf. [Natural Language Processing with Transformers](https://transformersbook.com/)

If you have no background in neural networks but would like to take the course anyway, you might well find one of these books helpful to give you more background:

- Michael A. Nielsen. [Neural Networks and Deep Learning](http://neuralnetworksanddeeplearning.com)
- Eugene Charniak. [Introduction to Deep Learning](https://mitpress.mit.edu/books/introduction-deep-learning)

CS 224N: Natural Language Processing with Deep Learning

Regular Expressions

Minimum Edit Distance

Naive Bayes

Sentiment Analysis

Logistic Regression

Information Retrieval

Vector Semantics and Embeddings

Chatbots

Web graphs, Links, and PageRank

Social Networks

CS 124: From Languages to Information Stanford University Winter 2023 This course is centered on extracting information from unstructured data in language and social networks using machine learning tools. It covers techniques like sentiment analysis, chatbot development, and social network analysis. The online world has a vast array of unstructured information in the form of language and social networks. Learn how to make sense of it using neural networks and other machine learning tools, and how to interact with humans via language, from answering questions to giving advice! Extracting meaning, information, and structure from human language text, speech, web pages, social networks. Introducing methods (string algorithms, edit distance, language modeling, machine learning, logistic regression, neural networks, neural embeddings, inverted indices, collaborative filtering, PageRank), applications (chatbots, sentiment analysis, information retrieval, text classification, social networks, recommender systems), and ethical issues. CS106B. CS 107 can be helpful, but is fine if you haven't had it, we'll cover the required UNIX material. Math 51 can also be helpful, but isn't required, since we will introduce the basic vectors knowledge we need in the class. #### Textbook

There is no required textbook, but I'll expect you to know the textbook/reading material listed above, and will test it on the midterms.
- Online new chapters from [Jurafsky and Martin. third edition in progress. *Speech and Language Processing*](https://web.stanford.edu/~jurafsky/slp3/).
- Chapters from Manning, Raghavan, and Schutze. 2008. Introduction to Information Retrieval. Cambridge University Press. You can buy the book, get it from the library, or it's also available online [*HERE*](https://site.ebrary.com/lib/stanford/docDetail.action?docID=10240274).

**Is 106B the only prereq? Do I need 109 or 221 before I take CS124?** : 106B is the only prereq. Taking the course as a sophomore is recommended, but we also get lots of juniors and a reasonable number of frosh; the course is designed to be taken early in your Stanford career. It will help if you have at least done some programming beyond 106B, and is also useful to have had 107 or Math 51, but not required; we'll try to give you pointers to places to make up missing background.

**Can I take this course as a non-CS grad student?**: Yes, although this course is not appropriate for CS grad students (there are graduate versions of all the material in this course), it's very commonly taken by PhD students in the social sciences or humanities who plan to use text processing methods in their research.

CS 124: From Languages to Information

Vector Embeddings

PyTorch

Finite-state machine

Syntax

Conditional Random Fields (CRF)

Semantics

Linguistic Structure Prediction

Neural Machine Translation

Multinomial Logistic Regression

Weighted Finite-State Transducers

CSE 447 and 517 Natural Language Processing University of Washington Winter 2022 This course provides a comprehensive overview of Natural Language Processing (NLP), including core components like text classification, machine translation, and syntax analysis. It offers two project types: implementation problem-solving for CSE 447, and reproducing experiments from recent NLP papers for CSE 517.  Natural language processing (NLP) seeks to endow computers with the ability to intelligently process human language. NLP components are used in conversational agents and other systems that engage in dialogue with humans, automatic translation between human languages, automatic answering of questions using large text collections, the extraction of structured information from text, tools that help human authors, and many, many more. This course will teach you the fundamental ideas used in key NLP components. It is organized into several “greatest hits” topics, each with a more-or-less self-contained lecture and associated readings, problems, and implementation exercises.  The courses are similar in breadth and use the same lecture content. The projects are quite different; 447’s project is a predefined implementation problem that gives teams freedom in developing a solution. It is designed to encourage iterative improvement and an understanding of inherent tradeoffs in building an NLP system. 517’s project is more research-oriented; it asks teams to reproduce experiments in recently published NLP papers. Teams have great flexibility in the choice of a paper to reproduce.

Additionally, there will be differences in the assignments.

CSE 447 and 517 Natural Language Processing

Information extraction

Words

Language models and smoothing

Lexicons

Noisy channel models

Edit Distance

Part of speech tags

Sequence Tagging

Classification

Deep learning

Syntactic representations of natural language

Chomsky hierarchy and natural language

Treebanks

Lexical semantics

Vector semantics

Contextualized representations (BERT)

Verb / sentence semantics

Compositional semantics

Semantic Parsing

Discourse

Entity linking

Pragmatics

Speech

11-411/611 Natural Language Processing Carnegie Mellon University Spring 2021 Focused on computational systems for human languages, this course introduces various NLP applications, such as translation and summarization. It encompasses a broad scope, from machine learning to linguistics, with a software engineering perspective. This course is about a variety of ways to represent human languages (like English and Chinese) as computational systems, and how to exploit those representations to write programs that do neat stuff with text and speech data, like

- translation,
- summarization,
- extracting information,
- question answering,
- natural interfaces to databases, and
- conversational agents.

This field is called Natural Language Processing or Computational Linguistics, and it is extremely multidisciplinary. This course will therefore include some ideas central to Machine Learning and to Linguistics.

We'll cover computational treatments of words, sounds, sentences, meanings, and conversations. We'll see how probabilities and real-world text data can help. We'll see how different levels interact in state-of-the-art approaches to applications like translation and information extraction.

From a software engineering perspective, there will be an emphasis on rapid prototyping, a useful skill in many other areas of Computer Science. ### Should I take this course?

Yes, if:

- you're a CS student interested in languages, language technology, or information processing
- you're a CS student who needs an "applications" credit
- you're a language technology minor (this course is an elective option)
- you're a linguistics student who can write computer programs (this course is an elective option)
- you always suspected natural language was kind of like Lisp (or Java or ...)
- you want computers to take over the world
- you don't want computers to take over the world, but if they do, you want to negotiate your release
- you like AI, machine learning, and/or theoretical computer science, and want to apply them to a hard real-world problem CS courses on data structures and algorithms, and strong programming skills. [Speech and Language Processing](https://home.cs.colorado.edu/~martin/slp2.html) (2nd Edition, 2007, Prentice-Hall), by Daniel Jurafsky and James Martin

### FAQ
Should I take this course?

Yes, if:

- you're a CS student interested in languages, language technology, or information processing
- you're a CS student who needs an "applications" credit
- you're a language technology minor (this course is an elective option)
- you're a linguistics student who can write computer programs (this course is an elective option)
- you always suspected natural language was kind of like Lisp (or Java or ...)
- you want computers to take over the world
- you don't want computers to take over the world, but if they do, you want to negotiate your release
- you like AI, machine learning, and/or theoretical computer science, and want to apply them to a hard real-world problem

11-411/611 Natural Language Processing

Natural language processing (NLP) enables computers to understand, interpret, and generate human language. Common sub-topics include  text classification, sentiment analysis, named entity recognition, and machine translation. It applies techniques from [*Machine Learning*](/topic/machine-learning) and [*Deep Learning*](/topic/deep-learning)

Natural Language Processing

Computer Science

Natural Language Processing

Prerequisites

Common concepts