Dimensionality Reduction

Property-preserving lossy compression

Nearest Neighbor

k-d tree

Regularization (mathematics)

L1 regularization

Matrix compression

Laplacian of a graph

Reservoir sampling

Markov chain Monte Carlo (MCMC)

Majority elements

Distance-preserving compression

PAC Guarantees

Linear-Algebraic Techniques

De-noising

Eigenvectors/eigenvalues

Markov's inequality

The Fourier Perspective

Linear programming

Approximate heavy hitters

Locality-Sensitive Hashing (LSH)

MinHash

Principal Component Analysis (PCA)

Matrix completion

Spectral embeddings

Chebyshev‘s inequality

Convolution

Convex optimization

Empirical risk minimization

Bloom filters

Jaccard Similarity

Polynomial embedding

Maximizing variance

Uniqueness of low-rank tensor factorizations

Graph coloring

Importance Sampling

Sparse Vector/Matrix Recovery

Privacy Preserving Computation

Count-min sketch

Euclidean Distance

Johnson-Lindenstrauss Transform

Random projection

Power iteration algorithm

Jenrich's algorithm

Spectral clustering

Markov Chains

Compressed Sensing

Differential privacy

Similarity Search

Lp distance

Generalization

L2 regularization

Singular Value Decomposition (SVD)

Spectral Graph Theory

Interpretations of the second eigenvalue

Stationary distributions

Mathematical Programming

CS 168: The Modern Algorithmic Toolbox Stanford University Spring 2022 CS 168 provides a comprehensive introduction to modern algorithm concepts, covering hashing, dimension reduction, programming, gradient descent, and regression. It emphasizes both theoretical understanding and practical application, with each topic complemented by a mini-project. It's suitable for those who have taken CS107 and CS161.  This course will provide a rigorous and hands-on introduction to the central ideas and algorithms that constitute the core of the modern algorithms toolkit. Emphasis will be on understanding the high-level theoretical intuitions and principles underlying the algorithms we discuss, as well as developing a concrete understanding of when and how to implement and apply the algorithms. The course will be structured as a sequence of one-week investigations; each week will introduce one algorithmic idea, and discuss the motivation, theoretical underpinning, and practical applications of that algorithmic idea. Each topic will be accompanied by a mini-project in which students will be guided through a practical application of the ideas of the week. Topics include modern techniques in hashing, dimension reduction, linear and convex programming, gradient descent and regression, sampling and estimation, compressive sensing, and linear-algebraic techniques (principal components analysis, singular value decomposition, spectral techniques).  CS107 and CS161, or permission from the instructor. 

CS 168: The Modern Algorithmic Toolbox

Theoretical Computer Science

Apache Spark

Frequent Itemsets Mining

Locality-Sensitive Hashing

Clustering

Recommender Systems

PageRank

Community Detection in Graphs

Graph Representation Learning

Graph neural network (GNN)

Learning Embeddings

Decision Trees

Mining Data Streams

Matrix Sketching

Computational Advertising

Learning through Experimentation

Optimizing Submodular Functions

CS246: Mining Massive Data Sets Stanford University Spring 2023 This course focuses on data mining and machine learning algorithms for large scale data analysis. The emphasis is on parallel algorithms with tools like MapReduce and Spark. Topics include frequent itemsets, locality sensitive hashing, clustering, link analysis, and large-scale supervised machine learning. Familiarity with Java, Python, basic probability theory, linear algebra, and algorithmic analysis is required. ### What is this course about? [[Info Handout](https://web.stanford.edu/class/cs246/handouts/CS246_Info_Handout.pdf)]

The course will discuss data mining and machine learning algorithms for analyzing very large amounts of data. The emphasis will be on MapReduce and [Spark](http://spark.apache.org/) as tools for creating parallel algorithms that can process very large amounts of data.

**Topics include**: Frequent itemsets and Association rules, Near Neighbor Search in High Dimensional Data, Locality Sensitive Hashing (LSH), Dimensionality reduction, Recommendation Systems, Clustering, Link Analysis, Large-scale Supervised Machine Learning, Data streams, Mining the Web for Structured Data, Web Advertising.  Students are expected to have the following background:

- Knowledge of basic computer science principles and skills, at a level sufficient to write a reasonably non-trivial computer program (e.g., CS107 or CS145 or equivalent are recommended).
- Good knowledge of Java and Python will be extremely helpful since most assignments will require the use of Spark.
- Familiarity with basic probability theory (CS109 or Stat116 or equivalent is sufficient but not necessary).
- Familiarity with writing rigorous proofs (at a minimum, at the level of CS 103).
- Familiarity with basic linear algebra (e.g., any of Math 51, Math 103, Math 113, CS 205, or EE 263 would be much more than necessary).
- Familiarity with algorithmic analysis (e.g., CS 161 would be much more than necessary). ### Reference Text

The following text is useful, but not required. It can be downloaded for free, or purchased from Cambridge University Press.

[Leskovec-Rajaraman-Ullman: Mining of Massive Dataset](http://www.mmds.org/)

CS246: Mining Massive Data Sets

Machine Learning

Decision Tree Learning

Margins

Naive Bayes

Maximizing Conditional Likelihood

Kernelizing Algorithms

Primal and Dual Forms

Finite Hypothesis Classes

k-fold cross-validation

Neural network

Convolutional neural network (CNN)

Objective-Based Clustering

Semi-Supervised Learning

Learning Linear Separators

Maximum Likelihood Estimation (MLE)

Bag of Words

Computer Vision

Generalization and Overfitting

Model Selection and Regularization

Support Vector Machine (SVM)

Markov Decision Process (MDP)

Minimizing Squared Error

Deep Networks

Adaboost

Learning Representations

Active Learning

Co-training

Q-learning

Overfitting

Conditional Independence

Gradient descent

Kernelizing Perceptron

Kernelizing SVM

VC Dimension Based Bounds

Linear regression

Backpropagation

Boosting

Hierarchical Clustering

Interactive Learning

Transductive SVM

Value iteration

Bayes' theorem

Perceptron

Maximum A Posteriori (MAP)

Logistic Regression

Kernel (operating system)

Geometric Margins

Sample Complexity

Structural Risk Minimization

Maximizing Data Likelihood

Unsupervised learning

Sampling Bias

Reinforcement learning (RL)

10-401 Introduction to Machine Learning Carnegie Mellon University Spring 2018 A comprehensive exploration of machine learning theories and practical algorithms. Covers a broad spectrum of topics like decision tree learning, neural networks, statistical learning, and reinforcement learning. Encourages hands-on learning via programming assignments.  Machine Learning is concerned with computer programs that automatically improve their performance through experience (e.g., programs that learn to recognize human faces, recommend music and movies, and drive autonomous robots). This course covers the theory and practical algorithms for machine learning from a variety of perspectives. We cover topics such as decision tree learning, Support Vector Machines, neural networks, boosting, statistical learning methods, unsupervised learning, active leaerning, and reinforcement learning. Short programming assignments include hands-on experiments with various learning algorithms.   - Machine Learning, Tom Mitchell. (optional)
- Pattern Recognition and Machine Learning, Christopher Bishop. (optional)
- Machine Learning: A Probabilistic Perspective, Kevin P. Murphy, [available online](https://ebookcentral.proquest.com/lib/cm/detail.action?docID=3339490), (optional)

10-401 Introduction to Machine Learning