Prefix Sum

Lock (computer science)

Parallel computing

Parallel efficiency

Multi-core processor

SIMD (Single Instruction, Multiple Data)

Multithreading

Latency

Bandwidth (computing)

ISPC (Intel SPMD Program Compiler)

Work Stealing

Message Passing

Asynchronous Communication

Pipelining

Arithmetic Intensity

Contention

GPU Architecture

CUDA (Compute Unified Device Architecture)

Data Parallelism

Apache Spark

Resilient Distributed Dataset (RDD)

Cache coherence

MSI Protocol

MESI Protocol

False Sharing

Memory Consistency

Non-blocking algorithm

Transactional memory

Software Transactional Memory (STM)

Hardware Transactional Memory (HTM)

Energy-Efficient Computing

Heterogeneous Computing

Fixed-function

Field-Programmable Gate Array (FPGA)

System on a Chip (SoC)

Domain-Specific Language (DSL)

Halide (programming language)

Graph Processing

DRAM (Dynamic Random-Access Memory)

Reconfigurable Computing

Deep Neural Network

Matrix Multiplication

Layer Fusion

TensorCore

Tensor Processing Unit (TPU)

CS 149 PARALLEL COMPUTING Stanford University  Fall 2022 Focused on principles and trade-offs in designing modern parallel computing systems, this course also teaches parallel programming techniques. It is intended for students looking to understand both parallel hardware and software design. Prerequisite knowledge in computer systems is required. From smart phones, to multi-core CPUs and GPUs, to the world's largest supercomputers and web sites, parallel processing is ubiquitous in modern computing. The goal of this course is to provide a deep understanding of the fundamental principles and engineering trade-offs involved in designing modern parallel computing systems as well as to teach parallel programming techniques necessary to effectively utilize these machines. Because writing good parallel programs requires an understanding of key machine performance characteristics, this course will cover both parallel hardware and software design.  [CS110: Principles of Computer Systems](http://web.stanford.edu/class/cs110/) is strong prerequisite for CS149. If you have not taken CS110 and feel like you have the background to take CS149, please contact your friendly instructors and let's talk it over. There is no required textbook for CS149. However, you may find the following textbooks helpful. However, in general these days there's plenty of great parallel programming resources available for free on the web.

John L. Hennessy and David A. Patterson

Computer Architecture, Sixth Edition: A Quantitative Approach. Morgan Kaufmann, 2017

[[ On Amazon ]](https://www.amazon.com/Computer-Architecture-Quantitative-Approach-Kaufmann/dp/0128119055)


Jason Sanders

CUDA by Example: An Introduction to General-Purpose GPU Programming. 1st Edition. Addison-Wesley. 2010

[[ On Amazon ]](https://www.amazon.com/CUDA-Example-Introduction-General-Purpose-Programming/dp/0131387685)


David Kirk and Wen-mei Hwu

Programming Massively Parallel Processors, Second Edition: A Hands-on Approach. 2nd Edition. Morgan Kauffmann. 2012

[[ On Amazon ]](https://www.amazon.com/Programming-Massively-Parallel-Processors-Second/dp/0124159923)

CS 149 PARALLEL COMPUTING

Parallelism and Distributed Systems

Prefix Sum

1 courses cover this concept

CS 149 PARALLEL COMPUTING