Lock (computer science)

Parallel computing

Parallel efficiency

Multi-core processor

SIMD (Single Instruction, Multiple Data)

Multithreading

Latency

Bandwidth (computing)

ISPC (Intel SPMD Program Compiler)

Work Stealing

Message Passing

Asynchronous Communication

Pipelining

Arithmetic Intensity

Contention

GPU Architecture

CUDA (Compute Unified Device Architecture)

Data Parallelism

Prefix Sum

Apache Spark

Resilient Distributed Dataset (RDD)

Cache coherence

MSI Protocol

MESI Protocol

False Sharing

Memory Consistency

Non-blocking algorithm

Transactional memory

Software Transactional Memory (STM)

Hardware Transactional Memory (HTM)

Energy-Efficient Computing

Heterogeneous Computing

Fixed-function

Field-Programmable Gate Array (FPGA)

System on a Chip (SoC)

Domain-Specific Language (DSL)

Halide (programming language)

Graph Processing

DRAM (Dynamic Random-Access Memory)

Reconfigurable Computing

Deep Neural Network

Matrix Multiplication

Layer Fusion

TensorCore

Tensor Processing Unit (TPU)

CS 149 PARALLEL COMPUTING Stanford University  Fall 2022 Focused on principles and trade-offs in designing modern parallel computing systems, this course also teaches parallel programming techniques. It is intended for students looking to understand both parallel hardware and software design. Prerequisite knowledge in computer systems is required. From smart phones, to multi-core CPUs and GPUs, to the world's largest supercomputers and web sites, parallel processing is ubiquitous in modern computing. The goal of this course is to provide a deep understanding of the fundamental principles and engineering trade-offs involved in designing modern parallel computing systems as well as to teach parallel programming techniques necessary to effectively utilize these machines. Because writing good parallel programs requires an understanding of key machine performance characteristics, this course will cover both parallel hardware and software design.  [CS110: Principles of Computer Systems](http://web.stanford.edu/class/cs110/) is strong prerequisite for CS149. If you have not taken CS110 and feel like you have the background to take CS149, please contact your friendly instructors and let's talk it over. There is no required textbook for CS149. However, you may find the following textbooks helpful. However, in general these days there's plenty of great parallel programming resources available for free on the web.

John L. Hennessy and David A. Patterson

Computer Architecture, Sixth Edition: A Quantitative Approach. Morgan Kaufmann, 2017

[[ On Amazon ]](https://www.amazon.com/Computer-Architecture-Quantitative-Approach-Kaufmann/dp/0128119055)


Jason Sanders

CUDA by Example: An Introduction to General-Purpose GPU Programming. 1st Edition. Addison-Wesley. 2010

[[ On Amazon ]](https://www.amazon.com/CUDA-Example-Introduction-General-Purpose-Programming/dp/0131387685)


David Kirk and Wen-mei Hwu

Programming Massively Parallel Processors, Second Edition: A Hands-on Approach. 2nd Edition. Morgan Kauffmann. 2012

[[ On Amazon ]](https://www.amazon.com/Programming-Massively-Parallel-Processors-Second/dp/0124159923)

CS 149 PARALLEL COMPUTING

Parallelism and Distributed Systems

Concurrent Threads

Scheduling (computing)

Linkers

Virtual memory

File system

File System Crash Recovery

Virtual Machine (VM)

Locks Implementation

Dynamic Linking

Demand Paging

Directories

Protection

Dispatching

Condition variables

Deadlock

Dynamic Storage Management

Disk Devices

Links

Flash Memory

Threads & Processes

CS 140: Operating Systems Stanford University Spring 2020 This course provides an in-depth understanding of the basic facilities provided by modern operating systems. It's structured into three major sections: concurrency, memory management, and file systems, followed by some smaller topics like virtual machines. The class includes one problem set and four programming projects based on the Pintos kernel, requiring a significant commitment of time. This class introduces the basic facilities provided by modern operating systems. The course divides into three major sections. The first part of the course discusses concurrency: how to manage multiple tasks that execute at the same time and share resources. Topics in this section include processes and threads, context switching, synchronization, scheduling, and deadlock. The second part of the course addresses the problem of memory management; it will cover topics such as linking, dynamic memory allocation, dynamic address translation, virtual memory, and demand paging. The third major part of the course concerns file systems, including topics such as storage devices, disk management and scheduling, directories, protection, and crash recovery. After these three major topics, the class will conclude with a few smaller topics such as virtual machines.

The class work consists of one problem set and a series of four programming projects based on the Pintos kernel. You will learn a lot from these projects, but be prepared to spend a significant amount of time working on them.   There is no required textbook for this class: the material of the course is defined by the lectures and does not exactly correspond to any existing book. However, I recommend the following book if you would like an additional source of material to supplement lectures:

- [Operating Systems: Principles and Practice (2nd Edition)](http://www.amazon.com/Operating-Systems-Principles-Thomas-Anderson/dp/0985673524/ref=pd_sim_14_2?ie=UTF8&dpID=51SFTQvZ7zL&dpSrc=sims&preST=_AC_UL160_SR123%2C160_&refRID=0ZR0KTC2DWEZCFFVAR26), by Thomas Anderson and Michael Dahlin.

Each page of lecture notes lists related readings in this book at the front of the notes page. Most students find that the material from lecture is sufficient for their course needs, so I recommend that you start the course without the book and only purchase the book if you are having difficulties understanding the lectures.

CS 140: Operating Systems

Operating Systems

Concurrency

Magnetic Disks

Virtual Machine Monitors

Lock Implementation

CS 111 Operating Systems Principles Stanford University Autumn 2022 An introductory course to operating systems, CS 111 builds upon programming experience to explore how operating systems function. The course provides an understanding of OS design challenges, such as filesystems, system calls, concurrency, virtual memory, demand paging, and others. Knowledge in C/C++ and Unix/Linux environment is prerequisite. CS111 is Stanford's introductory operating systems course. The CS106 courses provide you with a solid foundation in programming methodology and abstractions, and CS107 builds up and expands your breadth and depth of programming experience and techniques, working from the C programming language down to the microprocessor to de-mystify the machine. CS111 leverages this programming experience to introduce operating systems and how they work. With an understanding of both how to leverage operating system functionality in your own programs as well as how operating systems manage tasks behind the scenes, you will have a better understanding of how operating systems work, the kinds of design challenges present in operating systems and other large systems, and key computing ideas such as virtualization that are applicable in many different areas. Topics covered include: filesystems, system calls, concurrency, multiprocessing, multithreading, race conditions, synchronization primitives, virtual memory, demand paging and operating system design challenges.  The prerequisite for CS111 is CS107 (or equivalent). You should have practical C/C++ skills and be able to write programs with complex use of memory and pointers and dynamic memory allocation (`malloc`/`realloc`/`free`/`new`/`delete`). You should understand C++ classes, methods, and be able to work with appropriate data structures (arrays, maps, etc.) and standard algorithms (searching, sorting, hashing). (There are C++ features you're not expected to know, but you should be comfortable picking those features up and consulting references as needed). You should be familiar with programming in a Unix/Linux environment using tools such as `make`, `gcc/g++`, `valgrind`, `gdb`, etc. and have a working understanding of the basics of computer architecture (x86-64 as it’s taught in CS107, or exposure to some other architecture and the ability to pick up x86-64 as we reference it). Come talk with us if you need help determining the right placement for you. ### CS111A

CS111A, also called Pathfinders (or ACE), is a supplementary instruction program that meets for a weekly section and holds Pathfinders-specific review sessions. CS111A is application-only; CS111A is done in addition to all the normal requirements for CS111. You will receive an extra unit of course credit for the work you do in this program. Enrollment in CS111A is by application, and you can find more information at this link: [click here](https://engineering.stanford.edu/students-academics/equity-and-inclusion-initiatives/undergraduate-programs/additional-calculus). Once enrollment decisions are made, students who are accepted will then be given a permission number to enroll in CS111A on Axess. If you have questions about CS111A, please email the ACE CA (contact information listed on the course homepage).

### Textbook

There is no required textbook for CS111: the material of the course is defined by the lectures and does not exactly correspond to any existing book. However, we recommend the following book if you would like an additional source of material to supplement lectures:

*Operating Systems: Principles and Practice (2nd Edition), by Thomas Anderson and Michael Dahlin.*

Each lecture lists related readings in this book on the first slide, where applicable. Most students find that the material from lecture is sufficient for their course needs, so we recommend that you start the course without the book and only purchase the book if you decide you would like additional material and explanations beyond what is in lecture.

CS 111 Operating Systems Principles

Address Space

Atomic instructions

Monitors

Starvation

Address translation

Page table

TLB (Translation Lookaside Buffer)

General I/O

Queueing Theory

Reliability

Distributed Decision Making

Remote Procedure Call (RPC)

Key-Value Stores

Quantum Computing

Fork

Sockets

Synchronization (computer science)

Semaphores

Computer memory

Segments

Caching

Demand Paging Policies

Performance

Buffering

End-to-End Arguments

TCP/IP

NFS & AFS

DataCapsules

System Calls

Device Drivers

Mutual Exclusion

Readers/Writers

Paging

Storage Devices

Filesystem Design

Transactions

Networking

Distributed Storage

Chord

Inter-Process Communication (IPC)

Input/output

CS 162: Operating Systems and Systems Programming UC Berkeley Fall 2022 This course introduces operating systems design and related concepts. It covers topics like memory allocation, file systems, basic networking, transactions, and security. The course requires foundational knowledge in data structures, assembly language, C programming, and debugging. It aims to improve students' skills in debugging large programs and computational problem solving. The purpose of this course is to teach the design of operating systems and operating systems concepts that appear in other computer systems. Topics we will cover include concepts of operating systems, systems programming, networked and distributed systems, and storage systems, including multiple-program systems (processes, interprocess communication, and synchronization), memory allocation (segmentation, paging), resource allocation and scheduling, file systems, basic networking (sockets, layering, APIs, reliability), transactions, security, and privacy.  CS 61A, CS 61B, CS 61C, and CS 70, or equivalent courses. This means that you understand:

- Data structures: arrays, linked lists, binary trees, and hashing
- Assembly language programming
- The C programming language
- Debugging C using GDB
- CPU caches and memory hierarchy
- Virtual memory as covered in CS 61C
- CPU pipelines and basic digital logic design
- Basic knowledge of random variables and probability distributions as covered in CS 70


The TAs will spend a small amount of time reviewing some of the material you are less likely to remember. We will assume that you either know the material that is supposed to be covered in those courses, or that you are willing to learn the material as necessary. We will not spend time in lecture covering any of this material. **Perhaps more important than formal prequisites, however, is experience and maturity with debugging large programs, designing and implementing useful abstractions, and computational problem solving in general. This class will exercise your skills in these areas.**

If you feel that you would benefit from additional review of these prerequisites, we recommend using the following resources:

1. For a good review of probability as you’ll need it in this class, read through CS 70 Lecture Notes [15](https://inst.eecs.berkeley.edu/~cs70/sp14/notes/n15.pdf), [16](https://inst.eecs.berkeley.edu/~cs70/sp14/notes/n16.pdf), [17](https://inst.eecs.berkeley.edu/~cs70/sp14/notes/n17.pdf), [18](https://inst.eecs.berkeley.edu/~cs70/sp14/notes/n18.pdf), and [19](https://inst.eecs.berkeley.edu/~cs70/sp14/notes/n19.pdf).

1. Hennessy, John L. and Patterson, David A. *Computer Architecture: A Quantitative Approach*. 5th edition. **Although the main content of this book goes far more in-depth into computer architecture than you are expected to know for this class, the appendices provide an excellent overview of some of the prerequisites. Focus specifically on the sections below:**

- 2.1 and B.1 - B.4 (you can skip the sixth optimization in B.3) review caching, virtual memory, and memory hierarchies.
- C.1 - C.2 review CPU pipelines at the level of CS 61C. We are using the textbook “Operating Systems: Principles and Practice” by Anderson and Dahlin (A&D). The reading from this textbook is **strongly recommended**. There will be some additional readings that are not from this textbook that are freely available online. These will be linked on the course website in the "Readings" column for your viewing.

#### Strongly Recommended
[Operating Systems: Principles and Practice (2nd Edition)](http://ospp.cs.washington.edu/)
#### Recommended
[Operating Systems: Three Easy Pieces](http://pages.cs.wisc.edu/~remzi/OSTEP/)
#### Supplemental
[Linux Kernel Development (3rd Edition)](http://www.amazon.com/Linux-Kernel-Development-3rd-Edition/dp/0672329468)

CS 162: Operating Systems and Systems Programming

Lock (computer science)

4 courses cover this concept

CS 149 PARALLEL COMPUTING

CS 140: Operating Systems

CS 111 Operating Systems Principles

CS 162: Operating Systems and Systems Programming