Distributed Databases

Database System Architecture

Storage

Storage Formats

Query Execution

Query Optimization

Transactions

Failure Recovery

Concurrency

Cloud Database Systems

Streaming Systems

Security and Data Privacy

Database index

CS 245 Principles of Data-Intensive Systems Stanford University Winter 2022 This course delves into the architecture of modern data storage and processing systems, including databases, cluster computing, and machine learning systems. It covers database architecture, query optimization, transaction management, and parallel processing. CS 145 and CS 161, or equivalent, are expected as prerequisites. This course covers the architecture of modern data storage and processing systems, including relational databases, cluster computing systems, streaming and machine learning systems. Topics include database system architecture, storage, query optimization, transaction management, fault recovery, and parallel processing, with a focus on the key design ideas shared across many types of data-intensive systems.  Students should ideally have taken CS 145 and CS 161. If you haven't taken CS 145, the main thing we assume from it is knowledge of basic SQL syntax; you can also read a [SQL tutorial](https://www.w3schools.com/sql/) to an external site. to learn about this. ### Optional Textbook

[Database Systems: The Complete Book (2nd Edition)](https://www.pearson.com/us/higher-education/program/Garcia-Molina-Database-Systems-The-Complete-Book-2nd-Edition/PGM2429.html), by Garcia-Molina, Ullman and Widom, covers a lot of the technical material in the course and may be helpful as a study guide. We focus on chapters 13-20. We will also cover the material in lectures, but this book is a good source of additional information.

CS 245 Principles of Data-Intensive Systems

Database Systems

The Internet

Classical synchronization

Time Synchronization

Distributed Mutual Exclusion

Remote Procedure Call (RPC)

Distributed File Systems

Concurrency Control

Logging and Crash Recovery

Fault Tolerance

RAID (Redundant Array of Inexpensive Disks)

Distributed Replication

Spanner

Google File System (GFS)

Apache Hadoop Distributed File System (HDFS)

Data-Intensive Computing

MapReduce

Hadoop

In-memory cluster compute

Apache Spark

Distributed ML

Internet Content Delivery

DNS (Domain Name System)

Content Delivery Network (CDN)

Publish-Subscribe

Kafka

Virtualization

3-tier Web Service Architecture

Security Protocols

Byzantine Fault Tolerance

P2P Network

Distributed Ledgers

Blockchain

 15-440 Distributed Systems Carnegie Mellon University Fall 2020 A course offering both theoretical understanding and practical experience in distributed systems. Key themes include concurrency, scheduling, network communication, and security. Real-world protocols and paradigms like distributed filesystems, RPC, MapReduce are studied. Course utilizes C and Go programming languages. Most software is now distributed in some sense. This course is meant to serve as an introduction to distributed systems, emphasizing techniques for creating functional, usable, and high-performance distributed systems.

This course aims to: (1) provide students with an understanding of the principles and techniques behind the design of distributed systems, such as locking, concurrency, scheduling, and communication across the network. (2) provide students with practical experience designing, implementing, and debugging real distributed systems.

Major themes covered in this course include scarcity, scheduling, concurrency and concurrent programming, naming, abstraction and modularity, imperfect communication and other types of failure, protection from accidental and malicious harm, optimism, and the use of instrumentation and monitoring and debugging tools in problem solving. As the creation and management of software systems is a fundamental goal of any undergraduate systems course, students will design, implement, and debug large programming projects. After this course, students will have learned to…

- Implement and structure distributed systems programs.
- Write programs that can interoperate using well-defined protocols.
- Debug highly concurrent code that spans multiple programs running on multiple cores and machines.
- Reason about distributed algorithms for locking, synchronization and concurrency, scheduling, and replication.
- Use standard network communication primitives such as UDP and TCP.
- Understand the general properties of networked communication necessary for distributed systems programming in clusters and on the Internet.
- Employ and create common paradigms for easing the task of distributed systems programming, such as distributed filesystems, RPC, and MapReduce. Be able to clearly elucidate their benefits, drawbacks, and limitations.
- Identify the security challenges faced by distributed systems programs.
- Be able to select appropriate security solutions to meet the needs of commonly encountered distributed programming scenarios. Because this course has a big project component, you must be proficient in C and programming on UNIX systems. We will use the [Go programming language](https://golang.org/) throughout the term. It is required that you have taken 15-213 and gotten a “C-“ or higher since many of the programming skills you will need are taught in that course. **However, if you received a C in 15-213, you must meet with your academic advisor to discuss your background before taking 15-440/640, perhaps taking an additional course to sharpen your systems skills. Your advisor must email us approval and an explanation of why you have sufficient background to take 15-440/640.** 

 15-440 Distributed Systems

Parallelism and Distributed Systems