MIT 6.824 Lecture 1: Introduction to Distributed Systems (Spring 2020)

Added:

The "COST" of Distributed Systems

While courses like MIT 6.824 emphasize scaling out using distributed frameworks like MapReduce, a prominent counterpoint is that distributed systems are often highly inefficient compared to optimized single-node systems. Introduced by Frank McSherry et al. in the paper "Scalability! But at what COST?", this perspective argues that many distributed data processing systems introduce immense overhead (network latency, serialization, coordination) to the point where they are actually slower than a well-designed, single-threaded program running on a single commodity machine for common dataset sizes. This critiques the "distributed by default" mindset taught in computer science, urging engineers to first exhaust single-node optimizations—such as multi-core parallelism and efficient memory usage—before adopting the complexity, financial cost, and failure modes inherent to distributed architectures.

Fundamental understanding of operating systems concepts, including processes, threads, concurrency, and synchronization mechanisms like locks and semaphores.
Basic knowledge of computer networking principles, particularly the TCP/IP stack, socket programming, and client-server architectures.
Proficiency in a systems-level programming language (preferably Go, as it is the primary language for the 6.824 course labs) to understand concurrent code execution.
Familiarity with basic file systems, input/output (I/O) bottlenecks, and standard data-processing workflows.
In-depth analysis of Consensus Algorithms, specifically the Raft and Paxos protocols, to achieve fault tolerance across distributed nodes.
Study of Distributed Storage Architectures and real-world implementations like the Google File System (GFS) and Google Spanner.
Exploration of consistency models (such as linearizability, sequential consistency, and eventual consistency) and the tradeoffs described by the CAP Theorem.
Understanding modern cluster computing, batch processing, and stream processing frameworks (such as Apache Spark, Flink, and Kafka) that evolved from MapReduce.
Practical design and implementation of decentralized, replicated key-value databases that can survive server failures and network partitions.
741.5K views0likes1:19:35@6.824Original Release: 2020-02-06

A distributed system is a set of cooperating computers communicating over a network to accomplish coherent tasks, with key motivations including high-performance parallelism, fault tolerance, natural physical distribution requirements, and security isolation. The course explores core challenges such as concurrent programming complexities, partial failures, and the difficulty of achieving linear scalability. Two fundamental tools for building distributed systems are Remote Procedure Call (RPC) for masking unreliable network communication and threads for structuring concurrent operations. The MapReduce framework exemplifies distributed system design by abstracting away infrastructure complexity, allowing programmers to express computations as simple map and reduce functions while the framework handles data distribution, fault tolerance, and parallel execution across thousands of computers.