System Design Fundamentals [5/15]: B-Trees vs LSM-Trees Explained

Added:

The Paradigm Shift of NVMe and Byte-Addressable Persistent Memory

Traditional database storage design is built on the assumption that random disk access is orders of magnitude slower than sequential access, justifying the trade-offs of B-Trees and LSM-Trees. However, modern storage technologies like ultra-fast NVMe SSDs and byte-addressable Non-Volatile Memory (NVM) fundamentally challenge this classic 'physics of persistence.' With highly parallel flash channels and near-zero seek times, the performance gap between random and sequential I/O has narrowed significantly. Consequently, traditional write-ahead logging (WAL) and CPU-heavy compaction cycles (in LSM-Trees) or page locking (in B-Trees) become major bottlenecks, wasting CPU cycles and hardware endurance. This technological shift has spurred the creation of hardware-conscious architectures—such as SPDK-enabled engines, persistent memory indexes, and learned indexes—that bypass traditional block-device assumptions entirely, suggesting that the classic B-Tree vs. LSM-Tree dichotomy is becoming obsolete in the face of modern hardware.

Basic understanding of computer memory hierarchy, specifically the latency difference between RAM, SSDs, and HDDs.
Core concepts of tree data structures, such as Binary Search Trees (BSTs) and self-balancing properties.
Fundamental database concepts, particularly the ACID properties (with a focus on Durability) and basic CRUD operations.
The distinction between Sequential I/O and Random I/O operations on physical storage drives.
Analyzing real-world database engine implementations (e.g., InnoDB/PostgreSQL using B-Trees vs. Cassandra/RocksDB/LevelDB using LSM-Trees).
In-depth study of Write Amplification, Read Amplification, and Space Amplification (RUM Conjecture) tradeoffs in storage engines.
Advanced optimization techniques for storage engines, including Bloom Filters, SSTable compaction strategies (Size-Tiered vs. Leveled), and MemTable configurations.
The design of distributed databases and how storage engine selection influences replication, consensus algorithms (like Raft), and sharding.
8.3K views417likes24:43@SeniorEngineer10xOriginal Release: 2026-01-29

This video explains the fundamental trade-off between read and write performance in database systems, revealing that B-Trees optimize for reads by using shallow, fat trees that minimize disk trips (4-5 layers for billions of rows), while LSM-Trees optimize for writes by only performing sequential appends to logs and immutable SSTables, with the ROM Conjecture stating that you can only optimize for two of three: Read, Update, or Memory. The OS hides disk slowness through buffered I/O, but databases must use fsync() for durability, and SSDs have physical constraints like write amplification and limited erase cycles that make sequential writes faster than random writes.