Mark-Sweep Garbage Collection: Algorithms and Implementation

Generational collection leverages the weak generational hypothesis that most objects die young. By focusing collection on the young generation where most objects are dead, efficiency improves dramatically - copying collectors become 50x more efficient. Objects surviving young generation collection are promoted to the old generation. Remembered sets track cross-generational pointers using card marking, where writes dirty cards indicating potential references needing verification. This approach requires balancing promotion thresholds: too low causes excessive promotion, too high loses generational efficiency benefits.

Modern GC evolved from stop-the-world to concurrent execution through three approaches: parallel GC splits work across multiple threads reducing pause time proportionally; incremental GC breaks work into small pieces interleaved with JavaScript, requiring fix-up steps; fully concurrent GC allows JavaScript and GC to run simultaneously with complex synchronization. V8 implements parallel scavenging (2.7ms pauses) and concurrent marking/sweeping for major GC (9.8ms pauses). GC triggers automatically based on allocation rate, object size, and survival rate, dynamically computing optimal heap limits. Embedders can cooperatively trigger GC via idle tasks during animation frames. Best practices include allocating short-lived objects freely since they're cheap under generational GC.

The generational hypothesis states that newly created objects tend to die young while long-lived objects tend to survive. Java implements this by dividing memory into young, old, and permanent generations, dramatically improving performance by reducing scanning overhead. Modern collectors run concurrently with applications, using parallel collection and the tricolor marking algorithm to minimize stop-the-world pauses. This enables garbage collection without pausing the entire application, critical for server responsiveness.

Generational garbage collection divides the heap into young and old generations. The young generation is small and frequently collected via scavenge (copying) collection, completing in 1-2ms since most objects die young. The old generation uses mark-sweep and optionally compaction. This approach minimizes the amount of live data examined during each collection, as most allocations become garbage quickly. Non-compacting collections cause fragmentation, while compacting collections eliminate fragmentation at higher cost. V8 chooses between them based on fragmentation levels.

The Three-Color Marking algorithm optimizes Mark and Sweep by using three states: White (unmarked), Gray (marked but children not processed), and Black (fully processed). This enables concurrent garbage collection by maintaining the invariant that Black objects never point to White objects. When Gray objects are exhausted, White objects can be safely deleted. Write barriers track pointer modifications during collection, ensuring consistency. This approach minimizes pause times by allowing the collector to run in parallel with the application.

The tri-color algorithm uses white, gray, and black states to enable incremental GC. White objects are candidates for collection; black objects are fully processed; gray objects require further processing. The algorithm progresses like a wave through the object graph, allowing GC work to be interleaved with program execution. This reduces halting time and increases throughput. However, when program execution resumes between GC steps, new allocations may reference black objects, requiring write barriers and remembered sets to track these references.

Golang's garbage collector uses a tri-color mark-and-sweep algorithm that identifies live objects through roots (global variables, stack variables, goroutine stacks, and CPU registers) and marks them as gray, then processes them to mark children as black (alive) or white (garbage), with GC triggered when heap memory grows by 100% (default GOGC=100), running concurrently with the program causing minimal stop-the-world pauses.

This comprehensive section covers Java garbage collection from fundamentals to concurrent GC architecture. The speaker explains that most GC code is not innovative but rather standard textbook material, with only one or two innovations per collector. The fundamental trade-off between pause times and throughput is introduced: applications can sacrifice 90% throughput for zero pauses, or accept 10% slowdown for acceptable pause times. The section covers three traditional Java GCs: Serial GC (stop-the-world, copying young generation, compacting old generation), Parallel GC (concurrent with application but no object movement, causing fragmentation), and G1 GC (regional approach with concurrent marking but still requires stop-the-world pauses). Shenandoah GC is introduced as a collector that eliminates stop-the-world pauses entirely through concurrent marking and compaction. The fundamental challenge of identifying garbage in concurrent environments is explained through the three-color abstraction (white, gray, black), but concurrent application mutations create critical problems: objects that were reachable may become unreachable, and objects that were unreachable may become reachable. Two approaches handle this: Incremental Update (intercepts new references) and Snapshot the Beginning (marks all reachable objects at collection start). The section details how concurrent GCs ensure correctness through barriers: read barriers ensure reads go through forwarding pointers to see the latest version, while write barriers ensure writes go to the correct (to-space) version. The speaker explains that concurrent GCs pay a price in barriers even when no collection is happening, which is why stop-the-world GCs can outperform concurrent GCs in raw throughput.

The generational hypothesis states that newly created objects tend to die young while long-lived objects tend to survive. Java implements this by dividing memory into young, old, and permanent generations, dramatically improving performance by reducing scanning overhead. Modern collectors run concurrently with applications, using parallel collection and the tricolor marking algorithm to minimize stop-the-world pauses. This enables garbage collection without pausing the entire application, critical for server responsiveness.

Stack allocators maintain moving pointers through buffers, enabling extremely fast allocation but requiring conditional deallocation. Free list allocators maintain linked lists of freed blocks for specific sizes, providing fast allocation but suffering from fragmentation, poor thread affinity, and cache unfriendliness. Thread-local free lists address contention by giving each thread its own mini-free list. Augmenting allocators add metadata (prefixes/suffixes) to blocks, enabling debugging features like tracking allocation sites. These patterns demonstrate how allocators can be composed to add functionality while delegating core memory management to underlying strategies.

Thread Local Allocation Buffers (TLABs) are memory regions allocated per thread within the Eden space of the young generation. They enable fast object allocation without synchronization by allowing each thread to allocate directly from its own buffer. When a TLAB fills, the thread must synchronize with others to obtain a new buffer. HotSpot manages TLAB fragmentation by defaulting to retire and replace buffers when less than 1.5% remains free, balancing space utilization against allocation overhead.

To make object allocation fast, each thread has a private allocation buffer (TLAB). Objects are allocated by moving a pointer within this pre-allocated memory region (bump-pointer technique). This avoids global synchronization overhead for frequent allocations.

Buddy allocators divide memory into blocks of sizes that are powers of 2 (16, 32, 64, etc.), providing fast allocation and deallocation while minimizing fragmentation. This approach is particularly effective for game AI objects, which often have sizes that are powers of 2. The allocator recursively divides larger blocks in half until a suitable size is found, and merges adjacent free blocks of the same size when freed. Size-class allocators divide memory into blocks of sizes that are powers of 2, with configuration determined through profiling to match actual object sizes. Fixed-size block allocators divide memory into equal-sized chunks, providing predictable allocation behavior but wasting memory if requested sizes don't match chunk sizes. Object pools for particle systems and weather effects eliminate allocation overhead for frequently created and destroyed objects. Static memory allocation (pre-allocating all objects at startup) eliminates allocation overhead entirely but requires careful memory budgeting. Thread-local memory allocation provides each thread with its own memory pool, eliminating contention on system allocators and scaling well with the number of CPU cores, achieving 1.7-1.8x speedup on 4-core systems.

A segregated free list maintains multiple separate free lists, each dedicated to a specific size class of blocks. Small size classes handle individual sizes, while larger classes handle ranges of sizes (often powers of two). Allocation involves finding the appropriate size class, searching its list for a suitable block, splitting if necessary, and inserting the remainder into the correct class. This approach improves both performance (faster searches due to smaller lists) and memory utilization (approximates best-fit behavior).