Advanced cgroups v2: Release Notification, Delegation, Thread Mode

Added:

Architectural Rigidity and Complexity in cgroups v2

While cgroups v2 improves upon v1 by enforcing a single unified hierarchy, this architectural shift introduces significant limitations, particularly regarding delegation and thread mode. Critics argue that the strict 'no-internal-process' rule in v2—which dictates that domain controllers cannot have processes in both parent and child groups—makes thread-level resource management highly restrictive. To address this, 'thread mode' was retrofitted as a complex compromise, but it lacks the elegant, native thread-level control of cgroups v1. Furthermore, safe unprivileged delegation remains difficult to configure securely without relying heavily on systemd or risking resource exhaustion. Consequently, some system architects favor alternative resource-control paradigms. For instance, eBPF-based scheduling (like sched_ext) offers highly dynamic, programmatic resource allocation without the rigid hierarchical constraints of cgroups. Additionally, lightweight hypervisors (such as microVMs) are increasingly preferred over OS-level cgroup isolation for multi-tenant environments, as they provide stronger security boundaries and bypass the intricate kernel delegation issues entirely.

Basic understanding of Linux control groups (cgroups v1 vs v2) and the unified hierarchy model.
Familiarity with the Linux process model, specifically the distinction between processes, threads, and thread groups.
Knowledge of how to navigate and manipulate virtual filesystems, particularly the '/sys/fs/cgroup' directory.
Understanding of Linux user privileges, file ownership, and Discretionary Access Control (DAC) mechanisms.
Designing and running rootless containers (e.g., using Podman or Docker in rootless mode) utilizing cgroups v2 delegation.
Implementing advanced systemd resource management using slices, scopes, and custom unit file configurations.
Monitoring and analyzing system resource pressure using Pressure Stall Information (PSI) combined with cgroups v2.
Developing custom userspace daemons or orchestrators in C, Go, or Rust to dynamically manage system resources via the cgroups v2 API.
5.6K views132likes1:01:50@NDCOriginal Release: 2021-11-03

Cgroups v2 introduces three advanced features: (1) Release notification uses cgroup.events files with a 'populated' key to detect when a cgroup becomes empty, enabling manager processes to know when workers have completed their tasks; (2) Delegation allows privileged users to pass ownership of cgroup subhierarchies to unprivileged users by changing directory and file ownership, enabling unprivileged container management while maintaining security through delegation containment rules that restrict process movement between containers; (3) Thread mode resolves the conflict between process-level granularity (default in v2) and thread-level granularity (possible in v1) by introducing threaded subtrees where threads of a multi-threaded process can be split across different cgroups within the subtree, with controllers classified as threaded (supporting thread-level granularity like CPU and PIDs controllers) or domain (only supporting process-level granularity like memory controller).