How Digital Audio Compression Works | Psychoacoustics Explained

Transform coding reduces spatial redundancy by converting data from spatial to frequency domains. Key transforms include Fourier Transform (FT), Discrete Fourier Transform (DFT), Discrete Cosine Transform (DCT), and Hadamard Transform, with DCT dominating video and image coding. Mathematically, vector spaces use basis vectors for representation, generalizing to Hilbert spaces with infinite bases. Functions like eˣ decompose into Taylor series, requiring infinite terms theoretically but achieving good approximation with truncation. The DFT extends these principles to finite discrete data, producing complex coefficients where the first represents the mean value. The DCT solves complex number overhead by using only cosine functions, producing purely real-valued coefficients while maintaining approximation capabilities.

The Modified Discrete Cosine Transform (MDCT) is a lapped transform based on the type-IV discrete cosine transform, designed for signal compression applications. Proposed by Princen, Johnson, and Bradley in 1987, the MDCT processes consecutive overlapping blocks where the last half of one block coincides with the first half of the next block, avoiding artifacts at block boundaries. The transform maps 2N inputs to N outputs through a linear function, achieving perfect invertibility through time-domain aliasing cancellation (TDAC). For even N, the MDCT is mathematically equivalent to a DCT-IV with shifted input; for odd N, it becomes equivalent to DCT-III/II. Efficient computation achieves O(N log N) complexity through recursive factorization. Window functions satisfy the Princen-Bradley condition (W · W + Wᴿ · Wᴿ = 2) to ensure invertibility. Different audio standards employ various window types: modulated lapped transforms for MP3/Vorbis, Kaiser-Bessel derived windows for AC-3/AAC. TDAC works by having IMDCT results from overlapping blocks cancel reversed terms, recovering original data. This mechanism enables the MDCT's widespread use in modern lossy audio formats including MP3, AAC, Vorbis, AC-3, and Opus.

Polyphase filter bank implementation is mathematically equivalent to applying the Modified Discrete Cosine Transform (MDCT). Convolution with the derived impulse response reveals that multiplying input samples by cosine terms performs Discrete Cosine Transform (DCT), while multiplying by sine terms performs Discrete Sine Transform (DST). Together, they constitute the MDCT operation. The prototype low-pass filter h[n] serves as the window function applied to time-domain samples. Efficient implementation uses an input buffer of 512 samples holding 16 blocks of 32 samples each; the current block is multiplied by the analysis window, windowed outputs are buffered, partitioned into eight 64-sample blocks, summed to form a 64-element vector, and finally processed by the MDCT block to produce 32 subband samples. This critically sampled system demonstrates how theoretical polyphase concepts translate to practical audio coding implementations.

The MDCT transform coefficients α_n,k are computed using projection onto the basis functions φ_n,k(t) = G(t)ψ_k(t). The computation involves dividing the analysis into three regions based on the window's shape: the rising edge (-0.25 to 0.25), the flat region (0.25 to 0.75), and the decaying region (0.75 to 1.25). The even symmetry of the Cosine 4 basis functions ψ_k(-t) = ψ_k(t) allows simplification of integrals over negative time intervals to positive intervals. This systematic approach transforms the original signal into the frequency domain while preserving the mathematical properties needed for perfect reconstruction.

MDCT (Modified Discrete Cosine Transform) is a modified Fourier transform used on overlapping blocks of information because MP3 encoding is lossy. Unlike perfect reconstruction transforms, the MDCT in the encoder does not achieve perfect reconstruction of the audio signal. The Fast Fourier Transform (FFT) derives the masking threshold from the input signal and enables the perceptual model to work in the frequency domain. In practice, 1,24 frequencies are passed into the perceptual model, and everything is coded in C or assembly language for embedded systems.

This comprehensive section covers the fundamental psychoacoustic principles underlying spatial audio mixing. Early reflections within the first 100 milliseconds primarily determine perceived room size, while the diffuse tail characterizes acoustic properties like wall materials and room shape. Perceived distance is controlled by the ratio between direct and reflected sound—louder reflections indicate greater distance. High-frequency attenuation varies by distance: closer instruments have less high-frequency reverb, while distant sources have more. For distant instruments, spatial processing should match dry sound since they naturally arrive mixed with reverb. Reverb panning strategies can expand virtual space by directing reflections to opposite pan positions. A two-reverb system separates volume modeling (early reflections) from acoustic property modeling (diffuse tail). Three send channels with progressive delay (80ms, 40ms, none), frequency attenuation, and level balancing create natural spatial depth across near, mid, and far fields.

This video explains how to create realistic spatial audio environments in music production by leveraging psychoacoustic principles: the human brain uses interaural level differences (high frequencies are blocked by the head, creating amplitude differences between ears) and interaural time differences (sound reaches one ear slightly before the other) to localize sound sources. To position instruments in a virtual space, producers should reduce high frequencies on the channel opposite to the desired position, use pre-delay settings in reverb to control perceived distance (longer pre-delay creates a closer illusion), and combine delay with reverb to simulate natural room acoustics where early reflections are louder and clearer while late reflections are softer and more diffuse.

This video teaches psychoacoustic techniques for placing audio objects in a sound field: to place an object in the front right, add delay, cut high frequencies, and reduce volume; to move an object to the background, simultaneously reduce overall volume, decrease high-frequency spectrum amplitude, and add scattered reflections; these techniques can be automated for dynamic movement and apply to music, film sound design, and audio books.

Dolby Atmos is an immersive audio technology that creates a 3D soundscape by adding ceiling-mounted speakers that bounce sound off the ceiling and back down, unlike traditional stereo (2 channels) or surround sound (5.1 or 7.1 channels) which only project sound horizontally. To experience Dolby Atmos on Apple Music, users need compatible hardware (iPhone/iPad with iOS 14.6+, Mac with macOS 11.4+, compatible headphones like Sony, Beats, or Sennheiser with H1/W1 chip, or external speakers) and compatible content (tracks marked with the Dolby Atmos logo). Configuration involves enabling the feature in Music app preferences on Mac or Settings > Music on iOS devices, and users can find Dolby Atmos music through the 'Discover Spatial Audio' playlist or by browsing the 'Audio Spatial with Dolby Atmos' section organized by genre.

Mixing combines multiple audio tracks into a cohesive whole. Sound perception begins with vibration, travels through air as waves, and is processed by the brain. Humans hear frequencies from 20 Hz to 20,000 Hz, with frequencies below 50 Hz perceived by the chest cavity rather than eardrums. Three key psychoacoustic principles guide mixing: (1) Low frequencies below 200 Hz should be centered because the body cannot localize them; (2) Sounds within 3 milliseconds are perceived as one sound, enabling delay-based widening; (3) Masking occurs when similar frequencies overlap, requiring mixers to cut competing frequencies (e.g., removing low frequencies from vocals to prevent bass conflict).

HLS (HTTP Live Streaming) is an adaptive bitrate streaming protocol that divides videos into small segments (TS files) with an index file (m3u8), enabling smooth playback across varying network conditions by allowing the video player to dynamically switch between different quality levels (360p, 480p, 720p, 1080p) based on user bandwidth and device capabilities; the system architecture typically involves uploading raw videos to cloud storage (like S3), using Docker containers with FFMpeg to transcode videos into multiple quality variants, storing generated HLS files in a public bucket, and implementing a master playlist file that directs the video player to the appropriate quality level.

DashJS is an open-source DASH player (BSD3 license) that serves as the reference client for the DASH industry forum. It uses dependency injection (Djon framework) for extensibility, separating streaming package (core logic) from DASH package (specific implementation). The architecture enables other protocols like HLS to be built similarly. Live stream edge calculation is more complex than HLS because DASH manifests are typically loaded once rather than refreshed on every request. DashJS uses a rule-based system for adaptive bitrate decisions, running rules before each segment request including download ratio (bandwidth estimation), buffer management, and other heuristics. The DASH handler generates segment URLs using getSegmentRequestForTime (for seeking) and getNextSegment (for sequential playback), taking quality level parameters to construct URLs based on manifest formats.

Adaptive bitrate streaming (ABS) dynamically adjusts video quality based on viewer internet speed to prevent buffering. Videos are encoded at multiple bitrates, allowing devices to automatically switch between versions as network conditions change. ABS uses protocols like HTTP Live Streaming (HLS) and DASH (Dynamic Adaptive Streaming over HTTP), which divide videos into segments delivered via HTTP. DASH is widely adopted by major services including Netflix, YouTube, and Hulu. Video streaming is crucial in system design interviews because it involves complex distributed systems requiring understanding of protocols, server-network-client interactions, and optimization techniques for scalable video delivery.

Adaptive Bitrate Streaming (ABS) is a video delivery technology that provides multiple encoded versions of the same content at different quality levels (bit rates, resolutions), allowing the streaming player to automatically select the optimal version based on the viewer's available bandwidth and device capabilities, ensuring smooth playback without buffering while maximizing video quality.

Video streaming services implement adaptive bitrate streaming protocols (like HLS and DASH) that use HTTP range requests to deliver video in small chunks, allowing clients to dynamically switch between different quality levels based on their network conditions, thereby reducing buffering and ensuring smooth playback across varying network speeds.