WebAssembly Synth: C++, Web Audio & DSP

Learning Goal: Build a high-performance, real-time digital audio synthesizer from scratch. You will write core Digital Signal Processing (DSP) algorithms in C++, compile them to WebAssembly (Wasm) using the Emscripten toolchain, and integrate them into a low-latency, multi-threaded Web Audio API graph using custom Audio Worklets and the Web MIDI API.

Prerequisites

• Basic familiarity with HTML, CSS, and modern JavaScript (ES6+).
• No prior C++ or DSP experience required; concepts are introduced from first principles.

Estimated Study Time

• 42 Hours (including video lectures, interactive coding exercises, and final implementation).

---

Module 1: Sound Physics & Digital Audio Foundations

Module Overview

Before writing synthesis code, you must understand what sound actually is and how physical air vibrations are captured, digitized, and represented in computer memory. This module covers the physics of acoustic waves, the analog-to-digital conversion process, sample rates, bit depths, and the critical mathematical rules of the Nyquist-Shannon Sampling Theorem.

---

Why this video is valuable

This visual-first video breaks down how physical vibrations map directly to acoustic waves. Understanding air compression, rarefaction, and wave propagation is crucial for synthesizing these physics-based phenomena inside a computer.

Knowledge Checkpoint

• Explain how sound propagates through physical media as longitudinal waves.
• Define the relationship between wave frequency, wavelength, and the perceived pitch of a sound.
• Explain how acoustic wave fronts represent alternating pressure changes.

---

Why this video is valuable

This lecture explains how a continuous analog wave is converted into a series of discrete digital numbers. It contrasts sample rate (time resolution) with bit depth (amplitude resolution) to give you a clear baseline of digital audio storage mechanics.

Knowledge Checkpoint

• Describe how sample rate determines the temporal resolution of an audio file.
• Define how bit depth limits the dynamic range and the noise floor of digital audio.
• Calculate the physical size differences of 16-bit vs. 24-bit PCM raw audio streams.

---

Why this video is valuable

An essential deep dive into the Nyquist-Shannon Sampling Theorem. To write oscillators from scratch without producing horrific digital noise (aliasing), you must understand why sampling rates must be at least twice the maximum target frequency.

Knowledge Checkpoint

• Define the term "Nyquist Frequency" for a given sample rate (e.g., 44.1 kHz).
• Identify what happens when a digital oscillator generates frequencies above half the sample rate.
• Explain why brick-wall reconstruction filters are mathematically necessary during digital-to-analog conversion.

---

Module 2: C++ Language Basics

Module Overview

C++ is the industry standard for real-time DSP due to its predictable memory usage, execution speed, and control over low-level hardware structures. This module introduces core C++ programming concepts, syntax, raw pointers, and memory layout (stack vs. heap), which are essential for managing audio buffers inside our WebAssembly environment.

---

Why this video is valuable

This crash course is an efficient bridge for developers transitioning from high-level, garbage-collected languages like JavaScript to C++. It covers setting up a workspace, raw data types, conditions, functions, and standard libraries.

Knowledge Checkpoint

• Set up a basic compiler environment and write, compile, and execute a "Hello World" terminal app in C++.
• Declare and manipulate primitive types, conditional operators, and loops inside a console application.
• Explain how C++ handles variables and structural parameters differently than dynamically typed languages.

---

Why this video is valuable

Pointers are the most critical concept to master for C++ audio programming. This video explains pointers in a practical way, showing how to reference physical memory locations and modify the underlying data directly without creating slow copies.

Knowledge Checkpoint

• Write C++ code utilizing raw pointers (*) and the address-of operator (&).
• Contrast how dynamic memory works on the Stack versus how it is manually managed on the Heap.
• Explain how passing parameters "by reference" improves performance compared to passing "by value."

---

Why this video is valuable

This tutorial explains modern C++ memory management techniques. For synthesizers, we need high-performance arrays (std::vector) and safe pointers (std::unique_ptr) to allocate audio streams dynamically without triggering dangerous memory leaks.

Knowledge Checkpoint

• Define the Resource Acquisition Is Initialization (RAII) pattern and how it prevents memory leaks.
• Implement std::unique_ptr and describe how dynamic allocation works without manually using delete.
• Use std::vector to scale and access dynamic contiguous arrays in memory safely.

---

Module 3: Introduction to DSP Theory & Code

Module Overview

With basic C++ syntax mastered, you will now focus on the core math behind digital wave generation. You will learn to construct software-based oscillators using phase accumulators, step through simple wave-generation math, and explore how physical analog systems map to digital systems.

---

Why this video is valuable

This video walks you through coding real synthesizer components in C. It demonstrates how to configure an array of oscillators, handle custom frequencies/wave shapes, and pass execution logic safely using function pointers.

Knowledge Checkpoint

• Build a software-based oscillator structure using a phase accumulator.
• Write an equation to step the phase of an oscillator relative to sample rate and target pitch.
• Define how to use function pointers to dynamically switch between raw generator shapes at runtime.

---

Why this video is valuable

A thorough analysis of fundamental oscillator shapes (sawtooth, triangle, and square wave) and their octave alignments. It helps you understand how the mathematical properties of wave shapes map to actual acoustic timbres.

Knowledge Checkpoint

• Distinguish the physical and spectral differences between a sine wave, sawtooth wave, and square wave.
• Explain how duty cycle or pulse-width modification changes the harmonic spectrum of a square wave.
• Calculate the correct octave transposition relationships between multiple oscillators running concurrently.

---

Curriculum Gap Alert: Real-time C++ audio programming courses are rare on YouTube. The provided videos focus on fundamental architectures and analog waveforms. To write fully optimized, band-limited oscillators (like BLEP or PolyBLEP) in C++ to prevent aliasing, you should independently research:

• What to search: "How to write a synthesizer oscillator in C++" or "PolyBLEP bandlimited oscillator implementation C++"
• Key Concept to learn: Phase accumulators mapping values strictly from $[0.0, 1.0)$ to output float buffers of $[-1.0, 1.0]$.

---

Module 4: JavaScript & Web Audio API Fundamentals

Module Overview

Before compiling our C++ code to the web, we must learn the browser's native audio architecture. The Web Audio API utilizes a modular routing graph of AudioNodes. Here, you will learn the basics of this node graph, how the audio clock functions, and how to create simple instruments using built-in JavaScript nodes.

---

Why this video is valuable

This introduction explains the baseline context of Web Audio API architecture. It introduces the audio context, signal sources, processing nodes, and the final output node (the destination).

Knowledge Checkpoint

• Initialize an AudioContext inside a modern JavaScript web app.
• Programmatically instantiate and connect an OscillatorNode to an AudioDestinationNode to produce sound.
• Contrast the execution lifecycles of native Web Audio nodes with typical main-thread JavaScript execution.

---

Why this video is valuable

This concise talk illustrates how modular audio graphs behave. It compares the Web Audio API graph to physical signal paths (like connecting a guitar through distortion pedals), helping you visualize complex routing setups.

Knowledge Checkpoint

• Diagram a Web Audio node graph that routes a source through an effects processor before the output.
• Describe the modular nature of graph connections (source.connect(effect).connect(destination)).
• Explain how parameters of one node (e.g., LFO) can modulate inputs of another node in real time.

---

Why this video is valuable

This deep dive shows how to build an active browser-based instrument. It covers trigger envelopes, basic waveform nodes, dynamic pitch adjustments, and routing signals within an interactive user interface.

Knowledge Checkpoint

• Implement a trigger function in JavaScript that handles user interactions and plays matching synth tones.
• Write a basic amplitude envelope helper in JavaScript using the linearRampToValueAtTime parameter scheduling method.
• Explain why browser security restrictions require user interactions (like a button click) before initiating audio playback.

---

Module 5: WebAssembly & Emscripten Compilation

Module Overview

To run your high-performance C++ synth code in a web page, you must compile it into WebAssembly (Wasm). Wasm is a low-level, binary instruction format that runs at near-native speeds inside the browser. In this module, you will set up the Emscripten toolchain, compile C++ files, and establish a high-performance shared-memory pipeline between JavaScript and WebAssembly memory buffers.

---

Why this video is valuable

This step-by-step tutorial walks through installing the Emscripten SDK (emsdk) and compiling C++ code into WebAssembly files (.wasm, .js). You will learn how to initialize and instantiate compiled modules inside a vanilla web page.

Knowledge Checkpoint

• Install the Emscripten SDK and configure environment variables globally.
• Compile a basic C++ source file using the emcc CLI command with clean optimizations (-O3).
• Instantiate a compiled .wasm binary module in a client application using modern JavaScript.

---

Why this video is valuable

Synthesizers cannot afford slow serialization or copying steps when transferring audio samples from WebAssembly to JavaScript. This video demonstrates how to access WebAssembly memory buffers (Wasm.Memory) directly from JavaScript using typed arrays (Float32Array).

Knowledge Checkpoint

• Identify where WebAssembly memory is allocated, and access its underlying ArrayBuffer directly from JavaScript.
• Read and write float values to shared linear memory blocks using dynamic offsets.
• Explain how C++ pointers map to numeric offsets inside the shared JavaScript memory buffer.

---

Why this video is valuable

This case study highlights how WebAssembly operates in sandboxed browser environments. It discusses performance constraints, data transfer limits, and interoperability design choices when working with native web engines.

Knowledge Checkpoint

• Describe how JavaScript and WebAssembly exchange numerical references across the boundaries of a compiled module.
• Explain why WebAssembly cannot access the browser DOM or Web APIs directly, relying on JS bindings instead.
• Identify standard performance overhead sources during complex data transfers, and design optimized low-copy interfaces.

---

Curriculum Gap Alert: Although these videos cover basic compiled buffers and linear memory mapping, they don't cover real-time streaming audio setups.

• Self-Guided Study: Read about how to expose C++ class instances to JavaScript via Emscripten bindings (embind) or standard C interfaces (extern "C").
• Target Search: "Exposing C++ classes to JavaScript with Embind" and "How to write a fast C C++ WebAssembly memory wrapper".

---

Module 6: Audio Worklet & WebAssembly Audio Thread Integration

Module Overview

The main browser thread is responsible for UI layout, user input, and heavy page logic. If audio code runs on this main thread, any layout shift or script execution can stall the audio engine, causing annoying pops and clicks. To prevent this, you will use the Web Audio Worklet API to run your compiled WebAssembly C++ code inside a high-priority, dedicated audio thread.

---

Why this video is valuable

This presentation by a Mozilla Web Audio developer explains browser audio performance. It details why the main thread blocks and why the AudioWorkletProcessor is essential for real-time, low-latency performance.

Knowledge Checkpoint

• Explain why standard JavaScript execution can cause audible dropouts, and why separate thread rendering solves this issue.
• Define the roles of AudioWorkletNode (main thread) and AudioWorkletProcessor (audio thread).
• Explain the performance advantages of utilizing lock-free data rings or shared memory blocks in audio-processing workflows.

---

Curriculum Gap Alert: There is a lack of high-quality, step-by-step video tutorials on executing compiled WebAssembly within an Audio Worklet thread on YouTube.

• Supplemental Research Target: Search for "AudioWorklet WebAssembly Emscripten tutorial" or "How to compile WebAssembly for AudioWorklet thread".
• Implementation Blueprint: Your main page must fetch the .wasm file, compile it into a WebAssembly.Module object, and send it to your AudioWorkletProcessor subclass via a postMessage() call. Within the Worklet's render loop, you then instantiate that module and execute its high-priority, real-time C++ audio rendering loops directly on the audio thread.

---

Module 7: Synthesizer Architecture: ADSR, Polyphony & MIDI

Module Overview

With our WebAssembly C++ engine running smoothly in the Audio Worklet thread, we can now assemble the complete synthesizer. In this module, you will design an ADSR envelope generator, build a voice allocation system to handle polyphony (playing multiple notes at once), and map external physical MIDI keyboard inputs to control your custom web instrument.

---

Why this video is valuable

This classic tutorial walks through the math and code structures needed to build ADSR (Attack, Decay, Sustain, Release) envelope generators. This model transforms raw, on-and-off oscillator waves into natural-sounding instruments.

Knowledge Checkpoint

• Explain the four phases of the ADSR model and how they modify audio amplitude over time.
• Write dynamic code equations to scale sample amplitude linearly or exponentially over envelope phase lengths.
• Map note-on and note-off events to trigger and release phases inside your envelope generator.

---

Why this video is valuable

An analog hardware perspective on envelope design. This video illustrates how analog capacitors and circuits generate charging curve shapes. This gives you a clear mental model when recreating these classic shapes in software.

Knowledge Checkpoint

• Contrast how logarithmic charging curves behave differently than linear ramps.
• Explain how physical discharge paths shape the release decay phases of sound events.
• Implement exponential transitions in your envelope code to mimic physical analog gear.

---

Why this video is valuable

This tutorial explains the concept of polyphonic voice allocation. You will learn how to track active and idle voice units, assign new keyboard notes to quiet voices, and reuse voices when they finish playing.

Knowledge Checkpoint

• Define the role of a polyphonic voice allocation system in software synthesizers.
• Implement a simple voice-stealing algorithm (e.g., allocating a new note to the oldest active voice).
• Track voice state flags ("active," "releasing," "idle") to optimize processing performance.

---

Why this video is valuable

A complete guide to using the browser's Web MIDI API. You will learn how to access USB MIDI keyboards and map incoming performance data (like MIDI note numbers and velocity values) to play your custom software synthesizer.

Knowledge Checkpoint

• Request MIDI access in the browser using the native navigator.requestMIDIAccess() API.
• Parse raw MIDI message bytes to identify note-on/note-off actions and key velocity parameters.
• Route parsed MIDI note inputs directly into your synthesizers' voice allocator.

---

Curriculum Gap Alert: Combining C++ polyphony voice management with browser Web MIDI arrays is an advanced architecture with limited video coverage on YouTube.

• Self-Guided Study: Combine the JavaScript MIDI bytes parser from this module with a shared data buffer, and pass those parameters into your high-performance WebAssembly Audio Worklet rendering loops.
• Target Search: "C++ polyphonic voice allocator synth" and "Wasm AudioWorklet MIDI message passing".

---

Course Map

---

Key People Index

• Harry Nyquist & Claude Shannon
  • Context: Pioneered the core sampling theorem that underlies all modern digital audio. Their research dictates how frequently we must measure continuous signals to record and play them back perfectly.
• Paul Adenot
  • Context: A Mozilla engineer and co-author of the W3C Web Audio API specification. He is a leading advocate for Audio Worklets and WebAssembly-based audio compilation pipelines in modern browsers.
• Steve Porcaro
  • Context: Iconic keyboardist and songwriter for Toto, known for custom synthesizer configurations. His classic techniques (e.g., stacking square and sawtooth waveforms) highlight the value of physical signal routing in synthesizer design.

---

Final Self-Assessment

Test your practical and theoretical understanding of your compiled synthesizer by completing this final checkpoint:

• Explain how sound compression/rarefaction maps to an array of $[-1.0, 1.0]$ float values inside computer memory.
• Mathematically calculate the frequency step value of an oscillator phase accumulator at a $44.1\text{ kHz}$ sample rate.
• Write a C++ helper function that generates clean sine wave frames without using dynamic memory allocations on the stack.
• Set up and compile a multi-file C++ project into clean Wasm files using Emscripten compiler optimizations.
• Route raw audio buffers safely from WebAssembly's linear memory space into JavaScript using typed Float32Array buffers.
• Build an active Web Audio AudioWorkletNode that runs compiled Wasm modules on the high-priority audio thread.
• Write an ADSR state machine in C++ that transitions cleanly between Attack, Decay, Sustain, and Release phases.
• Build a polyphonic voice allocation system that handles simultaneous key events and implements voice stealing.
• Access a hardware MIDI controller using navigator.requestMIDIAccess() and route key messages to play your browser synth.