Differential Privacy with Opacus: Training PyTorch Models Privately

PyTorch datasets are classes implementing __getitem__ and __len__ to define data access. DataLoaders wrap datasets, handling batching, shuffling, and parallel loading. The complete training pipeline: initialize model, optimizer (AdamW recommended), and scheduler; iterate over DataLoader batches; zero gradients; forward pass; calculate loss; backward pass; optimizer step; scheduler step. This pattern trains any PyTorch model from simple classifiers to complex transformers. AdamW with cosine scheduler and warmup is recommended for most applications.

PyTorch supports custom loss functions (MSE, cross-entropy, sigmoid cross-entropy, triplet margin loss) and activation functions. Datasets implement __len__ and __getitem__ for data loading, while DataLoaders provide batching and shuffling. Advanced techniques include gradient accumulation (increasing effective batch size), gradient clipping (preventing exploding gradients), and learning rate scheduling. Optimizers beyond SGD include Adam and AdamW.

PyTorch provides two dataset types: Map-style (stores all data in memory) and Iterable-style (generates samples on-the-fly for large datasets). Custom datasets require implementing __len__() and __getitem__() methods. Data transformations (torchvision.transforms) apply augmentations like rotation, flipping, and normalization. DataLoaders handle batching, shuffling, and parallel data loading with parameters like batch_size, shuffle, num_workers, and prefetch_factor. Neural networks are defined by inheriting from nn.Module and implementing __init__() (layer initialization) and forward() (data flow). A typical training loop includes: model.train(), optimizer.zero_grad(), forward pass, loss computation, backward pass, and optimizer.step(). The forward pass computes predictions, loss measures error, backward pass computes gradients, and optimizer.step() updates parameters. This structure is repeated for each epoch, with metrics tracked to monitor progress.

This tutorial explains how to use PyTorch's built-in Dataset and DataLoader classes to efficiently handle large datasets in batch training. The Dataset class requires implementing __init__, __getitem__, and __len__ methods to load and access data samples, while the DataLoader automates batch creation, shuffling, and iteration. Key concepts include: epoch (complete forward/backward pass of all samples), batch size (number of samples per pass), and iterations (number of passes per epoch). The tutorial demonstrates creating a custom wine dataset from CSV, using DataLoader with batch_size=4, shuffle=True, and workers=2, then implementing a training loop that processes batches automatically. PyTorch also provides built-in datasets like MNIST, FashionMNIST, CIFAR, and COCO for common machine learning tasks.

The training loop uses nested for loops: an outer loop for epochs and an inner loop for batches. For each batch, inputs and targets are extracted from the batch. optimizer.zero_grad() clears previous gradients. outputs = model(inputs) generates predictions. loss = criterion(outputs, targets) computes the error. loss.backward() calculates gradients. optimizer.step() updates model weights. running_loss += loss.item() accumulates losses for monitoring. The loss.item() detaches the tensor from the computation graph to convert it to a Python float.

This paper presents a systematic framework using game-based formalism to analyze five fundamental privacy risks in machine learning: property inference (inferring aggregate properties of training data distribution), membership inference (determining if a record is in the training set), attribute inference (inferring unknown attribute values), and data reconstruction (reconstructing individual records). The research establishes relationships between these risks, proving that differential privacy provides the strongest membership inference protection but does not protect against property inference attacks. The paper demonstrates that no single canonical game exists for measuring privacy in machine learning pipelines, instead providing a toolbox for formalizing threat models in specific scenarios.

This video explains how informed adversaries can reconstruct training data from machine learning model parameters, demonstrating that reconstruction attacks represent the maximum information leakage possible compared to membership inference attacks. The attack works by sampling from a prior distribution to generate multiple models differing by single training points, then learning a mapping from model parameters to training points. Key findings include: reconstruction works even with mini-batch SGD randomness, succeeds on well-generalized models (not just overfit ones), requires larger models with sufficient parameters, and is stable across learning hyperparameters. Differential privacy analysis shows that protecting against reconstruction requires epsilon values proportional to data dimensionality, potentially allowing larger privacy budgets than traditional membership inference protection.

Machine learning systems face complex privacy risks due to their ecosystem involving multiple datasets, continuous updates, cloud deployment, and federated learning. Privacy leakage occurs when adversaries can learn about specific training records beyond population-level information. Real-world attacks have demonstrated that ML-as-a-service platforms leak training data information, reconstruction attacks can recover sensitive data from large language models, and federated learning settings allow adversaries to reconstruct participant data through model snapshot analysis. The indistinguishability game framework provides a formal approach to quantify privacy risk by testing whether adversaries can distinguish between models trained with and without specific data points. Membership inference attacks simulate this game using the likelihood ratio test framework, with approaches evolving from shadow model attacks (training multiple shadow models on population data) to population data attacks (querying target model directly) to reference model attacks (using reference models trained on the population to create accurate baselines).

Membership inference attacks determine whether a specific example was used to train a machine learning model by analyzing the model's loss values on that example; a first-principles approach involves training multiple models with randomly masked training data, collecting loss distributions for examples that were and were not in the training set, fitting parametric distributions (such as Gaussians) to these losses, and using the ratio of probabilities to determine membership status with confidence levels, which reveals that many prior attacks perform poorly at low false positive rates despite good average-case results.

This comprehensive section covers three major privacy attacks against machine learning models. First, membership inference attacks determine whether specific data samples were part of a model's training dataset, exploiting overfitting where member samples produce high-confidence predictions while non-members have uniform confidence. Traditional approaches require shadow models trained on data from the same distribution, but attacks can be transferred across different data distributions because the attack model's input is the target model's posterior outputs. When models are overfitted, their behavior patterns for member versus non-member samples are similar, allowing decision boundaries to transfer. A simplified threshold-based attack achieves comparable performance. Second, when target models only provide class labels, transfer-based and perturbation-based attacks remain effective. Third, online learning creates new vulnerabilities as models are continuously updated with new user-generated data, with posterior differences reflecting the data used to update the model. The data reconstruction attack pipeline involves calculating posterior differences between model versions and using CPMGAN (Conditional Posterior Matching Generative Adversarial Network) to reconstruct specific training samples. The key innovation is the 'best match loss' that forces the generator to learn samples similar to those in the updating set.

The epsilon-delta definition of a limit states that for every ε > 0, there exists a δ > 0 such that if 0 < |x - a| < δ, then |f(x) - L| < ε, meaning we can make f(x) as close to L as desired by keeping x sufficiently close to a; to prove a limit using this definition, one typically uses the four-step method: Given ε > 0, choose δ (often expressed as a function of ε), suppose 0 < |x - a| < δ, and check that |f(x) - L| < ε holds, often requiring algebraic manipulation such as multiplying by conjugates to relate the expressions.

This segment introduces the epsilon-delta definition of limits, explaining that for limit x approaches 1 of 2x + 3 = 5, for every epsilon > 0, there exists delta > 0 such that if |x - 1| < delta, then |2x + 3 - 5| < epsilon. The instructor emphasizes that this rigorous definition ensures physicists, mathematicians, and everyone else are satisfied. The intuitive meaning is that if x is very close to 1, then 2x + 3 is very close to 5.

The epsilon-delta definition of a limit rigorously defines when the limit of f(x) as x approaches a equals L: for any ε > 0, there exists a δ > 0 such that whenever 0 < |x - a| < δ, it follows that |f(x) - L| < ε. This means that no matter how small a distance ε you specify around the limit value L, I can always find a corresponding distance δ around the point a such that all x-values within δ of a (but not equal to a) will produce f(x)-values within ε of L.

This section introduces the epsilon-delta definition of limit, explaining why the vague concept of 'closeness' needs precise mathematical quantification. The video introduces epsilon (ε) to measure how close f(x) should be to the limit L, expressed as |f(x) - L| < ε, and delta (δ) to measure how close x must be to x₀, expressed as |x - x₀| < δ. The formal definition states that for every positive ε, there exists a corresponding positive δ such that whenever x is within δ of x₀, f(x) is within ε of L. This removes vagueness by providing rigorous mathematical conditions.

The epsilon-delta definition provides a rigorous mathematical formulation of limits. For limₓ→ₐ f(x) = L to hold, for every ε > 0 (epsilon), there must exist a δ > 0 (delta) such that if 0 < |x - a| < δ, then |f(x) - L| < ε. This means that for any desired closeness (ε) to the limit L, we can find a corresponding closeness (δ) to the point a that guarantees the function values stay within ε of L.