Adversarial ML: PyTorch Attacks & Defenses

Learning Goal: Attack and defend deep learning image classifiers against adversarial perturbations using techniques like FGSM and adversarial training in PyTorch.

• Estimated Total Study Time: 24 Hours
• Prerequisites: Python programming, basic linear algebra (matrix operations), and foundational calculus (derivatives and the chain rule).

---

Module 1: Foundations of Machine Learning & Neural Networks

Before diving into how neural networks can be fooled, you must first master how they function under normal conditions. This module establishes the mathematical bedrock of artificial neural networks, detailing how information is propagated forward, how errors are quantified, and how models adjust their parameters using gradient descent and backpropagation.

Recommended Videos

Why this video

This video by Grant Sanderson provides the absolute best visual intuition for what a neural network actually is. It demystifies the structure of layers, weights, and biases, showing how high-dimensional pixel inputs are mathematically transformed into class probabilities. Understanding how weights act as filters is crucial because adversarial attacks directly manipulate these mathematical assumptions.

Knowledge Checkpoint

• Explain how a single neuron computes its output from a set of inputs using weights, a bias, and an activation function.
• Understand the role of activation functions (like Sigmoid or ReLU) in introducing non-linearity to the model.
• Comprehend how a 28x28 grayscale image is vectorized and fed into an input layer.

---

Why this video

Gradient descent is the optimization engine behind deep learning. This video visualizes how the cost function forms a massive multi-dimensional landscape and how the network takes steps "downhill" to minimize error. This is a foundational concept because adversarial attacks invert this process: instead of taking steps down the loss landscape to train the model, attackers compute gradients to step "uphill" and maximize prediction error.

Knowledge Checkpoint

• Define what a loss/cost function is and how it measures model performance.
• Explain the concept of a gradient as a vector pointing in the direction of the steepest ascent in the parameter space.
• Differentiate between model training (moving parameters to decrease loss) and adversarial generation (moving inputs to increase loss).

---

Why this video

Transitioning from mathematical theory to implementation requires a robust framework. This crash course introduces PyTorch's fundamental building blocks, including Tensors, computational graphs, and autograd (automatic differentiation). Knowing how PyTorch tracks gradients during a forward pass is directly applicable to extracting gradients with respect to input images for white-box attacks.

Knowledge Checkpoint

• Create and manipulate multi-dimensional PyTorch tensors and move them between CPU and GPU (device).
• Explain the difference between standard Python operations and operations tracked by PyTorch's computational graph (requires_grad=True).
• Write a basic PyTorch training loop sequence: forward pass, loss calculation, loss.backward(), and optimizer.step().

---

Module 2: Deep Learning for Computer Vision in PyTorch

To attack image classifiers, you must first construct one. This module transitions you into computer vision, teaching you how Convolutional Neural Networks (CNNs) process spatial hierarchies in image data. You will learn to construct, train, and validate a CNN classifier in PyTorch.

Recommended Videos

Why this video

Standard dense neural networks struggle with images due to parameter explosion and loss of spatial structure. This video introduces Convolutional Neural Networks (CNNs), demonstrating how sliding filters (kernels) capture localized features such as edges, textures, and shapes.

Knowledge Checkpoint

• Explain how a convolution operation works and how kernels extract specific spatial patterns.
• Understand the purpose of pooling layers (e.g., Max Pooling) in reducing dimensionality and providing translation invariance.
• Visualize how deeper convolutional layers assemble simple features (like edges) into complex representations (like objects).

---

Why this video

This hands-on walkthrough guides you through building a complete image classification pipeline from scratch in PyTorch. It details how to use torch.nn modules (such as convolutional, batch normalization, and fully connected layers) and structures a clean, executable model.

Knowledge Checkpoint

• Construct a custom PyTorch model class by subclassing nn.Module and implementing the __init__ and forward methods.
• Implement preprocessing and data-loading pipelines using torchvision.datasets and DataLoader.
• Evaluate an image classifier's accuracy on a validation dataset.

---

Why this video

This practical code-along reinforces PyTorch best practices. Rob Mulla highlights critical debugging techniques, training loop structures, and the physical setup of neural network pipelines. This clean-code foundation is crucial because security exploits depend heavily on highly structured PyTorch runs.

Knowledge Checkpoint

• Correctly transition models and batches between evaluation (model.eval()) and training (model.train()) modes.
• Trace tensor shapes through convolutional and linear layers to prevent dimension mismatch errors.
• Save model weights using torch.save to load them later for testing adversarial vulnerabilities.

---

Module 3: Introduction to Adversarial Machine Learning

With a functional CNN in place, you are ready to study its vulnerabilities. This module explores why state-of-the-art deep learning models fail catastrophically when presented with tiny, mathematically optimized changes (perturbations) that are completely imperceptible to humans.

Recommended Videos

Why this video

This Stanford lecture is the definitive academic guide to adversarial machine learning. It covers the core mathematical definitions of adversarial perturbations, the linear explanation of why attacks succeed in high-dimensional spaces, and how decision boundaries can be manipulated.

Knowledge Checkpoint

• Define an adversarial perturbation and explain why it remains imperceptible to humans while completely throwing off deep classifiers.
• Explain the difference between white-box attacks (where the attacker has complete access to the model weights) and black-box attacks (where the model is a queryable API).
• Understand the concept of "transferability"—why an adversarial image generated on one model often succeeds on an entirely different model architecture.

---

Why this video

Andrej Karpathy demonstrates how visualizing CNN activations reveals how models "see" images. This visualization maps perfectly to how adversarial inputs exploit the feature-extraction process, altering activations to steer the model towards incorrect target classes.

Knowledge Checkpoint

• Explain how optimization can be used to synthesize images or modify inputs instead of updating model weights.
• Understand how tiny modifications in the pixel space can cascade into massive, incorrect activation signals in the final layers.

---

Why this video

This video reviews the seminal 2013 paper "Intriguing properties of neural networks" by Szegedy et al., which first discovered adversarial examples. Studying this historical breakthrough grounds your understanding of decision boundaries and the global nature of these network vulnerabilities.

Knowledge Checkpoint

• Summarize the primary findings of the Szegedy et al. paper regarding neural networks' generalization behaviors.
• Explain why adversarial examples are not merely random noise, but rather highly structured vectors in the input space.

---

Module 4: Executing Attacks: FGSM and PGD in PyTorch

This module focuses on the practical mechanics of attacking your model. You will write PyTorch code to implement two prominent white-box adversarial attacks: the Fast Gradient Sign Method (FGSM) and its iterative counterpart, Projected Gradient Descent (PGD).

Recommended Videos

Why this video

This tutorial walks you through the step-by-step math and high-level execution of the Fast Gradient Sign Method (FGSM). FGSM works by calculating the gradient of the loss function with respect to the input image, taking the sign of those gradients, and adding a small fraction ($\epsilon$) of that sign to the original image.

Knowledge Checkpoint

• Write out the mathematical formula for FGSM: $x_{adv} = x + \epsilon \cdot \text{sign}(\nabla_x \mathcal{L}(\theta, x, y))$.
• Understand why we take the sign of the gradient rather than the raw gradient values (to maximize the perturbation within an $L_\infty$ bound).
• Conceptualize how an image's pixel values must be clamped after adding noise to remain in a valid range (e.g., $[0, 1]$ or $[-1, 1]$).

---

Why this video

This lecture bridges the gap between single-step attacks (like FGSM) and iterative multi-step attacks (like PGD). It explains why Projected Gradient Descent (PGD) acts as a stronger, universal first-order adversary by taking multiple small gradient steps and projecting the result back into the $\epsilon$-ball around the source image.

Knowledge Checkpoint

• Explain why PGD is mathematically considered an iterative, localized version of FGSM.
• Define the term "projection" in the context of keeping a perturbed image within a strict distance limit (the $\epsilon$-ball).
• Compare the success rates and visual impact of FGSM vs. PGD on a trained model.

---

⚠️ Coding Bridge: Implementing White-Box Attacks in PyTorch

The video pool contains outstanding conceptual and math walkthroughs but lacks line-by-line PyTorch notebooks for attacks. Below is the precise implementation pattern you must master to execute these attacks in your workspace:

import torch

def fgsm_attack(image, epsilon, data_grad): # Collect the sign of the input gradients sign_data_grad = data_grad.sign() # Perturb the image by stepping in the direction of the sign gradient perturbed_image = image + epsilon * sign_data_grad # Clamp the perturbed image to maintain the valid [0, 1] range perturbed_image = torch.clamp(perturbed_image, 0, 1) return perturbed_image

--- HOW TO COLLECT INPUT GRADIENTS IN PYTORCH ---

1. Enable gradient tracking on the raw input batch:

images.requires_grad = True

2. Run the forward pass:

outputs = model(images)

3. Calculate loss against the ground truth labels:

loss = criterion(outputs, labels)

4. Backpropagate (this populates images.grad):

model.zero_grad()

loss.backward()

5. Extract the gradient:

data_grad = images.grad.data

---

Module 5: Defending Classifiers: Adversarial Training

How do we secure our neural networks against these exploits? This final module teaches you the gold standard of defense: adversarial training. You will learn to construct a robust training pipeline that generates adversarial examples inline during training, forcing the model to learn stable features.

Recommended Videos

Why this video

This concise talk outlines the fundamental formulation of adversarial training. Dimitris Tsipras (a leading researcher in robustness) explains how traditional training minimizes standard loss, whereas robust training solves a min-max problem: minimizing the worst-case loss that an attacker can generate.

Knowledge Checkpoint

• Formulate the robust optimization objective (the min-max problem).
• Explain why standard models often rely on highly predictive yet fragile features, and how adversarial training forces the model to ignore them.

---

Why this video

Zico Kolter outlines the frontier of model defense, contrasting empirical adversarial training with provable (certified) defenses. This video provides critical perspective on the trade-offs of adversarial training, such as the drop in accuracy on clean images.

Knowledge Checkpoint

• Understand the fundamental trade-off between clean classification accuracy and robust adversarial accuracy.
• Differentiate between empirical defense (testing against known attacks) and provable defense (guaranteeing safety bounds).

---

⚠️ Coding Bridge: Implementing an Adversarial Training Loop in PyTorch

To build a robust classifier, you must integrate an attack directly into your PyTorch training loop. Use this practical implementation schema:

Conceptual PyTorch Robust Training Loop

for epoch in range(num_epochs): for images, labels in train_loader: images, labels = images.to(device), labels.to(device)

# 1. Temporarily enable gradients on images to generate perturbations images.requires_grad = True outputs = model(images) loss = criterion(outputs, labels) model.zero_grad() loss.backward() # Generate FGSM / PGD adversarial images on-the-fly data_grad = images.grad.data adv_images = fgsm_attack(images, epsilon=0.08, data_grad=data_grad) # 2. Detach adversarial images to stop tracking attack-generation gradients adv_images = adv_images.detach() # 3. Train the model weights using the adversarial images optimizer.zero_grad() adv_outputs = model(adv_images) robust_loss = criterion(adv_outputs, labels) # Still calculate loss against original labels! robust_loss.backward() optimizer.step()

---

Course Map

---

Key People Index

• Grant Sanderson (@3blue1brown): Creator of the award-winning mathematical animation channel. His visualizations of neural networks and gradient descent are globally regarded as the gold standard for intuitive learning.
• Andrej Karpathy: Former Director of AI at Tesla and Co-founder of OpenAI. His CS231n lectures at Stanford laid the foundation for how a generation of engineers understand computer vision and the mechanics of CNN backpropagation.
• Dimitris Tsipras: Machine Learning researcher whose work focuses extensively on why neural networks fail under adversarial conditions and how to construct models that align more closely with human visual features.
• J. Zico Kolter: Professor of Computer Science at Carnegie Mellon University and leading researcher in the field of robust deep learning, empirical verification, and certified adversarial defenses.

---

Final Self-Assessment

Test your mastery of adversarial machine learning by completing this comprehensive checklist:

• Explain why traditional deep learning model evaluation (e.g., standard testing split accuracy) fails to capture security vulnerabilities.
• Perform a manual forward pass trace of an input vector through weights, biases, and activation functions.
• Successfully instantiate a convolutional layer, max pooling layer, and fully-connected layer in PyTorch.
• Configure a training run that saves model checkpoints to disk when validation performance reaches a target accuracy threshold.
• Define the exact difference between an $L_2$ bounded attack and an $L_\infty$ bounded attack.
• Implement a function in PyTorch that takes a clean image batch, accesses the gradients of the model's loss with respect to that input batch, and outputs a set of perturbed FGSM adversarial images.
• Describe the geometric difference between stepping along the raw loss gradient vector vs. stepping along the sign of that gradient vector.
• Implement a multi-step PGD attack that correctly utilizes step size ($\alpha$), overall perturbation bound ($\epsilon$), and projection steps back to the boundary.
• Construct a PyTorch training loop that generates adversarial inputs on-the-fly and trains the model's parameters on those perturbed inputs.
• Quantify the standard-vs-robust accuracy trade-off by plotting validation accuracy across varying values of $\epsilon$ for both standard and adversarially trained models.