Adversarial Attacks on AI Models Using FGSM | PyTorch

This section covers advanced adversarial attack methodologies building upon FGSM. The Projected Gradient Descent (PGD) attack iteratively applies FGSM steps, solving max ∇J(x_k)^T × r at each iteration while projecting back to valid input space. This iterative accumulation creates stronger perturbations than single-step FGSM, making PGD one of the most powerful white-box attacks. Randomized smoothing further enhances attack effectiveness by adding random noise to inputs, making attacks less predictable and harder to defend against. These techniques illustrate the ongoing arms race between adversarial attacks and defensive mechanisms in machine learning security research.

Advanced attack strategies include: (1) position-based key search trying different starting positions in ciphertext, (2) interval division dividing [0,1] into segments and searching for initial values producing matching sequences, (3) skipping early iterations to improve security by using values from positions 10, 20, 30, etc. No encryption method is completely secure—all can potentially be broken with sufficient resources. The choice of method depends on specific security requirements and threat models.

AutoDA is an automated framework that discovers decision-based iterative adversarial attack algorithms by constructing a search space of mathematical operations and using efficient search algorithms to explore it, achieving comparable or better performance than human-designed state-of-the-art attacks on CIFAR-10 and ImageNet datasets while maintaining query efficiency.

This section presents advanced differential cryptanalysis techniques. Iterative characteristics solve the problem of extending attacks to many rounds by finding input-output difference pairs that produce zero output difference, allowing repeated application. For DES, certain three-S-box activations produce zero output difference with probability ~1/234. This enables practical attacks on 13-round DES with complexity ~2^47.2 and 15-round DES with complexity ~2^55. The section explains how these probabilities decrease exponentially with additional rounds, establishing practical limits. It contrasts different AR (Attack Rounds) strategies: zero AR uses full-length characteristics for highest probability, one AR adds one round with partial verification, and three AR adds three rounds with more complex analysis. Zero AR attacks are preferred when possible because shorter characteristics have higher probabilities, requiring less data for successful attacks.

Real attackers don't typically find jailbreaks on first try. Instead, they iteratively propose attacks, observe model behavior, and refine their approach. In self-play, attacker models can call the defender model during their own reasoning, observe responses, and iteratively build more sophisticated attacks. This iterative process is essential for discovering hard attacks.

Black box attacks against deep learning systems exploit the transferability property of adversarial examples. The adversary begins by creating and training a model on the same or similar data as the target model. The adversary then crafts adversarial examples for their own model. Due to transferability, these adversarial examples can be applied to the target model, and in most cases (more than 50%, sometimes over 80%), they cause the same misclassification. This enables practical attacks where the adversary does not need to know the target model's architecture or hyperparameters.

Blackbox attacks occur when adversaries lack access to model architecture, parameters, or scores, only receiving labels through an oracle interface. This mirrors real-world deployments on cloud platforms. A key enabler is adversarial example transferability—the property that examples crafted for one model mislead others. Two variants exist: cross-training data transferability (same technique, different datasets) and cross-technique transferability (different techniques, same dataset). Experiments reveal logistic regression shows 95%+ transferability, SVMs show stronger self-transfer, and deep neural networks show intermediate rates (70-80%). This enables attackers to probe one model to evade others.

Black-box attacks represent a more realistic threat model where adversaries lack direct access to model parameters. The key insight enabling practical black-box attacks is the transferability property: adversarial examples generated for one model often successfully fool other models. This phenomenon holds across diverse model types including deep neural networks, SVMs, and decision trees. Experimental results demonstrate that non-targeted adversarial examples transfer with at least 60% success rate between major image classification architectures, while targeted transferability remains poor (under 5%). This asymmetry between non-targeted and targeted transferability creates strategic trade-offs for attackers: non-targeted attacks offer higher reliability but lack precision, while targeted attacks demand more sophisticated generation techniques.

Transferability is a key property of adversarial examples where examples generated to attack one machine learning model often successfully attack other models with different architectures trained on the same task. This occurs because different models may learn similar decision boundaries despite having different internal structures. This property enables black-box attacks, where the attacker does not know the target model's architecture or parameters but can still generate effective adversarial examples by attacking a substitute model. The transferability phenomenon makes it difficult to defend against adversarial attacks since defenses effective against one model may fail against others.

While white-box attacks require complete knowledge of the target model's architecture and parameters, black-box attacks can still succeed through transferability. An attacker can collect training data from the same distribution as the target model, query the target classifier to gather responses, and train a substitute model that approximates the target. Attacks successful against this substitute often transfer to the original target model. Research shows attacks can even transfer across different algorithm types (e.g., from SVMs to neural networks), demonstrating that classifiers tend to rely on similar relevant features regardless of their underlying implementation.

A complete robustness assessment requires multiple complementary tests. The proprietary robustness index provides a 0-100% score by analyzing all available data combinations through different perspectives (72% indicates good robustness). Recency performance measures recent strategy performance (97% score indicates strong recent results). The speaker emphasizes that high ranking in the last three years is particularly important as it reflects the biggest indicator of next year's performance. All tests must align - in-sample/out-of-sample, neighbor values, rank heat maps, proprietary indices, and recency metrics - to provide confidence that probabilities favor the strategy's future success.

The robustness evaluation methodology consists of three interconnected steps. First, construct synthetic data transformations based on domain expertise about how signals change across different settings (hospitals, subjects). Second, apply these transforms to generate transformed data versions. Third, analyze model behavior by comparing embeddings from original and transformed data in latent space, measuring how well mixed or segregated the resulting point clouds are. This integrity measure reveals whether the encoder is robust to the applied transformations. Additional analysis examines prediction uncertainty under transformed conditions. The framework assumes a general encoder-linear layer architecture common in neural networks, making it broadly applicable across different model types.

This section validates the robustness framework through extensive empirical testing. Simulated linear models demonstrate that robustness depends on signal-to-noise ratio: high signal-to-noise yields stable analyses, while low ratios produce susceptibility to small data changes. Real-world applications include: (1) Microcredit study with 16,561 households where removing one point flipped effect sign; (2) Medicaid study with 21,000 participants where 0.05% data removal changed significance despite p < 0.01; (3) Cash transfer study showing robustness when proper outlier handling was applied. These examples demonstrate the framework's practical utility across diverse contexts.

This section presents the formal framework for robustness analysis. It defines robustness as the largest set of environmental deviations under which a system maintains its safety property. The weakest assumption concept is introduced: making fewer assumptions about the environment means greater robustness. The computation process involves: (1) computing the weakest assumption using existing verification techniques, (2) finding the difference between this assumption and the actual environment to identify robust deviations. The section addresses practical challenges: infinite trace sets require grouping into equivalence classes based on first deviation actions, and developers need explanations of what specific environmental deviations the system handles robustly.

This section presents a comprehensive evaluation of different robustness interventions across multiple distribution shifts. The evaluation framework plots in-distribution accuracy on the x-axis and out-of-distribution accuracy on the y-axis, with models trained only on ImageNet following a linear baseline. Models with effective robustness should fall above this baseline. The evaluation covers natural distribution shifts (ImageNet V2, ObjectNet, ImageNet Sketch), synthetic benign corruptions (ImageNet-C), and adversarial examples. The key finding is that training on more data provides consistent gains across all distribution shifts, while robustness interventions show limited effectiveness for natural shifts but work well for synthetic and adversarial cases.