Beyond Provable Robustness: Next Directions in Adversarial ML

Added:

The Robustness-Accuracy Trade-Off and the Non-Robust Features Hypothesis

A prominent counter-perspective in machine learning challenges the core pursuit of adversarial robustness, arguing that adversarial vulnerability is an inevitable consequence of maximizing accuracy. Under the 'non-robust features' hypothesis, adversarial perturbations are not bugs or security flaws, but highly predictive, human-uninterpretable features naturally present in datasets. Forcing models to be robust (via adversarial training or certified defenses) constrains them from utilizing these predictive features, leading to an inherent trade-off that degrades standard accuracy. Critics argue that striving for absolute adversarial robustness is fundamentally at odds with building the most capable models. Furthermore, skeptical practitioners point out that certified defenses often focus on mathematically convenient, norm-bounded perturbations (like Lp balls) that fail to reflect realistic real-world threat models, making the academic focus on adversarial ML less relevant to actual system security than broader challenges like data quality, distribution shift, and system-level exploits.

Understanding of standard adversarial attack methodologies, such as the Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD).
Familiarity with the distinction between empirical defenses (like basic adversarial training) and certified/provable defense techniques (such as randomized smoothing).
Foundational knowledge of deep learning optimization, loss landscapes, and mathematical perturbation bounds (e.g., L_p norms).
Basic concepts of statistical learning theory and robust optimization.
Exploring methods to scale adversarial robustness to foundation models, Large Language Models (LLMs), and generative AI systems.
Investigating the theoretical trade-offs between standard generalization (accuracy) and adversarial robustness in neural networks.
Applying robust ML principles to real-world safety-critical deployment pipelines, such as autonomous driving and medical imaging.
Studying systems-level security and auditing frameworks (e.g., NIST AI Risk Management Framework) for validating trustworthy AI.
5.3K views121likes39:56@zicokolter9110Original Release: 2020-04-26

This talk explores the limitations of current provable robustness approaches in adversarial machine learning, which focus on certifying robustness under LP norm threat models (small pixel-level perturbations), and presents ongoing research directions including alternative threat models based on optimal transport/Wasserstein distances, faster adversarial training methods using randomized FGSM, addressing overfitting challenges in robust training, and applying randomized smoothing to certify pre-trained black-box models. The speaker argues that while provable robustness provides valuable theoretical guarantees, the field must move beyond narrow LP norm assumptions to address more realistic security scenarios and broader notions of trustworthiness in machine learning systems.