Federated Learning: AI Training Explained | Tutorial

SHERPA is an explainable AI-based defense mechanism for federated learning that detects data poisoning attacks by analyzing feature attribution patterns using SHAP values; the system clusters clients based on their SHAP-based feature attributions and identifies malicious clients by detecting those whose attributions deviate significantly from their assigned cluster, achieving up to 98% detection accuracy even when 80% of clients are poisoned.

This paper presents an information theory approach to defend against data reconstruction attacks in federated learning by quantifying and constraining the information increment during model training. The key insight is that data reconstruction attacks succeed because model parameters accumulate information from the training data through each optimization round; by adding carefully calibrated noise to either model parameters or the data space, the information increment can be bounded, thereby limiting reconstruction accuracy. The proposed method achieves 25% higher reconstruction error than differential privacy while maintaining equivalent model accuracy, and operates efficiently in the data space (rather than parameter space) to achieve approximately five times faster training speed.

Adversarial attacks involve adding carefully crafted perturbations to input data that cause deep learning models to make incorrect predictions while remaining imperceptible to humans. In federated learning, malicious clients can launch poisoning attacks by corrupting their local model updates before sending them to the server. The challenge is that defenders don't know which clients are malicious. Most security mechanisms rely on the honest majority assumption—that honest clients form the majority. If malicious clients control the majority, defenders cannot distinguish between honest and malicious updates since both would appear to be the 'normal' behavior.

Federated learning introduces new security vulnerabilities beyond traditional machine learning. Inference attacks allow adversaries to reconstruct user data from shared model parameters by observing parameter differences across training rounds. Poisoning attacks exploit the distributed nature by having compromised users submit manipulated parameters to degrade model performance. These vulnerabilities arise because federated learning shares model updates rather than raw data, creating attack surfaces that differ fundamentally from centralized systems.

Federated learning enables decentralized machine learning where a central server coordinates multiple clients who train on private data without sharing raw information. Each iteration involves the server distributing a global model, clients training locally and returning updates, and the server aggregating improvements. However, this architecture faces critical security vulnerabilities through poisoning attacks: untargeted poisoning reduces overall accuracy by corrupting training data; targeted label flipping attacks mislabel specific classes; backdoor attacks embed hidden triggers; and model poisoning manipulates aggregated updates directly. Existing defenses suffer from unrealistic assumptions about server access to clean data or rely on distance-based anomaly detection that fails when benign clients have non-overlapping data distributions, making robust defense challenging.

This lecture presents two complementary privacy notions: Secure Multiparty Computation (SMC) enables parties to compute functions on private inputs without revealing those inputs, while Differential Privacy ensures that query results do not reveal whether any individual's data was present in the database. The speaker demonstrates how to combine these concepts by integrating distributed noise generation into SMC protocols, addressing challenges posed by malicious adversaries who might otherwise manipulate the noise addition process. This integrated approach provides both computational security (preventing information leakage during computation) and differential privacy guarantees (preventing inference attacks on the final output).

Differential privacy and secure multi-party computation (MPC) address complementary questions in privacy-preserving machine learning: DP answers 'what' functionality is safe to compute, while MPC answers 'how' to compute it securely. The shuffle model of computation transforms local differential privacy guarantees into stronger central differential privacy guarantees through amplification by shuffling, enabling more scalable privacy-preserving ML systems. This integration allows researchers to combine the strengths of both approaches for practical privacy-preserving machine learning applications.

This video presents a protocol for private record linkage that combines differential privacy with secure multi-party computation (SMC) to achieve three key goals: correctness (identifying all matching records), end-to-end privacy guarantees, and sub-quadratic computational/communication costs. The approach uses blocking with Laplace noise to reduce secure pairwise comparisons while maintaining privacy, and introduces a new privacy notion called 'alpha-constraint differential privacy' to formally analyze the composition of differential privacy mechanisms with SMC protocols. The protocol addresses the challenge that traditional SMC approaches are computationally infeasible for large datasets (e.g., 31.7 years for 1 million records), while naive blocking introduces information leakage that can be mitigated through differential privacy techniques.

Combining differential privacy with secret-sharing based MPC requires addressing fundamental efficiency challenges. Standard noise generation mechanisms like Laplace distribution are computationally expensive in distributed settings. Research explored geometric noise approximation and found significant performance penalties—queries increased from ~325 seconds without privacy to substantially longer durations with added noise. Privacy budget allocation remains an open research question similar to cryptographic key size recommendations, requiring community consensus rather than single-project solutions. Practical applications in military coalition operations and humanitarian assistance demonstrate real-world deployment challenges, where efficiency penalties push systems toward less secure alternatives like order-revealing encryption. This tension between privacy, security, and performance drives ongoing research into more efficient MPC-friendly differential privacy mechanisms.

SMPC protects private inputs during execution but faces challenges after query release: clients can deduce private inputs through repeated querying, and data owners may want to query the union of their data with peers. Differential privacy injects precisely calibrated noise into query answers to protect individual privacy, ensuring answers are sufficiently perturbed so no one can learn about private records. Key properties include composable privacy budgets (ε can be divided among queries) and computational differential privacy for computationally bounded adversaries. Combining SMPC with differential privacy extends PDF guarantees to cover prior knowledge of data owners about query answers, achieving compute privacy, strong data privacy, and improved efficiency.

This section presents TensorFlow Federated (TFF) as a specialized development environment for federated learning, designed for both research and production deployment. TFF introduces new programming abstractions because federated programs span distributed entities where communication is essential logic rather than secondary concern. Core concepts include federated values representing collections from multiple participants with federated types specifying constituent types and placements (clients or servers). Federated operators like mean and map model distributed computations: broadcast distributes values from server to all clients, while map applies computations to each client independently. TFF computations are defined with Python decorators specifying federated types; when called, they get traced into serialized representations independent of original Python code. This enables execution across different backends while maintaining strongly typed programming over distributed data. The framework supports compositional extensions for adding compression and differential privacy through modular code modifications.

This section provides a comprehensive guide to implementing federated learning with TensorFlow Federated (TFF). TFF is Google's extension to TensorFlow that provides tools for federated learning, combining TensorFlow's model creation capabilities with federated communication to use information from clients, join models, and produce aggregation processes. The framework includes different libraries of federation algorithms, runtimes, and datasets for experimentation. TFF consists of two main parts: the federated learning process (high-level interfaces for engineers to use federated algorithms with existing TensorFlow models) and the federated core (master classes for creating new federated algorithms, including client and server processes for information exchange and model aggregation). The implementation process involves: creating a TensorFlow model; creating a TensorFlow Federated model using the grappler system; creating federated datasets from client information; configuring the TFF learning model with accuracy metrics; defining input data structures, loss functions (e.g., sparse categorical cross-entropy), and evaluation metrics; configuring optimization algorithms for clients and servers (which can be different); initializing the federated machine learning process to get the system state; defining federated training data and metrics; and executing training steps through iterations. The key benefits include privacy preservation (data stays on local devices and only the model is sent), cost reduction (instead of sending gigabytes of image data, only the small model of 100-1000 kilobytes is sent), and network efficiency.

The Flower CLI provides five core commands for building federated learning applications: flower new creates apps from templates reducing boilerplate code; flower run executes workflows; flower log views status; flower stop terminates execution; flower list manages federations. A Flower app follows standardized pyproject.toml structure including publisher info, server/client module locations, and configuration tables for federated learning parameters (server rounds, fraction_fit, local epochs). The framework-agnostic design converts all model weights to NumPy arrays for transmission, enabling compatibility with PyTorch, TensorFlow, and other frameworks. Server apps implement aggregation strategies reading configurations and combining client weights. Client apps define neural networks, data loaders, and implement fit() and evaluate() methods for local training and assessment. This standardized structure enables reproducible research and easy modification of federated learning hyperparameters.

TensorFlow Federated (TFF) is an open source project providing: (1) A federated learning API with implementations of standard algorithms like federated averaging and evaluation, (2) A federated core API for expressing arbitrary federated algorithms, (3) A type system including data placement information (e.g., float32 at clients), (4) Expressive declarative syntax for describing distributed systems. Operators like federated mean and federated broadcast enable computing aggregates across devices while keeping data local. TFF enables simulation and experimentation in Google Colab, making federated learning accessible for research and production use.

Implementing federated learning with Keras models in TFF involves creating standard Keras models, wrapping them as `tff.learning.Model`, and using `tff.learning.build_federated_averaging_process` to construct federated training algorithms. The process initializes server state and executes rounds where the server distributes models, clients perform local training, and the server aggregates updates. Evaluation uses `tff.learning.build_federated_evaluation`. The Federated Core API provides lower-level distributed systems programming, introducing federated types with placement (clients/server) and data types. Federated aggregation protocols like averaging connect distributed values. Complex computations combine broadcast (server-to-client distribution), map (local computations per client), and aggregation operations. For example, broadcasting thresholds, mapping comparisons locally, then aggregating results to compute fractions exceeding thresholds. This enables sophisticated distributed analytics while keeping raw data decentralized.

Cross-device learning (mobile devices) faces unique challenges: (1) Limited client participation - users typically participate only once or rarely due to privacy concerns and communication costs; (2) Unreliable connectivity - devices may lose connection due to battery issues or poor network coverage; (3) Non-IID data - different users have very different data distributions. Applications include Google's Gboard keyboard (predicting next words using 1000+ rounds with millions of users) and LINE's sticker recommendation system. Adaptive optimization algorithms like Adam can be adapted for federated learning to handle sparse participation and unreliable connections.

One Keyboard is a macOS application that enables users to type on multiple connected devices (iPhone, iPad, other Macs) using a single physical keyboard, with each device assigned a specific keyboard shortcut for quick switching; however, it has limitations including lack of support for Android devices, restricted text formatting capabilities, and inconsistent Enter key functionality across messaging applications.

The physical QWERTY keyboard features an alt button for alternative characters, integrated symbols, and a virtual keyboard for additional symbols. Voice dictation activates by holding the voice button. Keyboard shortcuts include B for bottom, T for top, L for reload, S for search, H for history, W for tabs, K for bookmarks, and R for reader mode. Universal search types on the keyboard and searches across calendar, messages, and other content. Blackberry Assistant provides voice-activated assistance for weather, appointments, and media control. Quick Settings provides access to toggles for flashlight, Wi-Fi, alarm, and other features. Do Not Disturb profiles include Normal, Phone Call Only, Vibrate Only, Silent, Meeting, and Bedside modes with customizable app permissions. BlackBerry Blend enables cross-device integration with iPads, Macs, and PCs for real-time access to BlackBerry Hub, email, BBM, and SMS. Device monitoring shows battery, data, storage, and CPU usage with Battery Saver Mode available.

Traditional analytics tools cannot easily cross-device data because mobile apps exist as separate properties. Analysis revealed that 40% of mobile app users abandoned purchases at the product or cart page, with 30% abandoning combo products. Cross-referencing with desktop purchase data showed 79% of these users compared multiple products, and 77% gave low ratings for ease of purchase. The solution was adding a dedicated promo section in the app menu, resulting in a 5% increase in combo product sales.

Analyzing data from a single silo provides limited insights. True business intelligence emerges when combining signals from multiple sources—such as OS logs, authentication events, and cloud device telemetry—into unified analysis. This cross-silo approach reveals complete event patterns that individual data sources cannot provide alone.