Federated AI Simulations with Flower: A 2025 Tutorial for Beginners

The classical federated learning analysis assumes all clients have identically distributed data—a major assumption violated in practice. Different clients may have different label distributions (some hospitals see more sick patients), different feature distributions (different camera resolutions or sensor types), or both. This causes the global model to perform poorly on individual clients' data. Methods like FedProx (proximal gradient descent) and FedBN (using batch normalization to homogenize local data) address these challenges. These approaches modify the learning algorithm to be more robust to non-IID data while still achieving convergence guarantees under certain conditions.

Non-IID (non-independent and identically distributed) data presents significant challenges for federated learning convergence. When local datasets are very different across parties, local models tend to converge toward local optima specific to each party's data distribution. When these local updates are aggregated, the global model converges toward an average of these local optima rather than the true global optimum. This phenomenon slows down convergence compared to centralized training with well-mixed data. Research has shown that federated averaging can perform worse than parallel SGD with large batch sizes when data is strongly non-IID. Solutions include algorithms like Scaffold that introduce correction terms to account for client-specific gradient differences.

Research has developed a comprehensive taxonomy of methods to address non-IID data challenges in Federated Learning, classified into data-based, model-based, and algorithm-based approaches. Data-based approaches include training a warm-up model using shared data on the server, then distributing a subset of this shared data to each client to reduce the difference between server and device data distributions. Another approach involves dynamic sample selection that selects training sample sizes based on locally available data during each interaction, being more tolerant for practical applications by not altering the underlying data distribution.

Data heterogeneity, or Non-IID data, occurs when different clients have different data distributions, creating two main problems: label bias (clients may have different label distributions) and feature bias (clients may have different feature distributions even with the same labels). When clients update parameters in different directions, simple averaging can cancel out useful updates, degrading model performance. Personalized federated learning addresses this by creating client-specific models rather than a single global model. Six main approaches include: clustering (grouping similar clients), model mixing (combining global and local models), parameter separation (sharing only subsets of parameters), knowledge distillation (training local models to mimic global outputs), meta-learning (finding initial parameters that adapt quickly), and neural architecture search (automatically discovering optimal architectures for each client).

Non-IID (Non-Independent and Identically Distributed) data distribution is a fundamental challenge in Federated Learning where data across participants is not identically distributed. Different users or organizations may have very different data distributions, causing the global model to converge more slowly and potentially achieve lower accuracy compared to centralized training. Research is actively exploring techniques to handle non-IID data, including personalized models and adaptive aggregation methods. The convergence process in Federated Learning is more complex than in centralized settings, requiring careful balancing of communication efficiency and model accuracy.

Secure aggregation enables private model update aggregation by having device pairs generate random vectors that cancel out during server summation. This requires efficient generation of large random vectors and handling client dropouts. Diffie-Hellman key agreement enables efficient secret sharing between devices, while threshold secret sharing allows recovery from dropped-out clients. The protocol achieves security against malicious clients while maintaining communication efficiency. Dense secure aggregation faces scalability challenges with n² computational growth, addressed by extending to sparse graphs where each client connects to only log(n) neighbors. Private sliced model download uses homomorphic encryption for targeted model retrieval. Differential privacy protects individual privacy by ensuring query result distributions remain close for neighboring datasets. User-level DP, common in federated learning, protects against adversaries determining user participation. Implementation involves client sampling, update clipping to bound sensitivity, and Gaussian noise addition. Privacy budget accounting tracks epsilon and delta accumulation across rounds. Amplification via sampling improves privacy guarantees with sufficient users. The trade-off between local and central DP determines trust assumptions versus utility.

Single-server secure aggregation enables devices to share secret shares of gradients through a server without revealing individual contributions. Combined with distributed DP where devices add their own noise, this eliminates trust in the server's noise addition behavior. The server only receives already-noisy aggregated updates, enabling open-sourcing client code for verification. This distributed approach provides stronger privacy guarantees but requires careful handling of discrete noise distributions over finite precision representations.

The three-step protocol integrates secure multi-party computation with differential privacy: (1) Clients generate secret s_i and error e_i vectors, compute B_i = A×s_i + e_i, and send encrypted vectors to the server; (2) Clients use packed secret sharing (Pashmir) to securely aggregate all s_i vectors; (3) The server decrypts the sum by combining aggregated s vectors with received encrypted vectors. Error terms from Gaussian-distributed e_i satisfy differential privacy since their sum remains Gaussian-distributed. This approach achieves nearly optimal expansion factors around 1.6, with client runtime under 0.3 seconds and server runtime under 10 seconds.

The interactive heavy hitters algorithm can be combined with secure aggregation (secure multi-party computation) to further reduce trust. This allows the server to see only the thresholded tree without knowing which specific clients contributed which counts. Threshold cryptography can ensure the server only sees extensions that received counts above the threshold. The combination of random sampling and thresholding provides differential privacy without explicit noise addition, leveraging a known result in differential privacy literature.

Understanding privacy requires distinguishing between differential privacy (defining safe functions by adding noise) and private aggregation (providing efficient computation mechanisms)—orthogonal concepts that can be layered together. The critical vulnerability in basic schemes is malicious clients submitting malformed shares to corrupt computations. The solution involves sketch verification schemes that let servers verify data structure without learning actual values. The BGI 16 protocol uses random vectors r and R (component-wise squares), with servers computing inner products and performing constant-size MPC to verify (little_t + little_tb)^2 - big_ta - big_tb = 0. This reduces verification of large inputs (like 10,000 website counters) to constant communication and computation, enabling efficient malicious client detection while preserving privacy.

Cross-device federated learning involves participating nodes (clients) that are typically constrained devices such as mobile phones, embedded devices, or IoT devices. A prominent real-world example is the Android keyboard, where federated learning enables predictive text suggestions, spelling corrections, and word predictions without copying user typing data to Google servers. This approach addresses privacy concerns by allowing users to benefit from improved models while keeping their personal typing data private and never transmitting it to third parties.

Productionizing federated learning requires building comprehensive toolchains addressing current ecosystem gaps: large-scale simulation supporting hundreds of thousands of workers with realistic constraints, automated deployment pipelines converting cloud ML to federated algorithms, and integration frameworks connecting with cloud infrastructure and mobile applications. Advanced techniques include gradient sketching (compressing communication from O(n) to O(log n)), distributed debugging with edge device logging, personalized FL enabling local fine-tuning when data drift occurs, and hierarchical architectures supporting both global sharing and local personalization. Industry adoption demonstrates viability: Gboard processes billions of typing interactions across one billion devices, Siri and Google Assistant leverage FL for personalized experiences, and healthcare applications enable hospitals to collaborate on diagnostic models while maintaining data sovereignty. These deployments prove federated learning's scalability, though broader adoption requires continued maturation of accessible tooling beyond specialized research teams.

Federated learning is a paradigm that enables collaborative machine learning across distributed devices without centralizing sensitive data. The core workflow involves iterative rounds where the server deploys a global model, devices train locally using private data, and the server aggregates updates through averaging. Two primary algorithms exist: Federated SGD (requiring frequent gradient communication) and FedAvg (allowing multiple local training iterations before communication). FedAvg achieves superior communication efficiency—experiments demonstrate 2.8x to 95x faster convergence than federated SGD. The approach preserves privacy by transmitting only model parameters (weights/biases) rather than raw data. Under non-IID data distributions (real-world typical scenario), FedAvg outperforms federated SGD with proper hyperparameter tuning. Google's TensorFlow now supports federated learning simulation, enabling practical deployment without physical device infrastructure.

Federated learning is a distributed machine learning approach where a global model is trained across multiple decentralized edge devices (such as mobile phones) without exchanging raw user data, enabling privacy-preserving AI model training at scale; Qualcomm AI Research has developed a scalable and customizable software framework that extends TensorFlow and PyTorch-based training workflows to Android devices, allowing thousands of worker nodes to collaboratively train models like voice user verification systems while keeping sensitive user data on individual devices.

End-to-end deployment demonstrates connecting multiple client devices to a federated network. Steps include: configuring client apps with superlink address and root certificates; running supernode Docker containers on each client device; executing flower run to join federation. Multiple devices (including Raspberry Pi) connect simultaneously, forming distributed federations. Code changes propagate automatically: developers modify local code, execute flower run, and changes propagate to superlink and supernodes. The same CLI commands manage both simulation and deployment runtimes. This unified workflow eliminates manual code copying to client devices. The demonstration shows real-time federation formation with clients joining from different locations, executing federated training rounds, and producing aggregated results while maintaining data privacy on each device.