Academic Databases
Explore Our Semantic Databases
STEM & Tech
Math, Physics, CS & Engineering...
Economics
Macro, Micro, Econometrics...
Business
Management, Marketing, Finance...
Health & Life Sciences
Biology, Medicine, Neuroscience...
Humanities & Social Sciences
Psychology, Law, Philosophy, History...
General Studies
General & interdisciplinary studies...
Explore Topics
Features

Federated Learning with Flower and PyTorch: A Step-by-Step Tutorial

Added: 2026-08-31

Deep Dive

Fundamental understanding of deep learning workflows, including training loops, backpropagation, and loss functions.

MIT 6.S191 (2021): Introduction to Deep Learning

MIT 6.S191 (2021): Introduction to Deep Learning

565.9K

Training neural networks requires quantifying prediction error through loss functions: cross-entropy for classification, mean squared error for regression. The empirical loss averages error across all training examples. Gradient descent optimizes weights by iteratively computing gradients (direction of steepest ascent) and taking steps in the opposite direction, scaled by a learning rate. Backpropagation computes these gradients using the chain rule recursively, propagating errors backward through the network. This enables efficient computation of gradients for all weights in deep networks. The learning rate determines step size—too low causes slow convergence, too high causes divergence. Adaptive optimizers adjust learning rates dynamically based on observed gradients and weight magnitudes.

Click to watch →

Qu'est-ce Que La Backpropagation En Deep Learning?

Qu'est-ce Que La Backpropagation En Deep Learning?

3.8K

Backpropagation is the fundamental algorithm for training neural networks that adjusts connection weights by minimizing prediction error through a four-step process: (1) preparing training data, (2) performing forward propagation to generate predictions using randomly initialized weights and biases, (3) calculating the loss function (e.g., Mean Squared Error for regression) to measure prediction error, and (4) updating all weights and biases using gradient descent with a learning rate. This iterative process repeats across multiple epochs until the loss function reaches a minimum, enabling the network to learn complex patterns from data.

Click to watch →

Backpropagation in Deep Learning | Part 1 | The What?

Backpropagation in Deep Learning | Part 1 | The What?

205.5K

The complete backpropagation workflow involves three phases: (1) Forward pass to compute predictions and loss; (2) Backward pass using chain rule to compute gradients for all parameters; (3) Parameter updates using gradient descent. This cycle repeats for multiple epochs until convergence. The process transforms raw data into meaningful parameter updates through systematic mathematical operations, enabling neural networks to learn optimal representations from training data.

Click to watch →

The spelled-out intro to language modeling: building makemore

The spelled-out intro to language modeling: building makemore

1.1M

This section covers the complete training workflow for neural network language models. We implement cross-entropy loss computation by extracting predicted probabilities for correct labels using advanced indexing, taking logarithms, averaging, and negating to get the loss value. The section demonstrates backpropagation in PyTorch, where calling loss.backward() populates gradient attributes for all parameters requiring gradients. We perform gradient descent updates: params.data -= learning_rate × params.grad. Multiple iterations of forward pass, backward pass, and update constitute one training epoch. The section shows how the neural network approach achieves comparable loss values (around 2.45) to the count-based approach, demonstrating mathematical equivalence between the two methods. This equivalence means both approaches minimize the same loss function but arrive at the solution through fundamentally different computational paths.

Click to watch →

How Machine Learning Works, End to End (Full Basics)

How Machine Learning Works, End to End (Full Basics)

283

Backpropagation is the algorithm that calculates how each weight and bias contributes to error, working backwards through the network. This is essential because neural networks can have millions or billions of parameters (GPT-4 has ~1.8 trillion). The training loop involves: prediction, loss calculation, backpropagation to calculate gradients, and gradient descent to update parameters. Deep learning refers to neural networks with many layers that can learn useful features from raw data without human-designed features. For images, early layers learn edges and curves, later layers combine these into shapes. AlexNet's 2012 ImageNet victory demonstrated deep learning's power. The combination of more data, better training methods, and GPUs enabled this revolution.

Click to watch →

Deep Dive

Proficiency in PyTorch for building, training, and evaluating neural network models.

How to Build Neural Network in Pytorch? | PyTorch Tutorial for Beginners | Simplilearn

How to Build Neural Network in Pytorch? | PyTorch Tutorial for Beginners | Simplilearn

4.2K

Training involves: (1) Creating model instance and moving to device, (2) Defining CrossEntropyLoss and Adam optimizer with learning rate 0.1, (3) Iterating through batches, (4) Moving data to device, (5) Reshaping images to (100, 1, 28, 28), (6) Forward pass, (7) Computing loss, (8) Zeroing gradients, (9) Backward pass, (10) Optimizer step. Evaluation involves: (1) Iterating through test data, (2) Forward pass, (3) Using torch.max for predictions, (4) Comparing predictions with labels, (5) Calculating accuracy as correct/total * 100.

Click to watch →

Machine Learning 1: Lesson 8

Machine Learning 1: Lesson 8

28.4K

This section covers complete neural network implementation in PyTorch. Define layers using nn.Sequential (linear + activation), apply .cuda() for GPU acceleration, and use built-in optimizers. Loss functions (cross-entropy) quantify prediction quality, penalizing both incorrect predictions and uncertainty. Evaluation uses accuracy derived from comparing predicted classes (argmax) against true labels. The complete workflow demonstrates training a logistic regression model achieving ~91.8% accuracy on MNIST, illustrating how PyTorch abstracts complex operations while enabling rapid experimentation.

Click to watch →

Neural Networks Essentials 1: What Is a Neural Network

Neural Networks Essentials 1: What Is a Neural Network

173

In PyTorch, neural networks are defined as classes inheriting from nn.Module. A simple CNN for digit recognition includes convolutional layers, activation functions, pooling layers, flatten layers, fully connected layers, and softmax activation. Training involves defining a loss function (cross-entropy for classification), an optimizer (like Adam), and a training loop that performs forward passes, computes loss, backward passes, and parameter updates. Evaluation uses test data to measure generalization performance.

Click to watch →

Build a Simple Neural Network Using PyTorch | Deep Learning | PyTorch

Build a Simple Neural Network Using PyTorch | Deep Learning | PyTorch

235

To build a neural network in PyTorch, you need to import several key packages: (1) `torch` for tensor operations and basic functionality, (2) `torch.nn` for neural network modules including layers like Linear, (3) `torch.optim` for optimization algorithms like Adam, and (4) `torch.nn.functional` for activation functions like ReLU. These packages provide all the necessary tools to construct, train, and evaluate neural networks.

Click to watch →

Daniel Voigt Godoy - Fundamentals of PyTorch [IndabaX SA 2021]

Daniel Voigt Godoy - Fundamentals of PyTorch [IndabaX SA 2021]

437

PyTorch models are implemented as classes with __init__ defining parameters and forward computing predictions. Parameters with requires_grad=True are automatically tracked. Model.state_dict() provides parameter snapshots for saving/loading. Training and evaluation modes enable different layer behaviors (dropout, batch norm). Sequential containers stack layers for simple architectures, while custom classes handle complex branching structures.

Click to watch →

Deep Dive

Basic knowledge of client-server network architecture and socket communication concepts.

شبكات الحاسوب 12 Socket Programming

شبكات الحاسوب 12 Socket Programming

39.5K

This section introduces the client-server network architecture as one of two primary models (the other being peer-to-peer). It explains that a socket serves as the gateway between the application layer and transport layer, enabling applications to communicate with network protocols. The two types of sockets are UDP (connectionless, unreliable) and TCP (connection-oriented, reliable). A simple client-server application example demonstrates the client reading keyboard input and sending it to the server, which converts it to uppercase and sends it back.

Click to watch →

Socket in Computer Networks: Types, Addresses, TCP & UDP Explained

Socket in Computer Networks: Types, Addresses, TCP & UDP Explained

92.4K

A socket is an endpoint of two-way communication between programs running on a network. In client-server architecture, each program (client and server) requires its own socket. A socket address combines an IP address (for host identification) and a port number (for process identification). When communication occurs, the client selects a free port, the server reserves an agreed port, and data flows through these connected endpoints. Sockets are essential for building network applications like web browsers and FTP clients that use client-server architecture.

Click to watch →

Delphi - Client Server : How to use socket components

Delphi - Client Server : How to use socket components

1.1K

Client-server architecture divides applications into servers (resource providers) and clients (requestors), powering systems like bank queues and hospital registrations. Communication requires local networks where each computer has a unique IP address. Sockets—server and client types—enable bidirectional message exchange. The server listens for connections, accepts requests, and sends responses. Localhost enables same-machine testing. Key events include OnConnect, OnMessageReceived, and OnError. Ports above 4000 avoid conflicts. Message parsing converts text into actionable commands. This foundation enables networked applications to handle real-time communication and data exchange efficiently.

Click to watch →

Communication in Client–Server Systems-Operating Systems-Unit-2-20A05402T

Communication in Client–Server Systems-Operating Systems-Unit-2-20A05402T

9K

Client-server systems involve multiple clients connecting to a central server via the internet for service requests. Communication uses circuits (sockets) defined by IP addresses and port numbers. Well-known ports (0-1023) serve specific purposes: 23 for Telnet, 21 for FTP, and 80 for web servers (HTTP). Each connection requires a unique port pair to ensure proper routing of packets based on destination port numbers.

Click to watch →

Lecture -9 Client Server Concepts DNS,Telnet,Ftp

Lecture -9 Client Server Concepts DNS,Telnet,Ftp

157.5K

The client-server model is the standard approach for network application development. A server is a continuously running process that offers services, while a client is a process that requests services. The server typically starts first and waits for client requests, then services them as they arrive. Servers can be iterative (handling one request at a time) or concurrent (creating new processes for each request). TCP connections require five components: protocol, source IP, destination IP, source port, and destination port. The Berkeley Socket Interface provides standard APIs for network communication, enabling programs to establish connections, send and receive packets. A socket represents a 'half-association' containing information about one side of a connection, enabling interprocess communication over networks.

Click to watch →

Deep Dive

Familiarity with the theoretical concept of Federated Learning and how it differs from centralized machine learning.

A visual Introduction to Federated or Collaborative Learning

A visual Introduction to Federated or Collaborative Learning

20.4K

This section traces the evolution from traditional centralized machine learning to federated learning. Traditional approaches require collecting vast amounts of data from diverse sources and storing it centrally in corporate servers, where algorithms are applied iteratively to generate models for business applications. Federated learning represents a fundamental paradigm shift by decentralizing the training process. Large organizations deploy initial models to billions of distributed devices globally. Each device trains on its local data without transmitting raw information, sending only model improvement updates (deltas) back to central servers. These deltas are aggregated and averaged to refine the global model, which is then distributed back to all devices. This approach enables continuous, collaborative model improvement while keeping sensitive data localized.

Click to watch →

What is Federated Learning?

What is Federated Learning?

97.1K

Federated learning is a decentralized machine learning approach where models are trained across multiple distributed devices or servers without centralizing user data, allowing collaborative model improvement while preserving privacy by keeping raw data local and only sharing aggregated model parameters between rounds.

Click to watch →

А.В. Гасников. Федеративное обучение. Фундаментальные аспекты. Открытые задачи.

А.В. Гасников. Федеративное обучение. Фундаментальные аспекты. Открытые задачи.

1.4K

This section traces the complete story of federated learning from its theoretical origins to practical motivation. It begins with Nesterov's 2003 work on distributed optimization and explains the practical scenario where multiple agents collaboratively solve optimization problems. The lecture discusses the fundamental question of whether distributed approaches outperform single-agent approaches, explaining why the practical success of federated learning was initially surprising given theoretical predictions. It analyzes the impact of local computation steps, showing that for quadratic functions, K local steps improve convergence by K², but for general convex functions, theoretical benefits are limited. This establishes the theoretical framework for understanding distributed optimization systems.

Click to watch →

Exposing Tesla's Insane Data Mining Operation

Exposing Tesla's Insane Data Mining Operation

49.9K

Two fundamentally different approaches exist for training artificial intelligence models: centralized and federated learning. In centralized learning (Tesla's approach), all training data is sent to central servers where models perform computations, then over-the-air updates are distributed back to vehicles. In contrast, federated learning keeps training data on user devices and only sends encrypted model updates to the cloud, grouped with thousands of other updates before decryption. This latter approach prevents individual user data from being inspected before averaging computations. The video argues that Tesla's choice of centralized learning maximizes data utility for their business model while minimizing privacy protections, despite existing technical alternatives that offer better privacy guarantees.

Click to watch →

CS-E4740 Guest Talk from Javier Fernandez-Marques (Flower Labs)

CS-E4740 Guest Talk from Javier Fernandez-Marques (Flower Labs)

446

Traditional centralized machine learning requires sending all user data to a centralized server with powerful GPUs for training, then deploying updated models back to users. This approach raises privacy concerns as sensitive data leaves user devices. Federated Learning reverses this paradigm by moving training to where the data is located—whether on small devices like smartphones or large servers in hospitals or factories. This approach maintains data privacy while still enabling collaborative model improvement across multiple participants.

Click to watch →

Setup

Define Client

Client Logic

Server Setup

Metrics Fix

Conclusion

Setup

0:00

Playing Section

1
Introduces PyTorch federated learning tutorial with Flower framework.
2
Requires installing Flower, PyTorch, and torchvision.
3
Centralized model code is prepared separately for reference.

Deep Dive

Exploring advanced federated aggregation strategies beyond FedAvg, such as FedProx, FedOpt, or FedMA.

Federated Learning: Train Models Across Devices Without Sharing Data

Federated Learning: Train Models Across Devices Without Sharing Data

FedProx is an improved version of FedAvg that addresses its limitations with two key modifications: (1) Handling stragglers - instead of dropping late devices, FedProx takes partial results from devices that haven't finished training, improving fairness and data pool utilization, (2) Limiting weight updates - adds a regularization term to the loss function that penalizes large changes in parameters, ensuring all devices' updates remain close to the global model. This prevents any single device from dominating the update and improves convergence stability.

Click to watch →

Hands-on 13: Building Federated Learning with FedAvg, FedProx, FedDANE & FedSGD

Hands-on 13: Building Federated Learning with FedAvg, FedProx, FedDANE & FedSGD

3.4K

Federated learning trains machine learning models directly on distributed devices (smartphones, edge sensors) without centralizing data. The core workflow involves: (1) partitioning datasets into client-specific slices, (2) local training on private data, (3) sending only model updates to a central server, and (4) server aggregation. Four key algorithms are implemented: FedAvg (averages client model weights proportionally to data size), FedSGD (averages gradients after single batch), FedDANE (solves local sub-problem approximating Newton step with quadratic terms), and FedProx (extends FedAvg with proximal term to penalize deviations and reduce client drift). Core hyperparameters include algorithm selection, number of clients, communication rounds, local epochs, sampling fraction, learning rates, batch size, and device configuration. Two fundamental aggregation approaches exist: model averaging (stacking and averaging weight tensors) and gradient averaging (stacking and averaging gradient vectors). Gradient averaging enables sophisticated server-side optimization strategies like momentum or adaptive optimizers but may amplify noise with skewed data distributions.

Click to watch →

Dr. Chong Shen Ng - Federated Learning Made Simple: A Beginner’s Guide with Flower

Dr. Chong Shen Ng - Federated Learning Made Simple: A Beginner’s Guide with Flower

336

This section explores advanced capabilities extending beyond basic federated learning. Multiple aggregation strategies (FedAvg, FedProx) allow adaptation to different data distributions and security requirements. For large language models, practitioners selectively refine specific layers rather than aggregating all parameters. Advanced features include app bundling (code sent to clients at runtime), isolated execution (minimal containers for security), node authentication (certificate-based access control), and custom message handlers for protocol customization. These features demonstrate how Flower provides flexibility for diverse research and production scenarios while maintaining security and isolation guarantees.

Click to watch →

TUTORIAL: On Device Recommender Systems A Tutorial on The New Generation Recommendation Paradigm

TUTORIAL: On Device Recommender Systems A Tutorial on The New Generation Recommendation Paradigm

151

After clients upload gradients or parameters, the server performs aggregation: (1) FedAvg (Federated Averaging) computes weighted averages based on the number of training samples per client; (2) FedProx adds proximal terms to handle heterogeneous data distributions; (3) Other variants incorporate additional constraints. These aggregation strategies balance learning from diverse client data while maintaining convergence properties.

Click to watch →

Stanford MLSys Seminar Episode 3: Virginia Smith

Stanford MLSys Seminar Episode 3: Virginia Smith

5.7K

Standard federated averaging suffers from instability and divergence in heterogeneous settings because averaging divergent local updates produces suboptimal global models. FedProx addresses this by adding a proximal term to each device's local optimization problem, limiting how much updates can deviate from the global model. This prevents updates from straying too far on diverse data. Additionally, FedProx incorporates partial work from slow devices rather than dropping them entirely. Theoretical analysis shows FedProx achieves 1/ε convergence rate dependent on statistical heterogeneity, with empirical results demonstrating approximately 22% absolute improvement in test accuracy across federated benchmarks.

Click to watch →

Deep Dive

Implementing privacy-preserving techniques in federated systems, including Differential Privacy (DP) and Secure Aggregation.

Brendan McMahan - Guarding user Privacy with Federated Learning and Differential Privacy

Brendan McMahan - Guarding user Privacy with Federated Learning and Differential Privacy

Federated learning provides inherent privacy benefits through data decentralization, but model updates still risk exposing sensitive information. Differential privacy formally protects user privacy by adding calibrated Gaussian noise to aggregated updates, requiring modifications including random user sampling, gradient clipping to bound influence, and noise addition proportional to clipping thresholds. Secure aggregation ensures the server learns only the aggregate without seeing individual contributions. Critically, communication-efficient algorithms like federated averaging provide dual benefits: reduced communication rounds mean fewer privacy budget expenditures, and larger datasets enable stronger privacy through amplification via sampling. With sufficient users, privacy costs can be absorbed through increased computation rather than accuracy loss, making deployment feasible for real-world applications.

Click to watch →

Survey on Privacy/Security in FL

Survey on Privacy/Security in FL

394

Federated learning enables privacy-preserving machine learning by keeping data on user devices while collaboratively training models, but achieving strong privacy guarantees requires combining multiple techniques: differential privacy (which ensures outputs don't reveal individual user data) with secure aggregation protocols (which prevent the server from seeing individual client updates). The key challenge is balancing privacy with model accuracy, as stronger privacy protections typically reduce model performance. Research has shown that using discrete Gaussian noise mechanisms with linear compression can achieve performance comparable to centralized differential privacy while maintaining communication efficiency, though significant challenges remain in extending these techniques to more advanced stateful machine learning algorithms.

Click to watch →

USENIX Security '22 - Efficient Differentially Private Secure Aggregation for Federated Learning...

USENIX Security '22 - Efficient Differentially Private Secure Aggregation for Federated Learning...

887

This paper presents a novel federated learning protocol that combines differential privacy with secure aggregation using Learning With Errors (LWE) to achieve optimal accuracy with minimal communication overhead. The protocol leverages the hardness of the LWE problem to enable efficient one-time padding of large vectors, reducing the secure aggregation problem to aggregation of shorter secret vectors. This approach achieves nearly optimal communication expansion factors (around 1.6) and demonstrates significant performance improvements over state-of-the-art methods, achieving less than 0.3 seconds per client and under 10 seconds server runtime even for large-scale scenarios with 500 clients and 100,000-element vectors.

Click to watch →

Differential Privacy + Federated Learning Explained (+ Tutorial) | #AI101

Differential Privacy + Federated Learning Explained (+ Tutorial) | #AI101

20.9K

This video explains two key privacy-preserving techniques in AI: differential privacy, which adds mathematical noise to data analysis results to prevent re-identification of individuals while still allowing useful aggregate insights (with the trade-off that higher privacy requires more noise and reduces accuracy), and federated learning, which trains machine learning models across distributed devices without centralizing user data, keeping sensitive information local while still improving global model performance; both approaches address different aspects of privacy protection in AI systems.

Click to watch →

PETER KAIROUZ: Federated Learning and Differential Privacy – Part 3

PETER KAIROUZ: Federated Learning and Differential Privacy – Part 3

260

This extensive section addresses the critical challenge of reducing trust assumptions in federated learning systems while maintaining strong privacy guarantees. It introduces three primary approaches for trust relaxation: trusted third parties that aggregate before forwarding to primary servers, trusted execution environments with remote attestation for isolated hardware operations, and multi-party computation protocols for secure aggregation. Single-server secure aggregation is explained as a multi-party computation protocol enabling servers to obtain summed gradients without inspecting individual contributions, though this requires handling device dropouts and collusion resistance challenges. The section then presents distributed differential privacy as a paradigm shift, moving noise addition from centralized servers to individual devices, eliminating trust in server behavior by having each device add its own noise before sharing. The mathematical foundations of this approach are thoroughly examined, including the conversion process from floating-point gradients to integers through L2 clipping, random rotation (flattening) to convert L2 to L-infinity geometry, and scaling with randomized rounding. Two primary discrete noise distributions are analyzed: discrete Gaussian noise, which approximates continuous Gaussians but lacks closure under summation, and Scalon distribution (difference of two Poisson variables), which provides excellent Gaussian approximation and closure under summation. Analysis demonstrates that 12-bit precision fails to match centralized Gaussian performance, but 16+ bits succeed in achieving equivalent privacy-accuracy trade-offs. The section concludes with a practical deployment example of Smart Text Selection model trained with distributed DP and secure aggregation, achieving production-quality results despite weaker epsilon values due to smaller population sizes and lack of amplification opportunities.

Click to watch →

Deep Dive

Addressing non-IID (non-independent and identically distributed) data challenges and client drift in federated networks.

Federated Learning Winter School - 24/11/2020

Federated Learning Winter School - 24/11/2020

1.5K

Non-IID (non-independent and identically distributed) data presents significant challenges for federated learning convergence. When local datasets are very different across parties, local models tend to converge toward local optima specific to each party's data distribution. When these local updates are aggregated, the global model converges toward an average of these local optima rather than the true global optimum. This phenomenon slows down convergence compared to centralized training with well-mixed data. Research has shown that federated averaging can perform worse than parallel SGD with large batch sizes when data is strongly non-IID. Solutions include algorithms like Scaffold that introduce correction terms to account for client-specific gradient differences.

Click to watch →

FLOW Seminar #4: Praneeth Karimireddy (EPFL) SCAFFOLD: an algorithm for federated learning

FLOW Seminar #4: Praneeth Karimireddy (EPFL) SCAFFOLD: an algorithm for federated learning

4.4K

SCAFFOLD is a novel federated learning algorithm that addresses the degradation problem in federated averaging caused by client drift, where taking multiple local updates causes clients to converge to an average of individual optima rather than the true global optimum. The algorithm introduces correction terms (control variates) to each client's update, allowing it to mimic centralized gradient descent more closely and achieve convergence rates comparable to or better than standard SGD while being resilient to data heterogeneity across clients.

Click to watch →

Efficient Federated Learning - Michael Kamp

Efficient Federated Learning - Michael Kamp

244

The classical federated learning analysis assumes all clients have identically distributed data—a major assumption violated in practice. Different clients may have different label distributions (some hospitals see more sick patients), different feature distributions (different camera resolutions or sensor types), or both. This causes the global model to perform poorly on individual clients' data. Methods like FedProx (proximal gradient descent) and FedBN (using batch normalization to homogenize local data) address these challenges. These approaches modify the learning algorithm to be more robust to non-IID data while still achieving convergence guarantees under certain conditions.

Click to watch →

Heterogeneity-Aware Algorithms for Federated Optimization | Prof. Gauri Joshi | CMU

Heterogeneity-Aware Algorithms for Federated Optimization | Prof. Gauri Joshi | CMU

619

Data heterogeneity causes fundamental convergence challenges in federated learning. When clients train on different local datasets, their models drift toward different local optima rather than the global optimum. With many local updates, models converge to local minima, and aggregation produces the average of local optima rather than the true global optimum. The error bound reveals client drift error increases with both gradient dissimilarity and the number of local updates τ. Mitigation strategies include limiting τ and η, adaptive τ reduction, and explicit correction terms like Scaffold. Surprisingly, regular FedAvg often performs well in practice despite theoretical concerns, suggesting the pessimistic gradient dissimilarity assumptions may be too conservative.

Click to watch →

SEMI - 2020 - Applying Deep Reinforcement Learning Techniques in Federated Learning

SEMI - 2020 - Applying Deep Reinforcement Learning Techniques in Federated Learning

1.9K

Non-IID data creates bias that slows convergence because local data on different devices is biased and not identically distributed. The shared data approach (using IID data from cloud alongside private local data) fails because it assumes all data is private and increases communication overhead. PCA reduces model weights to 2D for visualization, revealing that devices with similar data distributions cluster together. K-center clustering groups devices and selects equal representatives from each cluster, improving convergence over random selection. However, this approach is not optimal because model weights change over time, requiring better selection strategies that adapt to evolving data distributions.

Click to watch →

Deep Dive

Scaling and deploying Flower-based federated learning to real-world edge devices, such as mobile phones (Android/iOS) or IoT devices.

Getting the new Arduino Uno Q working with Flower Federated Learning

Getting the new Arduino Uno Q working with Flower Federated Learning

270

This video demonstrates running Flower federated learning on an Arduino Uno Q with Debian, showing that edge devices can participate in collaborative machine learning despite computational limitations, with accuracy improving from 29% (single client) to 47% (two clients) over 20 rounds.

Click to watch →

Flower Summit 2021 | Scaling up Federated Learning with Flower

Flower Summit 2021 | Scaling up Federated Learning with Flower

514

The Flower framework introduces a Virtual Client Manager that enables researchers to scale federated learning experiments by simulating thousands of clients per round on a single machine, using sub-rounds to allocate resources efficiently based on hardware constraints, thereby allowing realistic analysis of privacy, malicious actors, and algorithmic behavior in large-scale deployments.

Click to watch →

Daniel Beutel - Keynote: Federated Learning with Flower: AI's Next Frontier | PyData Global 2023

Daniel Beutel - Keynote: Federated Learning with Flower: AI's Next Frontier | PyData Global 2023

136

Federated learning operates in two main paradigms. Cross-silo federated learning connects different organizational silos: healthcare enables model comparison without data sharing; manufacturing enables predictive maintenance across geographically distributed plants; finance enables cross-regional model training without violating data sovereignty laws. Cross-device federated learning connects individual user devices: Android keyboard prediction uses millions of devices for local training; automotive applications leverage vehicle sensor data while respecting privacy regulations; personal computing applications train models on user devices without collecting sensitive data centrally. Flower is an open-source framework democratizing federated learning development. It supports diverse client types from laptops to mobile phones and vehicles, accommodating cross-silo and cross-device paradigms. The framework enables a unified workflow from development on single machines through simulation on compute clusters to deployment on real device fleets using the same codebase. With over 1,000 dependent projects and 3,300 GitHub stars, it represents a community-driven effort to bring federated learning mainstream.

Click to watch →

Week 11: Lecture 42: Introduction to Federated Learning

Week 11: Lecture 42: Introduction to Federated Learning

2.4K

Federated learning is a distributed machine learning approach designed to train a single neural network model on data distributed across millions of edge devices such as mobile phones and IoT devices. Unlike traditional centralized approaches, it brings the training process to the edge devices themselves rather than requiring all data to be collected at a central server. Edge devices contain large amounts of private user data, and the model learns common features across different users' data while respecting individual privacy. This approach is particularly useful for applications like next-word prediction in keyboards, where each user's typing data is private and distributed across millions of devices.

Click to watch →

Introduction to Federated Learning and Privacy-preserving Machine Learning with Flower (Session 1)

Introduction to Federated Learning and Privacy-preserving Machine Learning with Flower (Session 1)

1.5K

Federated learning represents a fundamental paradigm shift from centralized machine learning by keeping data localized while moving computation to the edge. This approach addresses critical challenges including user privacy, reduced data transfer overhead, lower trust requirements between organizations, and access to distributed data sources. Two primary deployment modes exist: cross-device learning for constrained devices like smartphones (exemplified by Android keyboard predictions), and cross-silo learning for well-resourced organizational nodes (such as hospital collaborations for medical research). The foundational algorithm, federated averaging, enables practical deployment through five coordinated stages: model initialization and distribution, local training, secure parameter transmission, aggregation through weighted averaging, and iterative repetition across multiple rounds. The Flower framework provides comprehensive infrastructure supporting multiple machine learning frameworks (PyTorch, TensorFlow, JAX, MXNet, XGBoost) with separation of concerns between server-side management and client-side execution. Critical to practical deployment is data partitioning, with built-in partitioners enabling rapid experimentation with different data distribution scenarios from idealistic IID setups to realistic non-IID configurations.

Click to watch →

Setup

Define Client

Client Logic

Server Setup

Metrics Fix

Conclusion

Setup

0:00

Playing Section

1
Introduces PyTorch federated learning tutorial with Flower framework.
2
Requires installing Flower, PyTorch, and torchvision.
3
Centralized model code is prepared separately for reference.

14.6K views15220:16@flowerlabsOriginal Release: 2023-08-08

This tutorial demonstrates how to transition from centralized machine learning to federated learning using PyTorch and the Flower framework. The key steps include: (1) implementing a centralized PyTorch model with training and evaluation functions, (2) creating a Flower client that implements get_parameters, fit, and evaluate functions to handle distributed training, (3) setting up a Flower server to coordinate multiple clients using FedAvg strategy, and (4) computing weighted average metrics across clients. The tutorial shows that federated learning achieves comparable accuracy to centralized training while preserving data privacy.

[00:00:00]hi everyone and welcome back to this new installment of the flower tutorial series last episode we looked at a very simple example using tensor flow and today we're going to focus on a similar example but writing our code using pytorch so first it's important to realize that pytorch is quite more verbose than tensor flow so the centralized part of the implementation is going to take a bit more time to write uh that's why we're going to speed it up because we want to focus on the Federated part of the implement if you want to follow along this tutorial you will first need to install uh well flower then torch and torch Vision as [Music] well there we go so once the installation is done I'll go ahead and create a new file centralized py and this file will contain everything um related to the centralized learning so this is just the py to code that we put in a separate file to make it cleaner and this is not relevant for Federated learning so I'll probably speed that P up um and catch you guys afterwards all right so now we should have everything ready for the centralized case so here we first Define the net so this is just the model architecture then we Define a few functions the train function um is just to train the the model for a given number of epoch on a given uh train set and there is the test function uh which evaluates the function and computes the loss and accuracy of the model on a given uh test set and finally some utility functions uh to load the to actually sorry return the data loaders for the training set and the test set and also to return the the model with also converted to the correct device and so this uh should be enough uh to train the model uh in a centralized way and we can see here I've just written um a few steps where we train the model for five E box and then test uh test it and print the results all right so if we go over to the terminal now if I do uh python centralized so it should first yeah download the data it should be quite [Music] quick all right okay nice so now we have our loss and accuracy so accuracy is not excellent but for 5 box uh it's pretty good especially because The Cypher 10 uh task is quite hard um so yeah this is for centralized training and now once we have all those functions we can actually start writing the code for our client so if I create my Cent py so the first things we're going to import are actually from the centralized uh file so we'll do from centralized I'm going to import so load data load model and then train and test that should be all we need from there and then obviously I'm going to import flower so flwr and I'm just going to import as FL to make it shorter during um um when I write it okay so the first thing we are going to need um for this client file is a new function called uh set parameters and this is just going to be a utility function um to give us the same kind of functionality we had with tensor flow where with tensor flow we could just update the weights easily uh in P TCH this is not uh written by default so we had to we have to write it by ourselves and so this function will take a model um as a parameter and then the parameters we want to set to the model and so um this will actually return the model itself I'm just not going to put the typing here to make it more simple okay so the first thing we're going to do is create a parameters deck which is going to be um so the input model State deck the state deck contains some information about the layers uh of the model as keys and then uh as values it should be the parameters but we actually want to update those parameters so what we're going to do is to only take the keys and to associate them with the new parameters and then to create the state dict itself it needs to be an ordered dict or sorry dict um and the only thing we need to do so we're going to use the same key so I'm going to write it like this so it's a bit easier to follow so we're going to take the keys and values from the parents dict and we're going to associate it each key to the value converted to a torch tensor so we also need to import torch [Music] here uh torch and also um [Music] um ordered yeah move that down okay so now we have a valid State dat and we only need to load it into our model so load load State de and we're going to set the option strict to true because the models uh should always be of the same dimensions and should contain the same layers okay so that's good um yeah we can even return the model I don't think it's necessary to actually return it so we can pass uh the object itself to the function okay so now that we have this set parameters utility function uh if we want to upd update the weights of the model we can just so well first we're going to Define our Ro model so in this case it's going to be r model okay and then um we can also download the data so train direct test Lo data here we go okay and now say we want to update the model we can just do set parameters then give our net and then the new parameters we want to set it and this is going to be very useful in our flower client okay so flower client is going to be a subass of fl. client.

[00:07:41]NP client there we go and just as in the tens of example we're going to first Define get parameters function which takes just a config as argument and in this case we can't just to model do get parameters because this function is not implemented um in pytorch we need to return the parameters manually so we first convert them and pass them to the CPU in order to be able to convert them to uh NP and then this is just using the um this T di um [Music] 10 all right so we're accessing the state dict of the model that we Define here and then um we're only taking the values which correspond to the parameters of this model of this state sorry and converting them to nump arrays and uh returning this as a list all right so this function is done and then we can focus on the next one which is going to be the fit function which takes parameters and config we actually ignore the config because we don't need it and in this function what we do um is we set the parameters to um the parameters we just received from the server you can actually do net equals um and then train um [Music] I'm just going to train it for one inut by default we could do more but um then it takes a bit longer in here um we're going to just return get parameters and we need to pass into the config which is empty in this case then the length of the train loer so train right um oh sorry um data set actually and this last one is if we would want to return a metric in this case we don't return anything because we don't compute The Matrix in the train function so this should be good and finally the evaluate function which takes parameters as well in config the only difference here um I'm actually going to remove this uh net equ because it shouldn't be um NE necessary right and here so we're going to do very similarly we set the parameters given by the by the server um and we update the model with it and then instead of train we're going to test and test returns loss and accuracy test and then net test loader all right note that those um uh variables net and test loader are defined globally which is not necessarily the best practice it's just to show quickly how it would work but here we could have an init function for the client where we actually pass in those models um those this model and those uh data sets right and here we're going to return it's uh so first the loss uh then the length of the test loader data set and finally the metric is DED and this time we actually calculated the accuracy so we can return this okay now we should have everything need for um our uh client to work uh in order to start it we can just add the f client start by client in this case um we actually use the numpy client wrappers so here and here to make it a bit simpler uh maybe we'll do a tutorial later on talking about the differences between the the row flower clant and the numai clant but this is out of the scope of this uh tutorial um okay and then we just need to pass in the client we want to use in this case it's the flower client we Define well all right so if we start this client it will try to um it will start of the nump C and try to connect to this address but now uh at the moment there is nothing at this address and this is why we need to create the server uh py file which is going to be very basic I'm just going to import flower as FL and then just um server. start server and here the server address it's actually going to be 0.0.0.0 8080 this is just so other machines on the same network can access uh This Server as well but in my case we're just going to be using the we're going to start the CLI and the server on the same machine so it doesn't matter too [Music] much right um and for the strategy we can use strategy fed average I'm going to use fed average with uh default parameters all right so now we should have everything I don't know if I've missed any Imports here um I mean we we're going to try and see how it goes so on this left terminal I'm going to start the server we go and then here I'm going to start one client and the other so here okay oh I guess yeah there's a typo obviously hopefully it's the only one um [Music] here all right let's check in quickly and see how it goes so again we start the server and then clients okay so now they're connected establish a connection the server actually requested initial parameters from one of the clients and then sent back uh those parameters it initialized uh to uh the other client as well um and then it was able to start the fit round with those parameters to the client um run the training locally sent back their um the resulting parameter to the server then the server Aggregates them so after this fit round and it sends back the aggregated weights to the the two clients which then run the uh test function the evaluate function and send back the results to the server and this ends the first round and then they start uh this whole process again so we've only have uh three rounds um so it should be quite quick uh it's not going to be comparable to the centralized results because we have less rounds but here we can see that the um the loss function the loss decreases quite a bit and so one thing that you might notice here is that we don't actually have uh the accuracy displayed and this is something I didn't touch upon on the last tutorial but I will quickly go through it now so the first thing we're we're going to need to do uh here is actually to modify our uh server file um we're going to add a function going call it weighted average um because we actually need to Pro um to provide a way for our strategy to know what to do with the different uh results it's going to um receiving those metric dates because anything can be passed uh in the metrix dictionary so here um we're first going to uh use this so um n examples times M accuracy c n example is going to be from Matrix and M as well Matrix okay and and then we're going to list the examples [Music] here Matrix and finally we can just return uh so we're going to return it as a dict with accuracy being the sum of the accuracies over the sum of all examples all right now we just need to pass it to our strategy um and the argument for the strategy is going to be evaluate uh Matrix agation function and right okay perfect so that should be it now if I set up the server again and my two clients we going to run exactly the same um workload except that now we're going to have at the end uh the metrix it's going to be just the weighted average of uh the metrix from the two clients okay so now it should finish quickly it's in the last round of evaluation perfect and now we can see uh the accuracy and it's actually quite comparable to what we had in the centralized case I think we had 49 um if I remember correctly okay nice so there you have it um this is everything you need to go from centralized to ferate with by torch this was a very uh quick and early example there's a lot more to it and we'll try to release many tutorials um to go through more topics and go more in depth on certain subjects as for now be sure to subscribe to the channel and especially tell us uh what you would like to see next and one thing I wanted to touch upon as well um was to go over the GitHub Roo it's very important for us that you uh give us a star on GitHub note that on the website here we have this button but it does not automatically start you actually need to press here afterwards and this is also very important for us to be able to see uh the community grow and to be able to understand better the uses and the needs of the community yeah that's it on my side see you guys [Music]

Up Next

PDMS Microfluidics Tutorial: Preparing a Test Pattern

@mitocw

41.1K views•2013-11-15

Triumph of Orthodoxy Icon: Byzantine Art & History Explained

@BenCallan

2.1K views•2024-08-06

FastAPI vs Flask vs Django: Choosing the Right Python Web Framework

@TechWithTim

302.5K views•2024-05-26

Game of Thrones Opening Credits: A Cinematic Analysis

@gameofthrones

46.3M views•2011-04-18

Related Study Plans & Knowledge Roadmaps

Structured learning paths in General & Interdisciplinary Studies

Browse General & Interdisciplinary Study Plans

general-studies

Elevator Engineering: History, Physics & Cities

01From Hoists to Safety Brakes: The Dawn of Vertical Lift

02Hydraulic vs. Traction: The Engineering Physics of Elevators

03Modern Speed, Safety, and Algorithms

+ 2 more learning modules

Explore Roadmap

general-studies

Rubber: Material Science, History & Geopolitics

01Origins of Natural Rubber & Mesoamerican History

02The Chemistry of Natural Rubber & Vulcanization

03The Geopolitics of the Rubber Boom & Colonization

+ 2 more learning modules

Explore Roadmap

general-studies

Containerization: Evolution, Logistics & Trade

01Ancient Trade & Bulk Cargo: Amphorae to Barrels

02The Industrial Revolution & Cardboard Packaging

03The Containerization Revolution: Malcom McLean

+ 2 more learning modules

Explore Roadmap

general-studies

Textile Tech: Evolution, Labor & Global Impact

01Pre-Industrial Textiles: Handloom & Cottage Industry

02Mechanization: The Spinning Jenny to the Power Loom

03The Factory System & Global Labor Transformation

+ 2 more learning modules

Explore Roadmap