Secure Aggregation in Flower: Salvia+ Protocols for Federated Learning

Shamir secret sharing encodes secrets as polynomials where any t+1 shares reconstruct the secret. With n=3 and t=1, each party holds one point on a line; knowing one point reveals nothing, but two points reveal everything. Addition works by summing shares. Multiplication requires a protocol: parties multiply their shares, secret-share intermediates, then compute linear combinations using Lagrange coefficients to obtain product shares. This enables multi-party computation for any number of parties under honest majority. The protocol achieves perfect security with passive adversaries and can be extended to active adversaries with additional mechanisms. This threshold approach generalizes to distributed storage and other applications beyond secure computation.

Secure Multiparty Computation (MPC) enables n parties to compute functions of joint inputs without revealing more than the final output, assuming a passive adversary can observe communications and corrupt up to n-1 parties. Traditional MPC solutions use linear secret sharing schemes with offline-online phases generating multiplication triples. Multiparty Homomorphic Encryption (MH) offers an alternative approach using four procedures: key generation, encryption, evaluation supporting arithmetic operations, and decryption. MH schemes enable non-interactive evaluation phases but historically suffered from inverted CPU-network cost trade-offs. Recent advances with BGV and CKKS schemes, along with actively maintained libraries and standardization efforts, have made MH-based MPC more practical for real-world privacy-preserving applications.

In the information age, individuals and organizations have enormous sensitive data including bank details, biometric information, and personal records. This data often needs computation while remaining private. The Secure Multi-Party Computation (MPC) problem involves n distrustful parties, each with private inputs, computing a common function f while ensuring correctness and privacy (nothing more than the output should be leaked). The trusted third party solution creates single points of failure, motivating the need for MPC without trusted parties. For any polynomially computable function, there is a corresponding circuit representation. Key building blocks include: circuit garbling (evaluating circuits on encoded inputs revealing only output), oblivious transfer (sender sends two messages while receiver chooses one without revealing choice), secret sharing (distributing secrets among parties), Byzantine agreement (honest parties agree despite corrupted parties), and message transmission. OT is complete for MPC, meaning any MPC protocol can be built from OT.

Secure Multi-Party Computation (SMPC) enables distributed parties to jointly compute functions over private inputs without revealing those inputs. Three adversarial models exist: semi-honest parties follow protocols but try to infer secrets, malicious parties actively deviate, and colluding parties form alliances. Secret sharing ensures coalitions of up to k-1 adversaries cannot recover private values. SMPC eliminates the need for trusted third parties by using cryptographic protocols, enabling privacy-preserving machine learning and collaborative computation without central trust assumptions.

Secure Multi-Party Computation (MPC) enables multiple parties with private inputs to jointly compute a function without revealing their individual data. The ideal scenario involves a trusted third party collecting all data, computing the function, and distributing results. Since such trusted parties aren't always available, MPC protocols emulate this functionality through secure interactions. Functions are represented as arithmetic circuits over finite fields, supporting addition and multiplication operations. Security is measured across three dimensions: adversary behavior (semi-honest vs malicious), security basis (computational vs unconditional), and corruption threshold (honest majority vs dishonest majority). Shamir's secret sharing scheme forms the mathematical foundation, enabling secrets to be shared among parties such that any t+1 shares reconstruct the secret while any t shares reveal nothing. Critical homomorphic properties allow distributed computation: adding two degree-t sharings produces a degree-t sharing of the sum, and multiplying two degree-t sharings produces a degree-2t sharing of the product.

This section examines security vulnerabilities in federated learning systems. Despite the promise of keeping data local, sharing gradients between devices can actually leak sensitive training information. Gradient leakage attacks demonstrate that by observing gradients, attackers can reconstruct original input images or text through iterative gradient matching techniques. The attack works by initializing random inputs and labels, then adjusting them to match observed gradients until recovering the true training data. This reveals fundamental privacy risks in gradient-based collaborative learning approaches.

Distributed machine learning faces complex security and privacy challenges including: (1) Gradient leakage where model updates may reveal training data information, (2) Poisoning attacks where malicious devices inject false updates to corrupt the learning model, (3) Difficulty distinguishing intentional misbehavior from genuine unique observations. Addressing these requires interdisciplinary approaches combining cryptography, differential privacy, and information-theoretic secrecy.

This comprehensive section establishes the foundational context for understanding privacy vulnerabilities in machine learning. The speaker introduces three main attack categories: integrity attacks that mislead models on specific inputs, availability attacks (poisoning) that modify training data to degrade performance, and privacy attacks that allow adversaries to infer information about user data. The focus narrows to privacy attacks, which threaten confidentiality and undermine trust in real-world deployment. The section introduces the federated learning framework proposed by McMahan et al. in 2016, where data remains on local client devices and only model updates (gradients) are transmitted to a parameter server. The fundamental question posed is whether this design actually protects privacy or if gradients reveal information about training data. The section distinguishes between membership inference attacks (determining if specific data points were in training) and reconstruction attacks (recovering actual training samples), establishing that reconstruction attacks are stronger as they inherently include membership inference capabilities. The speaker distinguishes between statistical identifiability (whether gradient information is theoretically sufficient to identify training samples given infinite computational power) and practical identifiability (whether efficient algorithms exist to recover samples). Previous empirical work led to beliefs that gradients alone are insufficient for reconstruction, particularly at random initialization or well-trained models, but these findings were based on gradient matching algorithm failures rather than theoretical analysis. The section then analyzes activation functions: with linear activations, the gradient is a rank-1 matrix representing a linear combination of training samples, allowing only directional information but not individual samples. With quadratic activations, the gradient reveals the second moment (x_i * x_i^T), enabling recovery of the data span but not individual samples due to non-unique matrix decomposition. Both cases demonstrate that these activation functions fundamentally lack the information necessary for reconstruction, regardless of computational power. Shepp's Lemma provides a mathematical method to extract higher-order moments from gradient queries by using random Gaussian initializations. The attacker queries gradients at random Gaussian weights, then applies Shepp's Lemma to extract the third-order tensor representing the third moment of the data. Tensor decomposition then uniquely identifies individual training samples from this tensor, provided they are linearly independent. The section presents an efficient two-step reconstruction algorithm: first, second-order tensor decomposition estimates the span of training data, requiring M to scale linearly with dimension D; second, third-order tensor decomposition on the projected data recovers individual samples. The reconstruction error is bounded by √(D/M), showing that quality improves with more hidden nodes and decreases with higher data dimension. This practical approach reduces computational complexity from D^3 to D, making the attack feasible for high-dimensional data while maintaining theoretical guarantees.

Distributed learning, while offering privacy benefits by allowing data holders to retain private data and only share gradients, still faces significant privacy risks; research demonstrates that speech data can be reconstructed from shared gradients, with Mel spectrogram reconstruction preserving speaker biometrics (over 90% verification success rate) while MFCC reconstruction shows higher distortion due to its decibel scale and high variance, revealing that gradient leakage attacks can expose sensitive personal information in speech recordings.

This section covers privacy risks in machine learning systems. Model inversion attacks allow attackers to reconstruct training data by querying model outputs, as demonstrated with facial recognition systems. Federated learning addresses these risks by keeping raw data local and only sharing model parameters. However, Deep Leakage from Gradients (DLG) attack shows that even model parameters can leak training data through gradient observations. This attack iteratively reconstructs original data from gradient updates, demonstrating that federated learning alone does not guarantee privacy protection.

The Flower framework consists of two key components: (1) Server - handles client sampling, model distribution, and aggregation; (2) Client - implements fit() for local training and evaluate() for model evaluation. Flower is framework-agnostic, supporting PyTorch, TensorFlow, and even non-ML tools like pandas or XGBoost. The server strategy defines how clients are sampled, how models are sent, and how aggregation is performed. The client API is lightweight, requiring only implementation of fit() and evaluate() methods. This design allows developers to adapt existing centralized training code for FL by placing it in the fit() method, making FL accessible to practitioners familiar with standard machine learning workflows.

This section covers creating a server.py file and implementing the server-side logic. The server is started using `flwr.server.start_server()` with an address of 0.0.0.0 to accept connections from any local network interface. The client connects using `flwr.client.start_numpy_client()` with the server address. The federation workflow begins with the server initializing global parameters, requesting initial weights from a random client if none exist, then proceeding through training rounds where clients receive weights, perform local training/evaluation, and return results for aggregation.

This section provides hands-on technical guidance for implementing federated learning with Flower. The Flower CLI offers five essential commands: flower new generates boilerplate code from templates, flower run executes federated learning workloads, flower log tracks progress, flower listen monitors running processes, and flower stop terminates training. The generated project structure includes client and server applications, task definitions, and configuration files. The internal architecture reveals how the server initializes global model parameters and defines aggregation strategies (such as federated averaging), while the client loads local data and implements the Flower Client class for model training and weight return. Both components operate within subprocesses coordinated by the superlink and supernodes, enabling secure, distributed model training across heterogeneous environments.

Client-server communication is the backbone of web applications where clients send requests to servers and receive responses. Flask is a Python web framework that handles this communication. To set up Flask, create a server folder, install Flask using pip, and create a Flask application object using Flask(__name__). The framework manages routing, request handling, and response generation, making it suitable for building web APIs.