Privacy Preserving Machine Learning Model Personalization through Federated Personalized Learning

📝 Paper Summary

Federated Learning Privacy-Preserving Machine Learning

This paper evaluates a framework combining the APPLE federated learning algorithm with Homomorphic Encryption to achieve high accuracy and privacy on decentralized medical data.

Core Problem

Centralized machine learning endangers user privacy, while traditional Federated Learning (FL) often struggles to balance rigorous data protection with the need for high-accuracy personalized models.

Why it matters:

Sensitive domains like healthcare (e.g., virus detection) require strict privacy that prevents users from sharing raw data centrally
Existing privacy techniques like Differential Privacy can degrade model utility (accuracy) by adding noise
Personalized services often fail in federated settings due to the heterogeneity of user data across decentralized silos

Concrete Example: In a distributed virus classification task (Virus-MNIST), a standard Federated Learning model might aggregate updates that leak patient trends. Alternatively, adding strong Differential Privacy noise could make the model too inaccurate to reliably diagnose the virus, failing the medical objective.

Key Novelty

PPMLFPL (Privacy Preserving Machine Learning with Federated Personalized Learning)

Integrates the APPLE (Adaptive Personalized Cross-Silo Federated Learning) algorithm with four distinct privacy backends: Differential Privacy, Homomorphic Encryption, Secure Aggregation, and Secure Multi-Party Computation
Conducts a comparative performance analysis to identify the optimal trade-off between privacy overhead, execution time, and classification accuracy on medical imaging data

Architecture

The workflow of the APPLE algorithm integrated with Homomorphic Encryption (APPLE+HE)

Evaluation Highlights

APPLE+HE (Homomorphic Encryption) achieves the highest accuracy of 99.34% on Virus-MNIST, outperforming the non-private APPLE baseline (97.41%)
APPLE+DP (Differential Privacy) offers the fastest execution time (22,119 ms for 200 clients) while maintaining 97.48% accuracy
The proposed APPLE+HE method significantly outperforms the Secure Multi-Party Computation variant (APPLE+SMPC), which reached only 85.38% accuracy

Breakthrough Assessment

4/10

The paper provides a useful benchmark of combining existing algorithms (APPLE + HE/DP) rather than proposing a fundamental new architecture. The results are strong but the method is combinatorial.

⚙️ Technical Details

Problem Definition

Setting: Federated Personalized Learning classification task across decentralized clients

Inputs: Images from the Virus-MNIST dataset distributed across 200 clients

Outputs: Predicted class labels for virus types

Pipeline Flow

Client Training (Local Data)
Privacy Preservation (DP/HE/SA/SMPC)
Server Aggregation
Global Model Update & Distribution

System Modules

Local Trainer

Train personalized models on local private data subsets

Model or implementation: LeNet-5 CNN with logistic regression layer

Privacy Engine

Apply cryptographic or noise-based protection to model updates before sharing

Model or implementation: Varies (HE, DP, SA, or SMPC protocols)

Aggregation Server

Aggregate protected updates to form global model components

Model or implementation: Weighted Averaging (customized by APPLE algorithm)

Novel Architectural Elements

Integration of the APPLE adaptive aggregation strategy specifically with Homomorphic Encryption pipelines to maintain personalization accuracy while operating on ciphertext

Modeling

Base Model: LeNet-5 CNN Architecture

Training Method: Federated Learning with cryptographic overhead

Objective Functions:

Purpose: Minimize classification error.

Formally: Cross-Entropy Loss (standard for classification tasks)
Purpose: Bound privacy leakage (for DP variant).

Formally: Sensitivity constraint △GlobalModel ≤ ϵ

Training Data:

Virus-MNIST dataset
Distributed across 200 clients
Non-IID data distribution implied by personalized learning context

Key Hyperparameters:

learning_rate: 0.001
optimizer: Adam
epochs: 50
+ 1 more
clients: 200

Compute: Server clock running times reported (e.g., ~26s for APPLE+HE at 200 clients). Specific GPU hardware not explicitly reported in the paper.

Comparison to Prior Work

vs. FedAvg: PPMLFPL uses APPLE for personalization and adds encryption layers (HE/DP)
vs. APFL: PPMLFPL integrates cryptographic privacy (HE) while APFL focuses only on personalization mechanics
vs. Cryptonets [not cited in paper]: Comparison to pure inference-on-encrypted-data approaches is missing; this paper focuses on training

Limitations

APPLE+SMPC introduces significant computational overhead (highest execution time)
Specific cryptographic parameters (e.g., poly modulus degree for HE) are not detailed
Evaluation is limited to a single dataset (Virus-MNIST) and relatively small image size
The accuracy gain of APPLE+HE (99.34%) over plain APPLE (97.41%) is unexplained, as encryption typically preserves or slightly reduces precision

Reproducibility

Code availability is not provided. Dataset (Virus-MNIST) is public on Kaggle. Hyperparameters (learning rate, optimizer) are listed, but specific encryption parameters (key sizes, noise scales for DP) are missing.

📊 Experiments & Results

Evaluation Setup

Distributed simulation with 200 clients on Virus-MNIST

Benchmarks:

Virus-MNIST (Image Classification)

Metrics:

Accuracy
Precision
Recall
F1-Score
Server Clock Running Time (ms)
Statistical methodology: Not explicitly reported in the paper

Key Results

Benchmark	Metric	Baseline	This Paper	Δ
Comparison of different base Federated Personalized Learning (FPL) algorithms without additional privacy layers on Virus-MNIST.
Virus-MNIST	Accuracy	93.72	97.41	+3.69
Evaluation of the APPLE algorithm integrated with different Privacy Preserving (PP) mechanisms.
Virus-MNIST	Accuracy	97.48	99.34	+1.86
Virus-MNIST	F1-Score	85.39	99.34	+13.95
Server Clock Time (200 Clients)	Running Time (ms)	26121	22119	-4002

Experiment Figures

Bar chart comparing Accuracy, Precision, Recall, and F1-Score for four PPMLFPL variants: APPLE+DP, APPLE+HE, APPLE+SA, and APPLE+SMPC

Main Takeaways

APPLE+HE is the recommended approach for high-stakes medical scenarios, offering the highest accuracy (99.34%) despite encryption overhead
APPLE+DP serves as an efficient alternative when speed is prioritized over maximum accuracy, processing 200 clients ~15% faster than HE
Secure Multi-Party Computation (SMPC) drastically reduces model utility (85.38% accuracy) and increases runtime, making it less suitable for this specific FPL setting
The APPLE algorithm generally outperforms other personalized FL baselines (like APFL, FedRep, and Ditto) on the Virus-MNIST task

📚 Prerequisite Knowledge

Prerequisites

Federated Learning fundamentals (Client-Server architecture)
Cryptographic privacy techniques (Encryption, SMPC)
Neural Network training (Backpropagation, Optimization)

Key Terms

PPMLFPL: Privacy Preserving Machine Learning with Federated Personalized Learning—the paper's overarching framework combining personalization algorithms with privacy mechanisms

APPLE: Adaptive Personalized Cross-Silo Federated Learning—a specific FL algorithm that adaptively balances global and local model components

Homomorphic Encryption (HE): A cryptographic method allowing computations to be performed directly on encrypted data without decrypting it first

Differential Privacy (DP): A technique that adds controlled noise to data or model updates to mask individual contributions and prevent reverse-engineering

Secure Multi-Party Computation (SMPC): A protocol where parties jointly compute a function over their inputs while keeping those inputs private from one another

Secure Aggregation (SA): A technique ensuring the server only sees the final sum of model updates, not individual client contributions

FedAvg: Federated Averaging—the standard baseline algorithm for Federated Learning that averages local model weights

LeNet-5: A classic Convolutional Neural Network architecture used as the backbone model in this study