Federated Recommendation with Additive Personalization

📝 Paper Summary

Federated Recommendation Systems (FRSs) Personalized Federated Learning

FedRAP improves federated recommendation by modeling item embeddings as the sum of a sparse global view and a user-specific local view, balanced via curriculum-based regularization.

Core Problem

Existing Federated Recommendation Systems share identical global item embeddings across all clients, failing to capture how different users perceive the same item differently.

Why it matters:

Identical item representations ignore user heterogeneity (e.g., users focusing on different attributes of the same item), leading to suboptimal personalization
Transmitting dense item embedding matrices between clients and servers in Federated Learning results in prohibitive communication costs and latency

Concrete Example: Consider the movie 'Titanic': User A may rate it highly for the romance, while User B rates it for the disaster effects. Standard federated models force a single shared item vector for 'Titanic', ignoring these distinct user-specific perspectives.

Key Novelty

Additive Item Personalization with Curriculum Regularization

Decomposes item representation into two parts: a globally shared embedding (general knowledge) and a locally maintained embedding (personal preference), which are summed to form the final representation
Applies a curriculum strategy that gradually increases regularization weights, transitioning the model from learning general features to refining additive personalized details over time

Architecture

The framework of FedRAP showing the interaction between Server and Clients.

Breakthrough Assessment

7/10

Proposes a logical and effective additive structure for personalization in federated settings, addressing both heterogeneity and communication cost, though the core technique is an extension of matrix factorization.

⚙️ Technical Details

Problem Definition

Setting: Federated Recommendation where rating data stays local on clients; each client corresponds to a single user

Inputs: User-item rating matrix R (distributed across n clients)

Outputs: Predicted ratings for unvisited items

Pipeline Flow

Server Initialization (Global Item Embedding)
Client Update (Local User/Item Embeddings + Global Update)
Server Aggregation (Update Global Embedding)

System Modules

Global Item Embedding (C)

Captures general item information shared across all users

Model or implementation: Sparse Matrix (m items x k dimensions)

Local Item Embedding (D) (Personalization)

Captures user-specific deviations/preferences for items

Model or implementation: Dense Matrix (m items x k dimensions)

User Embedding (u) (Personalization)

Represents the user's latent preferences

Model or implementation: Vector (k dimensions)

Novel Architectural Elements

Additive composition of item embeddings: Final embedding = Global (C) + Local (D)
Dual-regularization mechanism: Encouraging sparsity in C while enforcing divergence between C and D

Modeling

Base Model: Matrix Factorization with Additive Item Embeddings

Training Method: Federated Optimization (Alternating Updates)

Objective Functions:

Purpose: Minimize prediction error (Log Loss) between predicted and actual ratings.

Formally: sum of log(1 + exp(-u_i^T (D^(i) + C)_j))
Purpose: Enforce complementarity between global and local item views.

Formally: Maximize ||C - D^(i)||^2 (via minimization of negative norm)
Purpose: Induce sparsity in the global item embedding to save bandwidth.

Formally: Minimize ||C||_1

Key Hyperparameters:

learning_rate: η (eta)
regularization_caps: v1, v2 (max values for λ and μ)
clipping_threshold: τ (tau, for Differential Privacy)

Compute: Time complexity O(kmn) per iteration; Space complexity O((3nm + n + m)k + nm)

Comparison to Prior Work

vs. FedMF: FedRAP allows personalized item embeddings via the additive model, whereas FedMF assumes a single global item representation.
vs. PFedRec: FedRAP maintains a global item embedding (C) to facilitate knowledge transfer, while PFedRec neglects shared item information.
vs. Solow [not cited in paper]: Solow also decouples embeddings but focuses on social recommendation; FedRAP focuses purely on user-item additive structures.

Limitations

Privacy risk persists if server observes consecutive updates from the same client (addressed via Differential Privacy)
Requires tuning of curriculum hyperparameters (v1, v2) for stability
Performance depends on the trade-off between global consensus and local personalization
Quantitative experiment tables were not included in the provided text snippet, preventing verification of specific metric improvements

Reproducibility

No replication artifacts mentioned in the paper. Pseudo-code is provided in Algorithm 1.

📊 Experiments & Results

Evaluation Setup

Federated recommendation on real-world datasets using implicit or explicit feedback

Benchmarks:

Six real-data benchmarks (Rating Prediction / Item Recommendation)

Metrics:

Prediction Accuracy (Metric not explicitly named in snippet, likely RMSE or HR)
Statistical methodology: Not explicitly reported in the paper

Main Takeaways

FedRAP achieves the best performance in the FL setting on multiple benchmarks compared to recent federated recommendation methods.
The additive personalization strategy effectively balances global knowledge sharing and local personalization.
Sparse global item embeddings successfully reduce communication costs and latency.
The curriculum learning approach (increasing regularization weights) helps mitigate performance issues in early training stages caused by the additive model.

📚 Prerequisite Knowledge

Prerequisites

Federated Learning (Client-Server architecture)
Matrix Factorization for Recommendation
Differential Privacy (Gradient Clipping, Noise addition)

Key Terms

FL: Federated Learning—a machine learning approach where models are trained across multiple decentralized devices holding local data samples, without exchanging them

FRSs: Federated Recommendation Systems—recommendation algorithms adapted to the federated learning setting to preserve user privacy

Additive Personalization: A method where the final representation of an item is the sum of a global shared vector and a local user-specific vector

Curriculum Learning: A training strategy where the difficulty or complexity of the task (or regularization constraints) is gradually increased over time

Differential Privacy: A privacy standard that adds noise to computations (like gradients) to ensure the output doesn't reveal whether any specific individual's data was included

L1 Norm: A regularization technique that encourages sparsity (forcing many values to zero) in a matrix or vector