BiFair: A Fairness-aware Training Framework for LLM-enhanced Recommender Systems via Bi-level Optimization

📝 Paper Summary

LLM-enhanced Recommender Systems Fairness in Recommender Systems

BiFair improves fairness in LLM-enhanced recommender systems by simultaneously optimizing the frozen LLM's item representations and the recommendation model's projector via a bi-level optimization framework.

Core Problem

LLM-enhanced recommender systems introduce two distinct sources of unfairness: 'prior unfairness' inherited from the LLM's pre-trained representations and 'training unfairness' arising from the recommendation model's training on biased interaction data.

Why it matters:

Existing fairness methods address only one stage (pre-training, in-training, or post-training) and fail to coordinate the mitigation of both representation bias and training bias simultaneously
Despite LLMs improving general recommendation quality, significant fairness gaps persist, where unpopular or minority item groups receive disproportionately low exposure

Concrete Example: In an empirical study on the Amazon Book dataset, while LLM-enhanced models improved over traditional baselines, the worst-performing 25% of item groups still achieved only one-third of the average utility level.

Key Novelty

Bi-level Fairness Optimization (BiFair)

Decomposes unfairness into prior unfairness (from LLM) and training unfairness (from RS), targeting both simultaneously
Uses a bi-level optimization loop where the inner loop trains the recommendation projector and the outer loop refines the LLM-generated item representations based on the projector's feedback
Introduces an adaptive inter-group balancing mechanism that dynamically adjusts loss weights to maximize the entropy of the loss distribution across groups

Architecture

The BiFair framework's bi-level optimization process.

Evaluation Highlights

Empirical study reveals that while LLM-enhanced systems generally improve fairness over ID-based methods, a severe gap remains where the bottom 25% of groups reach only ~33% of average utility
Proposed method BiFair is claimed to significantly mitigate unfairness compared to state-of-the-art methods on three real-world datasets (specific numbers for BiFair not provided in text snippet)

Breakthrough Assessment

7/10

Novel decomposition of unfairness sources in the specific context of LLM-enhanced RS and a mathematically sound bi-level optimization approach. However, the provided text lacks the final quantitative results table for the proposed method.

⚙️ Technical Details

Problem Definition

Setting: Item-side Fairness in LLM-enhanced Recommender Systems

Inputs: User-item interaction dataset D and textual descriptions x_i for items

Outputs: Optimized item representations e_i and user representations e_u for ranking

Pipeline Flow

LLM Representation Generation (Stage I) -> Projector Training (Stage II, Inner Loop) -> Representation Refinement (Outer Loop)

System Modules

LLM Encoder

Generate semantic vector representations from item text descriptions

Model or implementation: Frozen LLM (e.g., pre-trained model)

Projector (Recommendation Model)

Map semantic representations to recommendation space

Model or implementation: Trainable mapping function g_theta (e.g., MLP)

Scoring Function (Recommendation Model)

Estimate likelihood of user-item interaction

Model or implementation: Dot product or similarity function s(e_u, e_i)

Novel Architectural Elements

Bi-level optimization structure where the input representations (Z) are updated in an outer loop based on the convergence of the inner loop model parameters (theta)

Modeling

Base Model: LLM-enhanced RS (e.g., Linear, AlphaRec structures used in empirical study)

Comparison to Prior Work

vs. Reweight/GroupDRO: BiFair uses adaptive weights that change dynamically based on training loss entropy, rather than static or worst-case only focus
vs. Post-training fairness methods: BiFair intervenes during training (in-processing) and representation generation (pre-processing), rather than just re-ranking final outputs

Limitations

Requires bi-level optimization which can be computationally intensive despite approximations
Relies on the assumption that unfairness can be decomposed cleanly into prior and training components
Performance depends on the quality of the finite difference approximation for gradients

Reproducibility

No code or datasets provided in the text. The method relies on approximating gradients for bi-level optimization. Datasets mentioned are real-world datasets like Amazon Book.

📊 Experiments & Results

Evaluation Setup

Top-K Recommendation on real-world datasets

Benchmarks:

Amazon Book (Item Recommendation)

Metrics:

Recall
Hit Rate (HR)
NDCG
Group Utility (Pop-R, Pop-N, Gen-R, Gen-N for fairness)
Statistical methodology: Not explicitly reported in the paper

Key Results

Benchmark	Metric	Baseline	This Paper	Δ
Empirical analysis results quantifying the fairness gap in current LLM-enhanced systems before applying BiFair.
Amazon Book	Relative Utility (Worst 25% vs Average)	1.0	0.33	-0.67

Experiment Figures

Empirical study results comparing fairness of Traditional RS vs. LLM-enhanced RS.

Main Takeaways

LLM-enhanced RSs inherently improve fairness over traditional ID-based RSs (e.g., Linear and AlphaRec show better fairness than SASRec), but a significant gap remains.
More complex architectures like AlphaRec may exacerbate popularity unfairness compared to simpler linear mappings, despite having high accuracy.
The proposed bi-level optimization framework is designed to close the remaining fairness gap by addressing both the frozen representation bias and the training process bias.

📚 Prerequisite Knowledge

Prerequisites

Recommender Systems (RS) basics (user/item embeddings, scoring functions)
Bi-level optimization (inner/outer loop objectives)
Concept of Item-side Fairness (IF) and Group Utility

Key Terms

LLM-enhanced RS: Recommender systems that use Large Language Models (LLMs) to generate semantic representations of items (e.g., from text) which are then projected into a recommendation space

Bi-level Optimization: An optimization problem where one problem (the upper-level) is constrained by the solution to another problem (the lower-level)

Item-side Fairness (IF): The principle that different groups of items (e.g., grouped by popularity or genre) should receive comparable recommendation quality or utility

Projector: A trainable neural network module that maps semantic vectors (from an LLM) into the latent space used for recommendation scoring

Prior Unfairness: Unfairness embedded in the item representations generated by the frozen LLM due to biases in its pre-training data

Training Unfairness: Unfairness introduced during the training of the recommendation model, often due to imbalances in interaction data (e.g., popularity bias)

GroupDRO: Group Distributionally Robust Optimization—a method that optimizes for the worst-case group performance to ensure robustness/fairness

Hessian: A matrix of second-order partial derivatives, used in optimization to understand curvature; computationally expensive to calculate for large models

Finite Difference: A numerical method to approximate derivatives (like the Hessian-vector product) by evaluating the function at two slightly different points

Entropy: A measure of the diversity or spread of a distribution; here used to ensure loss weights are distributed broadly across groups rather than concentrating on a few