Mitigating Propensity Bias of Large Language Models for Recommender Systems

📝 Paper Summary

LLM-enhanced Recommender Systems Debiasing in Recommender Systems

CLLMR mitigates LLM propensity bias in recommender systems by encoding side information with interaction spectral data to prevent dimensional collapse and removing biased causal effects via counterfactual inference.

Core Problem

Aligning LLM-generated side information with collaborative filtering embeddings causes 'dimensional collapse' due to lack of structural data, while LLM 'propensity bias' (stereotypes) distorts recommendations.

Why it matters:

LLM biases towards certain cultures or viewpoints create unfair user experiences and homogeneous representations
Dimensional collapse restricts the recommender's ability to capture nuanced user preferences, forcing representations into a low-dimensional subspace
Blindly removing all bias strips away valuable real-world knowledge (user/item propensity bias); only 'unrelated' bias should be removed

Concrete Example: For User ID 5596 in the Amazon dataset who reads thrilling books, an LLM overlooks this specific history and generalizes the preference simply as 'entertaining' due to propensity bias, ignoring the nuance.

Key Novelty

Counterfactual Large Language Model Recommendation (CLLMR)

Spectrum-based Side Information Encoder (SSE): Implicitly embeds interaction graph structure into LLM text representations and injects noise to prevent the vectors from collapsing into a limited subspace.
Causal Bias Mitigation: Models LLM propensity as a mediator variable in a causal graph, allowing the system to mathematically subtract 'unrelated propensity bias' (the indirect effect) while keeping useful preference signals.

Architecture

A causal graph (DAG) illustrating the relationships between Exposure (X), Mediator (M), and Outcome (Y), along with counterfactual scenarios

Breakthrough Assessment

7/10

Addresses a critical, specific failure mode of LLM-RecSys integration (dimensional collapse + propensity bias) with a theoretically grounded causal approach, though the text provided lacks the empirical results to confirm magnitude of improvement.

⚙️ Technical Details

Problem Definition

Setting: LLM-enhanced Collaborative Filtering where user/item side information is generated by LLMs and aligned with interaction-based embeddings

Inputs: Historical user-item interactions, LLM-generated user/item descriptions

Outputs: Recommendation probability/score adjusted for propensity bias

Pipeline Flow

LLM Generation: Generate user/item side information
Text Embedding: Project descriptions to feature space
Spectrum-based Encoding (SSE): Refine representations with graph structure and noise
Alignment: Align side representations with collaborative representations (e.g., from LightGCN)
Inference: Apply counterfactual adjustment to remove bias

System Modules

Large Language Model

Generate human-like descriptions/profiles for users and items based on available data

Model or implementation: Not specified in text

Spectrum-based Side Information Encoder (SSE)

Encode text embeddings while incorporating structural information from historical interactions to prevent dimensional collapse

Model or implementation: Spectrum-based encoder (details in missing sections)

Causal Inference Module

Calculate Total Effect (TE) and subtract Total Natural Indirect Effect (TNIE) to remove bias

Model or implementation: Counterfactual inference mechanism based on causal graph

Novel Architectural Elements

Integration of spectral graph information directly into the side-information encoder (SSE) specifically to combat dimensional collapse in LLM alignment
Causal graph topology explicitly modeling 'LLM Propensity' as a mediator between Exposure (User/Item) and Outcome

Modeling

Base Model: LightGCN (mentioned as backbone in motivation)

Training Method: Counterfactual LLM Recommendation (CLLMR) framework

Objective Functions:

Purpose: Calculate Total Effect of exposure on outcome.

Formally: TE = Y_j(x, M_j(x)) - Y_j(x*, M_j(x*))
Purpose: Calculate Pure Natural Direct Effect (unbiased preference).

Formally: PNDE = Y_j(x, M_j(x*)) - Y_j(x*, M_j(x*))
Purpose: Calculate Total Natural Indirect Effect (bias to be removed).

Formally: TNIE = TE - PNDE = Y_j(x, M_j(x)) - Y_j(x, M_j(x*))

Training Data:

Amazon dataset (mentioned in example)
Yelp (mentioned in abstract/intro implied)
Steam (mentioned in abstract/intro implied)

Comparison to Prior Work

vs. TALLRec/Agent4Rec: CLLMR specifically targets 'propensity bias' and 'dimensional collapse' via causal inference and spectral encoding, rather than just fine-tuning or simulation
vs. Standard Contrastive Learning: CLLMR incorporates graph spectrums into the text encoder to prevent the subspace restriction (dimensional collapse) common in standard contrastive alignment

Limitations

Relies on the validity of causal assumptions (no unmeasured confounding) which are hard to verify in real-world recommendation
Computational complexity of counterfactual inference and spectral encoding is not analyzed in the provided text

Reproducibility

The provided text does not contain implementation details, code URLs, or hyperparameter settings.

📚 Prerequisite Knowledge

Prerequisites

Collaborative Filtering
Causal Inference (DAGs, Mediators, Counterfactuals)
Graph Neural Networks (LightGCN context)
Contrastive Learning

Key Terms

Propensity Bias: The tendency of LLMs to generate content favoring specific cultures, viewpoints, or generalized stereotypes regardless of specific input details

Dimensional Collapse: A phenomenon where embedding vectors fall into a low-dimensional subspace, losing their ability to represent diverse or complex data features

SSE: Spectrum-based Side Information Encoder—a proposed module that embeds graph structural information into text representations to prevent dimensional collapse

PNDE: Pure Natural Direct Effect—the causal effect of the exposure on the outcome flowing directly, without passing through the mediator

TNIE: Total Natural Indirect Effect—the causal effect of the exposure on the outcome that is transmitted specifically through the mediator variable

Counterfactual Inference: Estimating what the outcome would have been under a different hypothetical scenario (e.g., if the bias mediator were set to a different level)