LoRec: Large Language Model for Robust Sequential Recommendation against Poisoning Attacks

📝 Paper Summary

Robust Sequential Recommendation Defense against Poisoning Attacks

LoRec improves recommender system robustness by using Large Language Models to identify and down-weight potential fraudsters based on natural language profiles and interaction patterns.

Core Problem

Sequential recommender systems are vulnerable to poisoning attacks where injected fraudsters manipulate item transition patterns, and existing defenses (adversarial training or rule-based detection) fail against unknown or optimization-based attacks.

Why it matters:

Attacks can drastically skew item exposure, damaging user experience and hindering long-term system development
Existing defenses rely on specific priors (hard-coded rules) or supervised data from known attacks, lacking generalization to new attack types like Deep Poisoning (DP)
Adversarial training's 'min-max' paradigm often assumes attackers want to degrade overall performance, failing against attackers who aim to *promote* specific items

Concrete Example: Adversarial training methods effectively defend against simple 'Bandwagon' attacks but fail against 'DP' (Deep Poisoning) attacks because DP attacks rely on optimization-based fraudster injection that deviates from the heuristic patterns adversarial training expects.

Key Novelty

LLM-Enhanced Calibration (LoRec)

Integrates a frozen LLM to assess the 'fraudulent potential' of a user's textual profile (open-world knowledge) combined with the recommender's internal embeddings (specific knowledge)
Uses a specialized CalibraTor (LCT) module to dynamically re-weight users during training, down-weighting probable fraudsters without requiring ground-truth fraudster labels for the entire dataset

Architecture

The LoRec framework structure, showing how the Sequential Recommender and the LLM-enhanced CalibraTor (LCT) interact.

Evaluation Highlights

Demonstrates qualitative superiority over adversarial training against optimization-based attacks (DP) where traditional methods fail (based on Introduction claims; specific numbers not in provided text)
Generalizes to unknown attacks by leveraging LLM open-world knowledge rather than relying solely on pre-defined heuristic rules

Breakthrough Assessment

7/10

Novel application of LLMs for *defense/calibration* in RecSys rather than recommendation itself. Addresses a critical gap (unknown attacks), though the provided text lacks the quantitative evidence to confirm the magnitude of improvement.

⚙️ Technical Details

Problem Definition

Setting: Sequential Recommendation under Poisoning Attacks

Inputs: User set U, Item set V, historical interaction sequences s_u = [v_1, ..., v_t]

Outputs: Probability P(v|s_u) of the next item v

Pipeline Flow

Group: Recommendation Backbone
Input Processing: Users' historical sequences converted to embeddings
Sequential Model: Generates prediction embeddings for next items
Group: Robustness Calibration (LCT)
Feature Extraction (Specific): Concatenates item embeddings with model prediction embeddings
Feature Extraction (General): LLM converts user history text to 'fraud potential' embedding
Fusion & Prediction: LCT fuses embeddings to predict user fraud score (p_u)
Calibration: Loss function weighted by p_u to down-weight potential fraudsters

System Modules

Sequential Recommender (Backbone)

Standard sequential recommendation (e.g., predicting next item)

Model or implementation: Generic Sequential Model (e.g., SASRec, Bert4Rec)

Dimension Transformation (DT) Block (Robustness Calibration (LCT))

Projects LLM outputs to the recommender's embedding space

Model or implementation: Learnable projection layer

LCT Encoder (Self-Attention) (Robustness Calibration (LCT))

Aggregates item embeddings and prediction errors to detect anomalies

Model or implementation: Self-attention mechanism with learnable token k_0

Projection Layer (Robustness Calibration (LCT))

Maps the fused representation to a scalar fraud probability

Model or implementation: Two-layer perceptron

Novel Architectural Elements

Dual-knowledge fusion: Combining 'Specific Knowledge' (model feedback/gradients) with 'Open-World Knowledge' (LLM assessment) inside a dedicated calibration module
Iterative Weight Compensation mechanism: An adaptive loop that updates user weights over epochs based on statistical distribution of logits (skewness)

Modeling

Base Model: Varies (Framework applies to Sequential Recommenders like SASRec, Bert4Rec)

Training Method: Joint training of Recommender and LCT with Adaptive Weighting

Objective Functions:

Purpose: Train the recommender while discounting fraudsters.

Formally: Weighted cross-entropy loss where weight depends on 1 - p_u (fraud probability)
Purpose: Train the LCT to identify fraudsters using limited supervision.

Formally: BCE loss on known fraudsters + Entropy Regularization on unlabeled users
Purpose: Integrate LLM knowledge into the LCT.

Formally: Maximize cosine similarity between LCT representation r_u and LLM representation l_u

Training Data:

Uses a small set of known fraudsters (U_atk) for supervision
Treats the general user set U as unlabeled (mixture of genuine and unknown fraudsters)

Compute: Not reported in the paper

Reproducibility

Code URL not provided in the text. Relies on an external LLM for feature extraction (prompts described: 'Please assess the likelihood of this user being a fraudster').

📊 Experiments & Results

Evaluation Setup

Defending against poisoning attacks in sequential recommendation

Benchmarks:

Not specified in text (Sequential Recommendation)

Metrics:

Not reported in the provided text
Statistical methodology: Not explicitly reported in the paper

Experiment Figures

Conceptual comparison of defense methods against different attacks (Bandwagon vs. DP).

Distribution of 'fraudster probability' logits for genuine users vs. fraudsters.

Main Takeaways

Numeric results are not available in the provided text snippet (Sections 1-4 only).
Qualitatively, the paper claims LoRec outperforms adversarial training (APR) and detection methods (GraphRfi, RAdabst) against unknown attacks like Deep Poisoning (DP).
The method relies on the observation that legitimate users usually have a right-skewed distribution of fraud scores (low probability), while fraudsters follow a different distribution, allowing for an adaptive threshold.

📚 Prerequisite Knowledge

Prerequisites

Sequential Recommender Systems (SASRec, Bert4Rec)
Poisoning Attacks (Bandwagon, DP)
Adversarial Training

Key Terms

Poisoning Attacks: Malicious injection of fake user profiles/interactions into training data to manipulate the recommender's learned patterns, often to promote target items

LCT: LLM-Enhanced CalibraTor—the core module proposed in this paper that predicts fraud probability and calibrates user weights

Bandwagon Attack: A heuristic-based attack where fraudsters interact with popular items to blend in, then interact with the target item to boost its recommendation probability

DP Attack: Deep Poisoning—an optimization-based attack that generates fraudster behaviors by maximizing a specific loss function to manipulate the model

Sequential Recommendation: Recommender systems that model user interests as dynamic sequences to predict the next interaction

Adversarial Training: A defense method using a min-max game to train models robust to small perturbations, often assuming attackers want to maximize error

Open-world knowledge: General knowledge encapsulated in LLMs (e.g., about typical user behavior or spam patterns) that is not present in the specific recommendation dataset