Exploring User Retrieval Integration towards Large Language Models for Cross-Domain Sequential Recommendation

📝 Paper Summary

Cross-Domain Sequential Recommendation (CDSR) LLM-based Recommendation

URLLM improves cross-domain recommendations by aligning collaborative graph data with semantic text and retrieving similar users to guide a Large Language Model, reducing out-of-domain hallucinations.

Core Problem

Existing methods fail to simultaneously capture collaborative structure and semantic item information, while LLMs often hallucinate items outside the target domain due to a lack of domain constraints.

Why it matters:

Traditional CDSR models suffer from cold-start issues by overlooking valuable semantic text buried in item features
LLMs applied to recommendation struggle to integrate structured collaborative history seamlessly
Uncontrollable LLM generation leads to 2% to 20% of recommendations belonging to the wrong domain, undermining system reliability

Concrete Example: In a movie-to-game recommendation scenario, a standard LLM might suggest a movie sequel instead of a game, or a game that doesn't exist, because it relies on common knowledge rather than specific domain constraints and user interaction history.

Key Novelty

User Retrieval and Domain Grounding on LLM (URLLM)

Dual-Graph modeling that aligns item-attribute graphs (semantic) with item-item sequence graphs (collaborative) to feed structured info into the LLM
A user retrieval paradigm that fetches similar users (via KNN) to serve as in-context demonstrations for the LLM, bridging collaborative filtering with language generation
Domain-specific refinement strategies to force the LLM to generate items strictly within the target domain

Architecture

The URLLM framework architecture, detailing the Dual Graph Sequence-Modeling Model and the User Retrieve-Generation Model.

Evaluation Highlights

Demonstrates effective information integration on Amazon datasets (Movie-Game and Art-Office)
Identifies positive correlation between the hit rate of retrieved users and overall model performance
Mitigates the issue where 2% to 20% of generated content belongs to other domains [baseline failure rate reported in motivation]

Breakthrough Assessment

7/10

Novel integration of retrieval-augmented generation specifically for the cross-domain recommendation cold-start problem, addressing the specific hallucination issues of LLMs in this context.

⚙️ Technical Details

Problem Definition

Setting: Cross-Domain Sequential Recommendation (CDSR) predicting next item in target domain given history in source and target domains

Inputs: User interaction sequence S_u containing items from source domain X and target domain Y

Outputs: Predicted next item i_{k+1} in the combined item set X U Y (specifically targeted)

Pipeline Flow

Graph Construction: Item-Attribute Graph (via LLM) + Item-Item Sequence Graphs
Dual Graph Modeling: GNN Encoder with Alignment & Contrastive Learning
User Retrieval: KNN search for similar users
LLM Generation: Instruct tuning with retrieved context
Refinement: Domain-specific filtering

System Modules

Graph Constructor

Builds structured representations of item relationships

Model or implementation: Rule-based + LLM for attributes

Dual Graph Encoder

Encodes collaborative and semantic info into embeddings

Model or implementation: GNN (Graph Neural Network)

User Retriever

Retrieves relevant user information to prompt the LLM

Model or implementation: KNN (K-Nearest Neighbors)

LLM Generator

Generates the final recommendation

Model or implementation: Large Language Model (Specific backbone not named in text)

Novel Architectural Elements

Dual-graph framework combining item-attribute (semantic) and item-item (collaborative) graphs with alignment loss
Integration of KNN-based user retrieval directly into the LLM instruction tuning pipeline for CDSR

Modeling

Base Model: Large Language Model (Specific backbone not explicit in text)

Training Method: Instruction Tuning (Fine-tuning)

Objective Functions:

Purpose: Guide the LLM output scope and format.

Formally: Maximize likelihood P(a_i,t | x, a_i,<t) where x is input context and a is answer token.
Purpose: Align graph representations.

Formally: Alignment loss (implied by text description of 'alignment and contrastive learning method')

Training Data:

Instruction data constructed from user interaction sequences and queries

Compute: Not reported in the paper

Comparison to Prior Work

vs. BIGRec: URLLM integrates info *during* generation via retrieval and graph encoding rather than post-hoc ensembling
vs. CoLLM: URLLM uses a dual-graph alignment specifically for cross-domain transfer, whereas CoLLM focuses on general collaborative alignment
vs. Standard LLMs: URLLM introduces domain-specific constraints and refinement to prevent the 2-20% out-of-domain generation rate observed in baselines

Limitations

Reliance on constructing graphs based on rules due to absence of explicit graph data in datasets
Complexity of maintaining dual graphs (attribute and sequence) simultaneously
Performance depends on the quality of the 'Chain-of-Thought' generated attributes for the item-attribute graph

Reproducibility

Code: https://github.com/TingJShen/URLLM

Code is publicly available at https://github.com/TingJShen/URLLM. The paper describes graph construction rules and instruction tuning formatting.

📊 Experiments & Results

Evaluation Setup

Cross-domain sequential recommendation on Amazon datasets

Benchmarks:

Amazon Movie-Game (Cross-domain recommendation)
Amazon Art-Office (Cross-domain recommendation)

Metrics:

Not explicitly reported in the paper
Statistical methodology: Not explicitly reported in the paper

Key Results

Benchmark	Metric	Baseline	This Paper	Δ
General LLM Baselines	Out-of-domain Generation Rate	0	20	+20

Main Takeaways

Improvement in URLLM is positively correlated with the types of information (collaborative vs semantic) most crucial to the specific dataset
There is a positive relation between the hit rate of retrieved users (via KNN) and the final performance of the model, validating the user retrieval module
Seamless integration of structural-semantic and collaborative information helps prevent skewed user preferences common in traditional CDSR models

📚 Prerequisite Knowledge

Prerequisites

Sequential Recommendation (SR) fundamentals
Graph Neural Networks (GNNs)
Large Language Models (LLMs) and instruction tuning
Cold-start problem in recommendation

Key Terms

CDSR: Cross-Domain Sequential Recommendation—transferring user preferences from a source domain (e.g., movies) to a target domain (e.g., games) to improve recommendations

Cold-start: The difficulty of recommending items to users with very few recorded interactions

Collaborative information: Data derived from user-item interactions (who bought what), representing behavioral patterns

Semantic information: Content-based data such as item descriptions, titles, and attributes

COT: Chain-of-Thought—a prompting technique where the LLM produces intermediate reasoning steps before the final answer

KNN: K-Nearest Neighbors—an algorithm used here to retrieve existing users with similar history to the target user

Hallucination: When an LLM generates plausible-sounding but incorrect or non-existent information (e.g., recommending a product that doesn't exist)