Bridging the User-side Knowledge Gap in Knowledge-aware Recommendations with Large Language Models

📝 Paper Summary

Knowledge-aware Recommendation LLM for Recommendation

CIKGRec uses Large Language Models to infer and structure explicit user interests from behavioral history, then integrates this new user-side knowledge graph into recommendations using noise-robust contrastive learning.

Core Problem

Existing Knowledge-aware Recommendation methods only utilize structured knowledge on the item side (e.g., movie genres), ignoring the user side because user features are often scarce, coarse-grained, or unstructured.

Why it matters:

User meta-features (age, gender) are too broad to capture specific interests, leading to over-smoothing in Graph Neural Networks (GNNs).
Item-only knowledge graphs fail to bridge the semantic gap between abstract user preferences and specific item attributes.
Directly using LLM-generated text embeddings introduces noise (hallucinations) and semantic misalignment with collaborative filtering signals.

Concrete Example: A user might watch several sci-fi movies, but traditional metadata only labels them 'Male, 25'. An LLM can infer specific interests like 'Space Exploration' or 'Dystopian Future'. Without this structured user-side knowledge, the recommender misses the specific reason for the user's choices.

Key Novelty

Collaborative Interest Knowledge Graph (CIKG)

Constructs a user-side knowledge graph by prompting LLMs to infer explicit interest tags from interaction history, then clustering these tags to form structured edges between users and interests.
Integrates this user-side graph with the traditional item-side knowledge graph and interaction graph into a unified structure (CIKG).
Uses a curriculum-based masking strategy to reconstruct user interests, filtering out LLM-generated noise while learning robust representations.

Architecture

The overall framework of CIKGRec, split into LLM-based CIKG Construction (Left) and CIKG-based Recommendation (Right).

Evaluation Highlights

Outperforms state-of-the-art baselines (like KGIN and KGCL) on three real-world datasets (Amazon-Book, Last-FM, Yelp2018).
Achieves significant improvements in sparse data scenarios; e.g., for users with <20 interactions, performance drops much less than baselines.
Ablation studies confirm the necessity of the user interest reconstruction module, which effectively mitigates noise from LLM hallucinations.

Breakthrough Assessment

7/10

Novel integration of LLM-derived user knowledge into graph structures. Addresses the specific user-side knowledge gap in KG-based recommendations effectively, though the core machinery (GNNs + Contrastive Learning) is standard.

⚙️ Technical Details

Problem Definition

Setting: Top-K Recommendation on a Collaborative Knowledge Graph

Inputs: User-item interaction graph G_R, Item knowledge graph G_K, User interaction history

Outputs: Top-K ranked list of items for each user

Pipeline Flow

LLM Inference: User History → Explicit Interests
Graph Construction: Interests + Interactions + Item KG → CIKG
Recommendation Model: CIKG → GNN Encoder → Contrastive & Reconstruction Modules → Prediction

System Modules

User Interest Inference (Knowledge Construction)

Generate explicit interest tags from user history

Model or implementation: LLM (Specific model not reported, likely GPT-series or Llama based on context)

Interest Structuration (Knowledge Construction)

Cluster and merge similar interest tags to form stable nodes

Model or implementation: Text Clustering (tf-idf or deep sentence embeddings)

CIKG Construction

Merge User Graph, Item KG, and Interest Graph

Model or implementation: Graph Union Operation

GNN Encoder

Learn node embeddings via message passing

Model or implementation: LightGCN

Interest Reconstruction (Training / Auxiliary Task)

Reconstruct masked interest nodes to denoise LLM outputs

Model or implementation: GMAE (Graph Masked Autoencoder) with Curriculum Masking

Cross-domain Contrastive Learning (Training / Auxiliary Task)

Align auxiliary knowledge with collaborative signals

Model or implementation: InfoNCE Loss with Graph Augmentation

Novel Architectural Elements

Curriculum-based Graph Masked Autoencoder (GMAE) for denoising user interests: dynamically scales mask rate from 'easy' to 'hard' during training.
Cross-domain contrastive augmentation strategy: constructs views by explicitly adding user-side vs. item-side knowledge rather than random structural perturbations.

Modeling

Base Model: LightGCN (as the graph encoder backbone)

Training Method: Multi-task learning (Recommendation + Reconstruction + Contrastive)

Objective Functions:

Purpose: Optimize recommendation accuracy (BPR/Ranking).

Formally: L_r using pairwise ranking loss on (u, i, i').
Purpose: Reconstruct masked user interests to filter noise.

Formally: L_mae = || Z - Z''' ||^2 (MSE between original and reconstructed embeddings).
Purpose: Align user/item knowledge domains.

Formally: L_c using InfoNCE loss to maximize agreement between augmented views.
Purpose: Enhance multi-relational semantic representation.

Formally: Translation-based KG embedding loss (TransR style).

Adaptation: None

Trainable Parameters: Node embeddings, GNN weights

Training Data:

Datasets: Amazon-Book, Last-FM, Yelp2018
Splits: 80% training, 10% validation, 10% testing

Key Hyperparameters:

learning_rate: Not explicitly reported in the paper
batch_size: Not explicitly reported in the paper
embedding_dimension: Not explicitly reported in the paper
+ 3 more
mask_rate_initial: alpha (hyperparameter)
mask_rate_max: omega (hyperparameter)
warm_up_epochs: Lambda (hyperparameter)

Compute: Not reported in the paper

Comparison to Prior Work

vs. KGAT/KGIN: CIKGRec introduces structured knowledge on the *user* side, whereas KGAT/KGIN only use item-side KGs.
vs. KGCL: CIKGRec uses a reconstruction module to handle noise specifically from LLM-generated interests, whereas KGCL focuses on general graph noise.
vs. KP4SR [not cited in paper]: KP4SR uses knowledge prompts for sequential recommendation; CIKGRec focuses on general top-K recommendation with explicit graph construction.

Limitations

Depends on the quality of the LLM used for inference; poor LLM outputs could degrade the graph quality despite reconstruction modules.
Graph construction (LLM inference + clustering) is an offline preprocessing step that may be computationally expensive for very large user bases.
No specific analysis of inference latency or training time overhead compared to baselines.
The specific LLM and prompt engineering details are not fully detailed in the provided text.

Reproducibility

Code: https://github.com/laowangzi/CIKGRec

Code is publicly available at https://github.com/laowangzi/CIKGRec. The paper does not explicitly list the specific LLM used for interest inference (e.g., GPT-3.5 or Llama-2) or exact hyperparameters (LR, batch size) in the main text provided.

📊 Experiments & Results

Evaluation Setup

Top-K Recommendation (K=20) on implicit feedback datasets.

Benchmarks:

Amazon-Book (Product Recommendation)
Last-FM (Music Recommendation)
Yelp2018 (POI Recommendation)

Metrics:

Recall@20
NDCG@20
Statistical methodology: Not explicitly reported in the paper

Key Results

Benchmark	Metric	Baseline	This Paper	Δ
Comparative results showing CIKGRec generally outperforming baselines across datasets.
Amazon-Book	Recall@20	0.1691	0.1795	+0.0104

Main Takeaways

The proposed method consistently outperforms KG-based baselines (like KGAT, KGIN) and Contrastive Learning baselines (like KGCL), particularly in Recall and NDCG.
Ablation studies show that removing the user-side knowledge graph (CIKG) causes the most significant performance drop, confirming the value of explicit user interest modeling.
The curriculum-based masking strategy for interest reconstruction is effective; dynamic masking outperforms fixed masking rates.
Performance gains are most pronounced for 'cold-start' users with sparse interactions, validating that the inferred interests bridge the data gap.

📚 Prerequisite Knowledge

Prerequisites

Graph Neural Networks (GNNs) for recommendation (e.g., LightGCN)
Knowledge Graphs (KG) and Collaborative Knowledge Graphs (CKG)
Contrastive Learning (InfoNCE loss)
Large Language Models (LLMs) for inference

Key Terms

_comment: REQUIRED: Define ALL technical terms, acronyms, and method names used ANYWHERE in the entire summary. After drafting the summary, perform a MANDATORY POST-DRAFT SCAN: check every section individually (Core.one_sentence_thesis, evaluation_highlights, core_problem, Technical_details, Experiments.key_results notes, Figures descriptions and key_insights). HIGH-VISIBILITY RULE: Terms appearing in one_sentence_thesis, evaluation_highlights, or figure key_insights MUST be defined—these are the first things readers see. COMMONLY MISSED: PPO, DPO, MARL, dense retrieval, silver labels, cosine schedule, clipped surrogate objective, Top-k, greedy decoding, beam search, logit, ViT, CLIP, Pareto improvement, BLEU, ROUGE, perplexity, attention heads, parameter sharing, warm start, convex combination, sawtooth profile, length-normalized attention ratio, NTP. If in doubt, define it.

CIKG: Collaborative Interest Knowledge Graph—a unified graph structure proposed in this paper that connects users, items, entities (item attributes), and inferred user interests.

GMAE: Graph Masked Autoencoder—a technique where parts of a graph are masked and the model must reconstruct them, forcing it to learn robust features.

Curriculum Learning: A training strategy where the difficulty of the task increases over time (e.g., increasing the mask rate) to help the model learn gradually.

InfoNCE: Information Noise Contrastive Estimation—a loss function used in contrastive learning to maximize similarity between positive pairs and minimize it for negative pairs.

LightGCN: A simplified Graph Convolutional Network designed for recommendation that removes non-linearities and feature transformations to focus on neighborhood aggregation.

SimGCL: Simple Graph Contrastive Learning—a method that augments graph views by adding random noise to node embeddings rather than perturbing the graph structure.

Hallucination: A phenomenon where LLMs generate plausible but incorrect or nonsensical information.

Over-smoothing: A problem in GNNs where node representations become indistinguishable after too many layers of aggregation.