DiffKG: Knowledge Graph Diffusion Model for Recommendation

📝 Paper Summary

Knowledge-aware Recommendation Graph Neural Networks (GNNs) Generative Diffusion Models

DiffKG improves recommendation accuracy by using a generative diffusion model to filter noisy connections in knowledge graphs and reconstruct task-relevant subgraphs guided by user interaction signals.

Core Problem

Real-world Knowledge Graphs (KGs) used in recommendation systems often contain noisy, irrelevant connections and long-tail distributions that mislead user preference modeling.

Why it matters:

Standard KG-enhanced methods assume all KG relations are useful, but irrelevant connections introduce noise that degrades recommendation accuracy.
Existing contrastive learning approaches rely on random augmentation, which can inadvertently break informative structures or retain noise.

Concrete Example: A movie recommendation system might link a user's favorite action movie to a drama via a shared but irrelevant 'director' entity. Standard methods treat this as a strong signal, recommending unwanted dramas. DiffKG would identify this 'director' link as noise for this specific user's preference and filter it out.

Key Novelty

Generative KG Diffusion for Structure Denoising

Applies a diffusion process to the Knowledge Graph structure itself, gradually adding noise to relations and learning to reverse it to recover a clean, robust graph structure.
Introduces a Collaborative KG Convolution mechanism that injects user-item interaction signals into the diffusion process, ensuring the reconstructed KG focuses on relations relevant to recommendation rather than just generic facts.

Architecture

The overall architecture of DiffKG.

Evaluation Highlights

Outperforms state-of-the-art baselines (including KGCL and KGIC) across three benchmark datasets (Alibaba-iFashion, Amazon-Book, Yelp2018).
Achieves significant performance gains in sparse data scenarios, demonstrating robustness against data scarcity.
Ablation studies confirm the necessity of both the KG diffusion module and the collaborative convolution mechanism for optimal performance.

Breakthrough Assessment

7/10

Novel application of diffusion models specifically for graph structure denoising in recommendation. While diffusion in RecSys is growing, integrating it with collaborative signals for KG filtering is a strong, distinct contribution.

⚙️ Technical Details

Problem Definition

Setting: Top-N Recommendation with Knowledge Graph auxiliary data

Inputs: User-item interaction graph G_u and Knowledge Graph G_k

Outputs: Predicted probability of user u interacting with item i

Pipeline Flow

KG Diffusion (Generates Denoised KG View)
Heterogeneous Knowledge Aggregation (Encodes Item Embeddings)
Local Graph Embedding Propagation (Refines User/Item Embeddings via CF)
Prediction & Loss Calculation

System Modules

KG Diffusion Model

Generates a task-relevant subgraph G_k' by removing noise from the original KG.

Model or implementation: Denoising Diffusion Probabilistic Model (DDPM) adapted for discrete graph structures

Collaborative KG Convolution (CKGC)

Injects collaborative signals (user-item interactions) into the diffusion process to ensure relevance.

Model or implementation: Custom convolution layer

Relation-Aware Embedding Layer (Encoder)

Aggregates neighbor information in the KG using attention to handle relation heterogeneity.

Model or implementation: Graph Attention Network (GAT) variant

Local Graph Embedding Propagation (Encoder)

Captures high-order collaborative signals on the user-item graph (LightGCN style).

Model or implementation: Simplified GCN (LightGCN)

Novel Architectural Elements

Collaborative Knowledge Graph Convolution (CKGC) module that specifically conditions the diffusion reverse process on user-item interaction data.
Integration of a discrete graph diffusion process directly into the recommendation training loop as a data augmentation view generator.

Modeling

Base Model: LightGCN backbone for collaborative filtering; MLP for diffusion reverse step

Training Method: Joint optimization of Recommendation task and Diffusion task

Objective Functions:

Purpose: Optimize recommendation ranking.

Formally: L_bpr (Bayesian Personalized Ranking loss)
Purpose: Maximize agreement between original and denoised KG views.

Formally: L_cl (Contrastive Loss with InfoNCE)
Purpose: Train diffusion model to reconstruct KG structure.

Formally: L_elbo (Evidence Lower Bound)
Purpose: Guide diffusion with collaborative signals.

Formally: L_ckgc (MSE between collaborative-aware and original item embeddings)

Training Data:

Datasets: Alibaba-iFashion, Amazon-Book, Yelp2018
Split: 80% training, 10% validation, 10% testing (random split)

Key Hyperparameters:

learning_rate: 1e-4
batch_size: 2048
embedding_size: 64
+ 3 more
diffusion_steps_T: 5
noise_schedule_beta: Linear scheduler
contrastive_temperature_tau: 0.2

Compute: Not explicitly reported in the paper

Comparison to Prior Work

vs. KGCL: DiffKG uses a learnable diffusion process to generate high-quality augmentations instead of random stochastic augmentation.
vs. KGIC: DiffKG explicitly filters noise via a generative process guided by collaborative signals, whereas KGIC relies on intuitive cross-view consistency.
vs. KGIN: DiffKG focuses on structural denoising of the input graph before/during embedding, rather than just attention mechanisms during aggregation.

Limitations

Computational complexity of the diffusion process (iterative denoising) may be higher than simple random augmentation.
Requires joint training of multiple objectives (Recommendation, Diffusion, Contrastive), adding hyperparameter sensitivity.
Performance dependency on the quality of the initial collaborative signals to guide the diffusion.

Reproducibility

Code: https://github.com/HKUDS/DiffKG

Code is publicly available at https://github.com/HKUDS/DiffKG. The paper lists datasets and hyperparameters (embedding dimension, batch size, learning rate) but does not specify exact GPU hardware used.

📊 Experiments & Results

Evaluation Setup

Top-N Recommendation

Benchmarks:

Alibaba-iFashion (Fashion recommendation)
Amazon-Book (Book recommendation)
Yelp2018 (Business recommendation)

Metrics:

Recall@20
NDCG@20
Statistical methodology: Not explicitly reported in the paper

Key Results

Benchmark	Metric	Baseline	This Paper	Δ
DiffKG consistently outperforms baselines on all three datasets, showing the effectiveness of the diffusion-based denoising.
Alibaba-iFashion	Recall@20	0.1086	0.1145	+0.0059
Amazon-Book	Recall@20	0.1695	0.1772	+0.0077
Yelp2018	Recall@20	0.0706	0.0748	+0.0042

Main Takeaways

DiffKG consistently outperforms state-of-the-art baselines (KGCL, KGIC, KGIN) across all datasets and metrics.
The model shows robust performance in sparse data scenarios (users with few interactions), indicating successful knowledge transfer from the KG.
Ablation studies confirm that removing the Collaborative KG Convolution (CKGC) leads to performance drops, validating the need for task-specific guidance in diffusion.
The generative diffusion augmentation is superior to random graph augmentation (as used in KGCL) for constructing informative views.

📚 Prerequisite Knowledge

Prerequisites

Collaborative Filtering
Graph Neural Networks (GNNs)
Knowledge Graphs
Denoising Diffusion Probabilistic Models (DDPM)

Key Terms

Knowledge Graph (KG): A structured representation of facts, consisting of entities (nodes) and relations (edges), used to enrich item information.

Diffusion Model: A generative model that learns to generate data by reversing a gradual noise-addition process.

Evidence Lower Bound (ELBO): A variational lower bound on the log-likelihood of data, used as the objective function to train the diffusion model.

Collaborative Signals: Information derived from user-item interactions (e.g., users who bought X also bought Y) used to guide the recommendation process.

Contrastive Learning (CL): A self-supervised learning technique that encourages the model to pull similar representations (positive pairs) closer and push dissimilar ones (negative pairs) apart.

BPR Loss: Bayesian Personalized Ranking loss, a standard objective function for ranking tasks that optimizes the relative order of positive and negative items.

Heterogeneous Knowledge Aggregation: A mechanism to aggregate information from different types of entities and relations in a KG, often using attention weights.