SPARK: Adaptive Low-Rank Knowledge Graph Modeling in Hybrid Geometric Spaces for Recommendation

📝 Paper Summary

Knowledge-aware Recommendation Graph Neural Networks (GNNs) Long-tail Recommendation

SPARK improves recommendation for long-tail items by denoising knowledge graphs via tensor decomposition and fusing representations from both Euclidean and Hyperbolic spaces based on item popularity.

Core Problem

Knowledge-enhanced recommender systems struggle with noisy knowledge graphs and fail to adequately represent sparse, long-tail items due to the limitations of Euclidean geometry.

Why it matters:

Real-world knowledge graphs are inherently noisy and have power-law distributions, meaning most entities have few connections, leading to poor learning for the 'long tail'.
Standard Euclidean models cannot effectively capture the hierarchical structures often found in user-item interactions and knowledge graphs.
Existing methods lack mechanisms to adaptively balance collaborative signals (good for popular items) with semantic knowledge (crucial for rare items).

Concrete Example: A rare book with few user interactions might be recommended poorly by standard collaborative filtering. Even with a KG, if the KG has noisy links, the book's embedding is corrupted. Furthermore, a Euclidean model might fail to capture the book's niche sub-genre hierarchy, treating it equidistantly to unrelated popular books.

Key Novelty

Multi-stage Hybrid Geometry Framework

Uses low-rank Tucker tensor decomposition to 'clean' the Knowledge Graph before use, creating robust initial entity embeddings resistant to noise.
Simultaneously learns item representations in both Euclidean space (for global popularity trends) and Hyperbolic space (for hierarchies and long-tail semantics).
Dynamically weights these signals based on item popularity: popular items rely more on Euclidean/collaborative signals, while rare items rely more on Hyperbolic/KG signals.

Architecture

The overall SPARK framework, detailing the flow from KG preprocessing to the dual-space GNNs and final fusion.

Evaluation Highlights

Outperforms state-of-the-art baselines on Amazon-Book by up to +10.8% in Recall@10.
Achieves significant gains on Alibaba-iFashion and Yelp2018 datasets, particularly for long-tail item recommendation.
Ablation studies confirm that removing the Hyperbolic pathway or the Tucker decomposition significantly degrades performance, validating their specific roles.

Breakthrough Assessment

7/10

Strong combination of existing advanced techniques (Tucker decomposition, Hyperbolic GNNs) into a cohesive, logically sound framework that effectively addresses a specific pain point (long-tail KG recommendation). While the components aren't individually new, their integration and the popularity-aware fusion are well-motivated.

⚙️ Technical Details

Problem Definition

Setting: Knowledge-aware Top-N Recommendation

Inputs: User set U, Item set I, Interaction matrix R, Knowledge Graph G_KG

Outputs: Ranked list of items for each user u

Pipeline Flow

KG Preprocessing: Low-rank Tucker decomposition → Denoised entity embeddings
Initialization: SVD on Interaction Matrix → Initial Euclidean embeddings
Dual GNN Learning: Euclidean GNN pathway + Hyperbolic GNN pathway (concurrent)
Fusion & Alignment: Popularity-aware gating + Contrastive alignment → Final scores

System Modules

TuckER KG Encoder (Input Processing)

Denoise KG and generate robust entity embeddings

Model or implementation: Tucker Decomposition layer

SVD Initializer (Input Processing)

Capture global collaborative trends to bootstrap GNNs

Model or implementation: Standard SVD + Exponential weighting

Euclidean GNN (Representation Learning)

Learn representations capturing standard collaborative patterns

Model or implementation: Standard Graph Convolution layers

Hyperbolic GNN (Representation Learning)

Learn representations capturing hierarchical/long-tail structures

Model or implementation: Lorentz Model GNN

Popularity-Aware Fusion Gate

Dynamically weight Euclidean vs. KG/Hyperbolic signals based on item frequency

Model or implementation: Parameterized Sigmoid Gate

Novel Architectural Elements

Coupling of SVD initialization with a dual-stream (Euclidean + Hyperbolic) GNN architecture.
Explicit popularity-based gating mechanism that dictates the mixture of geometric spaces for the final prediction.

Modeling

Base Model: Hybrid GNN (Custom Architecture)

Training Method: End-to-end multi-task learning

Objective Functions:

Purpose: Optimize recommendation ranking.

Formally: BPR Loss (Bayesian Personalized Ranking)
Purpose: Denoise and learn KG structure.

Formally: TuckER ranking loss (separating valid/invalid triples)
Purpose: Align Euclidean and KG-Semantic views.

Formally: InfoNCE Contrastive Loss (maximizing cosine similarity of positive pairs)
Purpose: Prevent overfitting.

Formally: L2 Regularization

Key Hyperparameters:

embedding_dim: 64
batch_size: 1024
learning_rate: 1e-4 to 5e-4
+ 2 more
gnn_layers: 1 to 4
optimizer: Adam

Compute: NVIDIA RTX 4090 GPU

Comparison to Prior Work

vs. KGAT/KGIN: SPARK uses Hyperbolic space to better model hierarchies and a Tucker decomposition for noise, rather than just attention over KG paths.
vs. HGCF: SPARK is hybrid (Euclidean + Hyperbolic) and uses SVD initialization, whereas HGCF is purely hyperbolic.
vs. LightGCN: SPARK integrates KG data and geometric priors, outperforming on sparse items where LightGCN fails.

Limitations

Computational complexity of maintaining dual geometric spaces (Hyperbolic operations are more expensive than Euclidean).
Reliance on pre-computed SVD adds a preprocessing overhead that may not scale instantly for streaming data.
Hyperbolic space benefits are contingent on the dataset possessing underlying hierarchical structure.

Reproducibility

Code: https://github.com/Applied-Machine-Learning-Lab/SPARK

Code is publicly available at https://github.com/Applied-Machine-Learning-Lab/SPARK. The paper specifies hyperparameters (batch size, dim, etc.) and baselines are implemented via RecBole.

📊 Experiments & Results

Evaluation Setup

Top-N Recommendation on implicit feedback datasets

Benchmarks:

Amazon-Book (Product Recommendation)
Alibaba-iFashion (Fashion Recommendation)
Yelp2018 (POI Recommendation)

Metrics:

Recall@10
NDCG@10
Recall@20
NDCG@20

Key Results

Benchmark	Metric	Baseline	This Paper	Δ
SPARK consistently outperforms all baselines across three datasets on Recall@10 and NDCG@10 metrics.
Amazon-Book	NDCG@10	0.0886	0.0965	+0.0079
Yelp2018	Recall@10	0.0768	0.0816	+0.0048
Alibaba-iFashion	Recall@10	0.1255	0.1348	+0.0093

Main Takeaways

SPARK yields the largest relative improvements on the Amazon-Book dataset, likely due to its high sparsity and rich semantic structure.
Ablation studies show that removing the Popularity-Aware Fusion (w/o PopFus) leads to performance drops, confirming the benefit of treating head and tail items differently.
The SVD initialization acts as a strong 'warm start', allowing the GNN components to focus on refining local and semantic structures rather than learning global trends from scratch.

📚 Prerequisite Knowledge

Prerequisites

Graph Neural Networks (GNNs)
Knowledge Graphs (KGs)
Hyperbolic Geometry (Lorentz Model)
Tensor Decomposition (Tucker)
Contrastive Learning

Key Terms

Knowledge Graph (KG): A structured representation of data where entities (nodes) are connected by relations (edges), providing semantic context.

Tucker Decomposition: A method to factorize a tensor into a core tensor and factor matrices, used here to compress and denoise KG triples.

Hyperbolic Geometry: A non-Euclidean geometry with negative curvature, particularly good at embedding hierarchical or tree-like structures with low distortion.

Lorentz Model: A specific mathematical model for representing hyperbolic space that offers numerical stability for neural networks.

SVD (Singular Value Decomposition): A matrix factorization method used here to initialize user/item embeddings, capturing global collaborative patterns.

Long-tail: Items that are rarely interacted with; they suffer from data sparsity and are hard to recommend accurately.

Contrastive Learning: A learning paradigm that encourages the model to pull similar representations (positive pairs) together and push dissimilar ones (negative pairs) apart.