KG-Infused RAG: Augmenting Corpus-Based RAG with External Knowledge Graphs

📝 Paper Summary

Graph-based RAG pipeline

KG-Infused RAG integrates pre-existing large-scale knowledge graphs into RAG pipelines via spreading activation and preference learning to improve structured knowledge retrieval without expensive ad-hoc graph construction.

Core Problem

Existing RAG methods either rely solely on text corpora (missing structural connections) or require building ad-hoc knowledge graphs from scratch (computationally expensive and limited in scope), failing to leverage existing high-quality large-scale KGs.

Why it matters:

Corpus-based methods often miss multi-hop connections between entities required for complex reasoning tasks
Ad-hoc KG construction (e.g., GraphRAG) is resource-intensive and prone to extraction errors, making it impractical for very large corpora
Current approaches treat KGs and text corpora as separate silos rather than integrating them to mutually enhance retrieval quality

Concrete Example: For a multi-hop question, a standard RAG might retrieve passages about Entity A and Entity B separately but miss the bridge entity connecting them. KG-Infused RAG uses spreading activation on a KG to find the bridge entity, expanding the query to retrieve the full reasoning chain.

Key Novelty

KG-Infused RAG

Simulates 'spreading activation' (a cognitive psychology concept) on pre-existing KGs to iteratively retrieve related entities and facts, starting from query seeds
Uses the retrieved KG subgraph to expand the natural language query, guiding the subsequent corpus retrieval to be more context-aware
Optimizes the knowledge augmentation stage using Direct Preference Optimization (DPO) to better align the integration of structured and textual knowledge

Architecture

The overall framework of KG-Infused RAG showing the three main stages: KG-Guided Spreading Activation, KG-Based Query Expansion, and KG-Augmented Answer Generation.

Evaluation Highlights

Outperforms vanilla RAG by 3.9% to 17.8% across five QA benchmarks (HotpotQA, 2WikiMQA, MuSiQue, Bamboogle, StrategyQA)
Surpasses ad-hoc KG construction baselines (GraphRAG, LightRAG) in both performance and efficiency by leveraging pre-existing KGs
Demonstrates plug-and-play capability by boosting performance when integrated into advanced RAG systems like Self-RAG and DeepNote

Breakthrough Assessment

8/10

Significant for efficiently bridging the gap between structured KGs and unstructured text retrieval without the high cost of graph construction. Strong empirical gains and plug-and-play versatility.

⚙️ Technical Details

Problem Definition

Setting: Question Answering given a query q, a Knowledge Graph G, and a text corpus Dc

Inputs: Natural language query q

Outputs: Generated answer a

Pipeline Flow

Functional Group: KG Retrieval -> KG-Guided Spreading Activation
Functional Group: Input Processing -> KG-Based Query Expansion
Functional Group: Generation -> KG-Augmented Answer Generation

System Modules

KG-Guided Spreading Activation

Retrieves a query-specific subgraph from the KG via iterative expansion

Model or implementation: LLM for triple selection (e.g., LLaMA-3-8B)

KG-Based Query Expansion

Generates an expanded query using the KG summary to improve corpus coverage

Model or implementation: LLM (e.g., LLaMA-3-8B)

KG-Augmented Answer Generation

Synthesizes final answer from retrieved passages and KG facts

Model or implementation: LLM (fine-tuned via DPO)

Novel Architectural Elements

Iterative spreading activation mechanism guided by LLM selection to traverse pre-existing KGs dynamically during inference
Integration of preference learning (DPO) specifically at the knowledge augmentation stage to align KG and text integration

Modeling

Base Model: Evaluated with Qwen2.5 (3B, 7B, 14B), LLaMA3.1 (8B, 70B), GPT-4o-mini

Training Method: Direct Preference Optimization (DPO)

Objective Functions:

Purpose: Optimize the model to prefer responses that better integrate KG and text knowledge.

Formally: DPO loss L_DPO minimizing -log σ (β log (π_θ(y_w|x)/π_ref(y_w|x)) - β log (π_θ(y_l|x)/π_ref(y_l|x)))

Trainable Parameters: Only the LLM used for the Knowledge Augmentation stage is fine-tuned

Training Data:

Constructed preference pairs D_KA by sampling responses from the KG-Guided KA stage

Key Hyperparameters:

max_activation_rounds: 6
retrieved_passages_k: 6 (3 for original query, 3 for expanded)
seed_entities_k: 3

Compute: Not reported in the paper

Comparison to Prior Work

vs. GraphRAG/LightRAG: Uses pre-existing large-scale KGs instead of building them from scratch, avoiding high cost and extraction errors
vs. Self-RAG: Adds explicit structured knowledge retrieval via spreading activation rather than relying solely on parametric/corpus knowledge
vs. HyDE [not cited in paper]: Uses actual KG facts for query expansion rather than hypothetical document generation

Limitations

Dependence on the quality and coverage of the pre-existing Knowledge Graph (Wikidata5M)
Potential noise propagation if the spreading activation retrieves irrelevant entities
Requires an initial entity linking/retrieval step to find seed nodes

Reproducibility

Code: https://github.com/thunlp/KG-Infused-RAG

Code and data available at https://github.com/thunlp/KG-Infused-RAG. Uses open-source Wikidata5M KG. Specific training time and compute resources not explicitly detailed.

📊 Experiments & Results

Evaluation Setup

Multi-hop and Commonsense Question Answering

Benchmarks:

HotpotQA (Multi-hop QA)
2WikiMultihopQA (Multi-hop QA)
MuSiQue (Multi-hop QA)
Bamboogle (Multi-hop QA)
StrategyQA (Commonsense QA)

Metrics:

F1 Score
Exact Match (EM)
Accuracy
Statistical methodology: Not explicitly reported in the paper

Key Results

Benchmark	Metric	Baseline	This Paper	Δ
KG-Infused RAG consistently outperforms Vanilla RAG across multiple benchmarks.
HotpotQA	F1	48.2	56.4	+8.2
2WikiMQA	F1	40.5	58.3	+17.8
MuSiQue	F1	24.1	29.7	+5.6
Comparison against KG-based baselines shows KG-Infused RAG is superior to methods that construct ad-hoc KGs.
HotpotQA	F1	45.1	56.4	+11.3
2WikiMQA	F1	42.8	58.3	+15.5
Integration with advanced RAG systems demonstrates plug-and-play effectiveness.
HotpotQA	F1	51.8	58.9	+7.1

Main Takeaways

KG-Infused RAG consistently improves performance over Vanilla RAG by 3.9% to 17.8% across 5 benchmarks.
Outperforms ad-hoc KG construction methods (GraphRAG, LightRAG) while avoiding the high computational cost of building graphs from scratch.
Demonstrates versatility as a plug-and-play module, improving advanced baselines like Self-RAG and DeepNote when integrated.
Preference learning (DPO) further enhances the model's ability to effectively utilize retrieved KG facts.

📚 Prerequisite Knowledge

Prerequisites

Retrieval-Augmented Generation (RAG) pipelines
Knowledge Graphs (structure and traversal)
Direct Preference Optimization (DPO)

Key Terms

Spreading Activation: A method for searching semantic networks where activation signals propagate from seed nodes to connected neighbors, used here to traverse the KG

DPO: Direct Preference Optimization—a method to align language models to human preferences by optimizing on preference pairs without a separate reward model

Ad-hoc KG: A knowledge graph constructed dynamically from a specific text corpus using LLMs, as opposed to using a pre-existing curated KG like Wikidata

Wikidata5M: A large-scale knowledge graph dataset derived from Wikidata and Wikipedia, used as the external structured knowledge source

Contriever: A dense information retrieval model used to encode queries and documents into vectors for similarity search

GraphRAG: A baseline method that uses LLMs to extract a knowledge graph from the corpus to aid retrieval

Self-RAG: A RAG framework that trains a model to critique its own retrieval and generation using special reflection tokens