🔍 Retrieval-Augmented Generation (RAG)

What: RAG triggering addresses when and whether to invoke external retrieval in a Retrieval-Augmented Generation pipeline, rather than always retrieving for every query or generation step.

Why: Always-on retrieval inflates costs, increases latency, and can degrade answer quality by introducing noisy or conflicting context when the LLM already possesses sufficient knowledge.

Baseline: The conventional approach retrieves external documents for every query unconditionally, concatenating retrieved passages with the prompt regardless of whether the LLM already knows the answer.

• LLMs are poorly calibrated and often exhibit high confidence even when wrong, making self-reported uncertainty unreliable for triggering decisions
• Binary retrieve-or-not decisions fail to exploit the LLM's ability to explicitly verbalize its internal knowledge as an alternative source
• Token-level confidence signals are reactive rather than proactive, often triggering retrieval only after hallucinations have already propagated
• Lightweight triggering classifiers must generalize across diverse query types and knowledge domains without expensive per-domain tuning

What: Query rewriting encompasses techniques that transform a user's original question or search query into one or more reformulated queries that better capture intent and improve retrieval effectiveness in retrieval-augmented generation (RAG) systems, including multi-query generation, query expansion, decomposition, and feedback-driven optimization.

Why: User queries are often vague, ambiguous, or use vocabulary that does not match relevant documents, causing retrieval failures that cascade into incorrect or hallucinated answers from LLMs.

Baseline: The conventional approach passes the user's original query directly to a retriever (sparse like BM25 or dense like a bi-encoder) without any transformation, relying entirely on surface-level or semantic similarity between the raw query and indexed documents.

• Vocabulary mismatch: user queries use different words than relevant documents, causing retrieval failures even when the information exists in the corpus
• Query ambiguity: complex or multi-faceted questions have multiple valid interpretations, but a single query retrieval typically captures only one perspective
• Balancing diversity and relevance: generating multiple query variants risks introducing noise and retrieving irrelevant documents, while being too conservative misses relevant content
• Feedback integration: incorporating signals from retrieval results or downstream generation to iteratively improve queries without excessive latency or computational cost

What: This topic covers methods for retrieving relevant information from external knowledge sources—including dense vector stores, sparse indices, structured databases, and multimodal corpora—and ranking the results to augment large language model generation.

Why: Retrieval is the critical bottleneck in RAG systems: the quality of retrieved documents directly determines generation accuracy, with studies showing retrieval choice can swing end-to-end performance by 17–34 percentage points. Effective retrieval grounds LLMs in factual evidence, reduces hallucinations, and enables access to dynamic or domain-specific knowledge without retraining.

Baseline: The conventional approach uses a fixed dense retriever (such as DPR or Contriever) to encode queries and documents into vector embeddings, performs approximate nearest-neighbor search, and concatenates the top-k results into the LLM prompt for generation.

• Balancing retrieval precision and recall: retrieving too many documents introduces noise and 'hard negatives' that degrade generation, while retrieving too few risks missing critical evidence
• Adapting retrieval to diverse query types: single-hop factoid questions, multi-hop reasoning chains, multi-aspect queries, and domain-specific jargon all require different retrieval strategies
• Scaling retrieval infrastructure: maintaining sub-second latency while indexing millions to billions of documents, with trade-offs between memory-efficient indices (IVF-PQ) and high-recall indices (HNSW)
• Defending against adversarial corpus poisoning: attackers can inject as few as 10 malicious passages to achieve 98% retrieval success rates, manipulating downstream generation

What: Post-processing in RAG encompasses techniques applied after initial retrieval to improve the quality of context passed to the generator, including re-ranking retrieved documents by relevance or utility, filtering out irrelevant or noisy passages, pruning context to remove redundant information, and dynamically adjusting chunk granularity.

Why: Raw retrieval results frequently contain irrelevant, redundant, or misleading content that degrades generation quality and increases latency. Effective post-processing bridges the gap between what retrievers find and what generators actually need, directly improving answer accuracy while reducing computational costs.

Baseline: The conventional approach concatenates all top-k retrieved passages into the generator's context window without any filtering or re-ordering. This naive strategy treats retrieval similarity as a proxy for generation utility, often flooding the model with noise and causing hallucinations or missed answers.

• Relevance-utility mismatch: documents that are semantically similar to the query may not actually help the generator produce correct answers, and high NDCG scores can even correlate negatively with QA performance
• Balancing compression and information loss: aggressive pruning or compression risks discarding critical evidence, while conservative approaches retain too much noise and increase latency quadratically
• Scalability and latency constraints: sophisticated re-ranking and filtering methods (especially LLM-based) add significant computational overhead, creating a tension between post-processing quality and real-time serving requirements
• Robustness to adversarial and noisy retrieval: poisoned or misleading documents can bypass similarity-based filters, and models must learn when to trust, ignore, or supplement retrieved content

What: Answer Generation in RAG focuses on producing accurate, faithful answers from a language model when the retrieved context is noisy, irrelevant, contradictory, or incomplete. It spans architectures, decoding strategies, training methods, and evaluation frameworks that make the generation step resilient to imperfect retrieval.

Why: Retrieval-augmented generation only helps if the generator can distinguish useful evidence from noise. Without robust answer generation, even perfect retrieval can be undermined by a single misleading passage, making this the critical bottleneck for trustworthy RAG systems.

Baseline: The conventional approach concatenates all top-k retrieved passages into the LLM's prompt and generates an answer via standard autoregressive decoding. This naive pipeline treats all passages equally and offers no mechanism to detect or suppress noisy, irrelevant, or contradictory content.

• Knowledge conflicts: the model must decide whether to trust retrieved context or its own parametric memory when they disagree
• Noise sensitivity: irrelevant or adversarial passages can corrupt the entire generation, especially when semantically similar to the query (hard negatives)
• Evidence aggregation: synthesizing a coherent answer from multiple passages without losing critical details or hallucinating unsupported facts
• Efficiency: processing long multi-document contexts is computationally expensive, creating a tension between comprehensiveness and latency

What: Embedding concatenation covers retrieval-augmented methods that operate at the representation level—concatenating or combining embeddings, key-value caches, or learned mappings—rather than prepending raw retrieved text into the language model's input.

Why: Concatenating raw documents into the input causes quadratic attention costs and injects irrelevant noise; working at the embedding level enables parallel encoding, selective context loading, and more efficient memory use.

Baseline: Standard dense RAG retrieves documents, concatenates their text into one long prompt, and feeds everything through the language model, incurring high latency and noise from irrelevant passages.

• Parallel-encoded document embeddings lose cross-document attention, making relevance scoring harder without full concatenation
• Nearest-neighbor search over massive embedding datastores is computationally expensive, especially at every token step in kNN-LM
• Low-frequency tokens suffer from hubness and quantization errors in embedding space, limiting retrieval accuracy for rare phenomena
• Replacing explicit datastores with learned mappings (e.g., MLPs) risks losing the fine-grained memorization that kNN retrieval provides

What: This topic covers research on modular Retrieval-Augmented Generation pipelines, where distinct stages—retrieval triggering, query rewriting, document retrieval, post-processing, and answer generation—are independently designed and optimized. It encompasses general advances in RAG evaluation, security, knowledge conflict resolution, and serving efficiency that span multiple pipeline stages.

Why: As RAG systems move from research prototypes to production deployments, ensuring their reliability, security, and efficiency becomes critical. Standardized evaluation, robustness to adversarial attacks, and graceful handling of knowledge conflicts are essential for trustworthy real-world RAG applications.

Baseline: A naive RAG pipeline retrieves top-k document chunks via dense or sparse retrieval, concatenates them into the LLM prompt, and generates an answer. This baseline lacks mechanisms to handle knowledge conflicts, detect adversarial inputs, or systematically evaluate output quality beyond surface-level metrics like BLEU or ROUGE.

• Knowledge conflicts between the LLM's parametric memory and retrieved context lead to hallucinations or outdated answers, and models struggle to decide which source to trust
• Evaluating long-form RAG outputs is difficult because standard metrics fail to capture faithfulness, citation accuracy, and factual completeness, while human evaluation is expensive and non-scalable
• RAG systems introduce new security vulnerabilities through their retrieval component, including indirect prompt injection, data exfiltration, and poisoning attacks via malicious documents
• Serving RAG systems efficiently requires balancing conflicting resource demands between CPU-bound retrieval and GPU-bound generation, especially on resource-constrained platforms

What: This topic covers methods that construct knowledge graphs from text corpora and leverage graph structures—including entity-relation triples, community hierarchies, and hypergraphs—to improve retrieval and reasoning in retrieval-augmented generation systems.

Why: Standard vector-based RAG retrieves isolated text chunks, missing structural relationships between entities and failing at multi-hop reasoning, cross-document synthesis, and complex queries that require connecting dispersed facts.

Baseline: The baseline approach is chunk-based vector retrieval, where documents are split into fixed-length segments, embedded into dense vectors, and retrieved via cosine similarity to the query, with retrieved chunks directly concatenated as context for an LLM.

• Multi-hop reasoning requires connecting multiple pieces of evidence across documents, which flat vector retrieval cannot navigate structurally
• Knowledge graph construction from unstructured text is noisy and expensive, often introducing hallucinated entities or relations that propagate errors downstream
• Balancing retrieval precision with coverage: graph traversal can introduce irrelevant noise from loosely connected nodes, while narrow retrieval misses critical context
• Scaling graph-based methods to large corpora while maintaining real-time inference speed, as graph construction and traversal add significant computational overhead

What: Agentic RAG Pipeline research addresses the challenge of dynamically deciding whether, when, and how to retrieve external information during language model generation, moving beyond static retrieve-then-read pipelines to autonomous, iterative retrieval-reasoning loops.

Why: Static single-pass retrieval fails on complex multi-hop questions where information needs evolve during reasoning, and indiscriminate retrieval wastes compute and introduces noise for questions the model can already answer.

Baseline: The conventional approach is retrieve-then-read: given a query, retrieve the top-k documents from a corpus using semantic similarity, concatenate them into the LLM's context, and generate an answer in a single pass without further retrieval.

• Determining when retrieval is necessary versus when the model's internal knowledge suffices, avoiding both unnecessary retrieval and knowledge gaps
• Handling noisy, irrelevant, or adversarial retrieved documents that can mislead the model and degrade answer quality
• Supporting multi-hop reasoning where each retrieval step depends on the results of previous reasoning, requiring dynamic query formulation
• Jointly optimizing retrieval and generation components that are typically trained independently with misaligned objectives

What: This topic covers research papers that do not fit the main RAG taxonomy categories, spanning RAG security and adversarial robustness, knowledge base question answering (KBQA), in-context learning optimization, QA benchmarks and evaluation methodology, and LLM knowledge management.

Why: These cross-cutting concerns are essential for building trustworthy, well-evaluated, and practically deployable RAG and QA systems. Without addressing security, evaluation gaps, and knowledge integration challenges, even state-of-the-art systems remain fragile in real-world deployment.

Baseline: Conventional approaches treat RAG as a straightforward retrieve-then-generate pipeline, rely on single-answer QA benchmarks, use random or similarity-based demonstration selection for in-context learning, and assume external knowledge provided to the LLM is always complete and accurate.

• RAG systems are vulnerable to adversarial attacks that inject misleading passages into the retrieval corpus, and data owners lack tools to detect unauthorized use of their content
• Standard QA benchmarks assume single correct answers and clean evidence, failing to capture real-world ambiguity, noise, and domain-specific complexity
• LLMs struggle to reliably integrate partial or conflicting external knowledge with their internal parametric memory, especially when fine-tuned knowledge is position-dependent
• Translating natural language questions into formal query languages for knowledge bases remains highly sensitive to the choice of formalism and entity linking quality

What: Complex question answering focuses on answering questions that require aggregating multiple pieces of information across different sources, often involving multi-step reasoning, query decomposition, and iterative retrieval to arrive at a final answer.

Why: Real-world questions are rarely answerable from a single document or retrieval step. Users routinely ask questions that require synthesizing information from multiple sources, following chains of reasoning, and resolving ambiguities—capabilities that basic retrieve-then-read systems fundamentally lack.

Baseline: Standard RAG systems retrieve a fixed set of documents using the original query and generate an answer in a single pass, which fails when the question requires connecting facts spread across multiple documents or reasoning over intermediate results.

• Multi-hop reasoning: Questions require chaining facts across multiple documents where later retrieval depends on results from earlier steps.
• Error propagation: Mistakes in early retrieval or reasoning steps compound through subsequent steps, degrading final answer quality.
• Query-document mismatch: The original complex question may not semantically match individual supporting documents, making single-step retrieval insufficient.
• Information noise: Retrieving many documents for complex queries increases the chance of including irrelevant or misleading information that confuses the model.

What: This topic covers empirical studies that evaluate, benchmark, and dissect Retrieval-Augmented Generation systems to expose performance gaps, failure modes, and design trade-offs across retrieval, generation, and end-to-end pipelines.

Why: Without rigorous analysis, practitioners cannot diagnose whether RAG failures stem from retrieval errors, generation hallucinations, or reasoning breakdowns, leading to wasted effort optimizing the wrong component. Standardized evaluation also enables fair comparison across rapidly proliferating RAG architectures.

Baseline: The conventional approach evaluates RAG using end-to-end metrics like Exact Match or F1 on Wikipedia-based QA datasets, treating the pipeline as a black box without isolating component-level failures or testing domain-specific challenges.

• Benchmark contamination: LLMs increasingly memorize test data during pre-training, making it impossible to distinguish genuine retrieval-based reasoning from parametric recall
• Component attribution: End-to-end metrics conflate retrieval quality with generation quality, hiding whether failures originate in the embedder, retriever, reranker, or generator
• Domain transfer: Benchmarks built on general knowledge (Wikipedia) fail to capture the complexity of specialized domains like law, finance, and medicine where RAG is most needed
• Evaluation scalability: Human annotation is expensive and slow, while automated metrics (BLEU, ROUGE) correlate poorly with actual RAG output quality

What: This topic covers papers that introduce new benchmark datasets, evaluation frameworks, and metrics for assessing Retrieval-Augmented Generation (RAG) systems, spanning end-to-end pipeline evaluation, domain-specific testing, and component-level diagnostics.

Why: Without standardized, rigorous benchmarks, it is impossible to compare RAG systems fairly or identify where they fail; these benchmarks drive reproducible progress and reveal blind spots in retrieval, generation, and their interaction.

Baseline: Traditional RAG evaluation relies on simple lexical overlap metrics (BLEU, ROUGE, F1, Exact Match) applied to single-turn factoid QA over Wikipedia, often using static golden-chunk annotations that break when chunking strategies change.

• Existing benchmarks lack diversity in knowledge sources, query types, and domains, leading to evaluations that do not reflect real-world RAG usage
• Evaluating RAG end-to-end is difficult because errors in retrieval and generation compound, requiring metrics that diagnose each pipeline stage independently
• Static benchmarks quickly become stale as LLMs memorize their content, and temporal questions demand continuously updated ground truth
• Long-form, multi-hop, and multi-turn RAG outputs are poorly captured by token-overlap metrics, requiring new evaluation paradigms like LLM-as-judge and nugget-based assessment

What: This topic covers papers that apply Retrieval-Augmented Generation (RAG) techniques to specific domains or tasks — such as healthcare, legal, education, telecommunications, and crisis response — highlighting both strengths and gaps of RAG in real-world settings.

Why: General-purpose RAG systems often fail in specialized domains due to domain-specific terminology, complex reasoning requirements, and the need for verified, traceable answers. Understanding how to adapt RAG to these domains is critical for deploying reliable AI in high-stakes applications.

Baseline: The conventional approach uses a general-purpose LLM with a standard vector-similarity retrieval pipeline over domain documents, typically chunked uniformly and embedded with general-purpose models like Contriever or OpenAI embeddings.

• Domain-specific terminology and jargon cause retrieval failures when general-purpose embeddings cannot distinguish nuanced meanings
• High-stakes domains (medicine, law, disaster response) require traceable, verified answers with minimal hallucination tolerance
• Complex domain reasoning often requires multi-hop connections across structured and unstructured knowledge sources
• Lack of domain-specific benchmarks with verified ground truths makes it difficult to evaluate and improve RAG systems systematically