Hallucination Detection in Large Language Models with Metamorphic Relations

📝 Paper Summary

Hallucination suppression Factuality evaluation

MetaQA detects LLM hallucinations without external databases by generating synonymous and antonymous mutations of a response and checking if the model's verification of these mutations is logically consistent.

Core Problem

Existing hallucination detection methods rely on unavailable external databases, privacy-invasive search engines, or inaccessible token probabilities (for closed models), while self-contained methods like SelfCheckGPT suffer because LLMs tend to repeat hallucinations when simply prompted multiple times.

Why it matters:

Fact-conflicting hallucinations in high-stakes domains (e.g., legal, medical) can mislead users and erode trust in LLM applications
Reliance on external resources limits detection to domains where comprehensive databases exist
Output probability-based methods (e.g., token entropy) are impossible to use with black-box commercial models like GPT-4

Concrete Example: When asked a legal question about 'Section 306 of the FD&C Act', ChatGPT hallucinates a response. SelfCheckGPT repeatedly asks the same question, receiving consistent incorrect answers (0.1 hallucination score). MetaQA mutates the response into antonyms (e.g., 'Does Section 306 NOT prohibit...'), forcing the model to reveal inconsistencies (1.0 score).

Key Novelty

Metamorphic Relation-based Hallucination Detection (MetaQA)

Applies software testing principles (Metamorphic Relations) to natural language: if a statement is true, its synonym should be true and its antonym should be false according to the same model
Uses the LLM itself to generate these mutations (synonyms/antonyms) and then verify them, acting as its own test oracle without needing external search engines or databases
Calculates a hallucination score based on the logical consistency between the base response and its mutations

Evaluation Highlights

MetaQA outperforms SelfCheckGPT on Mistral-7B with a +112.2% improvement in F1-score (0.435 vs 0.205)
Superiority margin over SelfCheckGPT ranges from 0.154 to 0.368 in F1-score across four LLMs (GPT-4, GPT-3.5, Llama3, Mistral)
Updates the TruthfulQA benchmark to create 'TruthfulQA-Enhanced' by correcting 238 questions, supporting more accurate evaluation

Breakthrough Assessment

7/10

Novel application of metamorphic testing to hallucination detection that outperforms the standard zero-resource baseline (SelfCheckGPT). While the method is clever and resource-efficient, it relies heavily on the model's ability to verify its own logic.

⚙️ Technical Details

Problem Definition

Setting: Zero-resource hallucination detection for Question Answering

Inputs: A question Q and an LLM-generated base response B

Outputs: A hallucination score S_QB representing the likelihood that B contains fact-conflicting hallucinations

Pipeline Flow

Concise Question-Answering (Step 1)
Mutation Generation (Step 2)
Mutation Verification (Step 3)
Hallucination Evaluation (Step 4)

System Modules

Concise Question-Answerer

Generate a brief, contextually grounded base response to the user query to avoid excessive verbosity

Model or implementation: Target LLM (e.g., GPT-4, Llama3)

Mutation Generator

Create multiple follow-up questions based on B using Metamorphic Relations (synonyms and antonyms)

Model or implementation: Target LLM

Mutation Verifier

Ask the LLM to verify the truthfulness of each generated mutation

Model or implementation: Target LLM

Scorer

Calculate hallucination score based on verification results

Model or implementation: Deterministic Algorithm

Novel Architectural Elements

Self-contained loop where the LLM acts as both the generator of mutations and the verifier of those mutations to detect its own hallucinations
Integration of antonymous relation checking in a zero-resource pipeline to force divergence in consistently hallucinated facts

Modeling

Base Model: Evaluated on GPT-4, GPT-3.5, Llama3, and Mistral (method is model-agnostic)

Training Method: Inference-only prompting strategy

Key Hyperparameters:

temperature: 0.0 (found to perform best in ablation)
threshold_theta: Used to classify hallucination based on score (specific value not detailed in summary text)

Compute: Not reported in the paper

Comparison to Prior Work

vs. SelfCheckGPT: Uses structured logical mutations (MRs) instead of stochastic sampling; detects hallucinations even when the model is consistent in its errors
vs. External Database methods: Does not require any external knowledge source or search engine
vs. Token-probability methods: Works on black-box models where log-probs are unavailable

Limitations

Relies on the LLM's own reasoning capability; if the model is fundamentally incapable of reasoning about the mutations, detection may fail
Requires multiple LLM calls (generation, mutation, verification), increasing latency and cost compared to single-pass methods
Performance depends on the quality of the generated mutations

📊 Experiments & Results

Evaluation Setup

Zero-resource hallucination detection on QA tasks

Benchmarks:

TruthfulQA-Enhanced (Question Answering) [New]
HotpotQA (Multi-hop Question Answering)
FreshQA (Question Answering (changing world knowledge))

Metrics:

Precision
Recall
F1 score
Statistical methodology: Not explicitly reported in the paper

Key Results

Benchmark	Metric	Baseline	This Paper	Δ
MetaQA consistently outperforms the baseline SelfCheckGPT across all evaluated LLMs in terms of F1 score, with significant margins.
Average across datasets	F1 score	0.205	0.435	+0.230
Average across datasets	F1 score	Not reported as single aggregate	Not reported as single aggregate	+0.154 to +0.368
Average across datasets	Precision	Not reported as single aggregate	Not reported as single aggregate	+0.041 to +0.113
Average across datasets	Recall	Not reported as single aggregate	Not reported as single aggregate	+0.143 to +0.430

Main Takeaways

MetaQA provides a robust zero-resource alternative to SelfCheckGPT, particularly effective when models are 'confidently wrong' (consistent hallucinations).
The method works across both open-source (Mistral, Llama3) and closed-source (GPT-4, GPT-3.5) models.
Ablation studies indicate that lower temperatures (e.g., 0.0) yield better performance for this verification task.
Current LLMs still exhibit high hallucination rates (17-55%) on QA benchmarks, with GPT-4 being significantly more reliable than GPT-3.5, Llama3, and Mistral.

📚 Prerequisite Knowledge

Prerequisites

Understanding of Large Language Models (LLMs) and hallucination phenomena
Basic knowledge of software testing concepts, specifically Metamorphic Testing
Familiarity with zero-shot prompting techniques

Key Terms

Metamorphic Relations (MR): Properties specifying how the output of a system should change (or not change) when the input is modified in a specific way; used here to check logical consistency

SelfCheckGPT: A baseline method that detects hallucinations by sampling multiple responses from an LLM and checking for consistency; often fails if the model consistently hallucinates the same error

Hallucination Score: A quantified metric (0 to 1) indicating the probability that a response is factually incorrect based on the violation of metamorphic relations

Synonymous Mutation: A generated variation of a sentence that preserves its original meaning (e.g., lexical substitution)

Antonymous Mutation: A generated variation of a sentence that conveys the opposite meaning

Test Oracle: A mechanism for determining whether a system has behaved correctly for a given test execution

Zero-resource: Methods that do not require external databases, search engines, or training data

TruthfulQA-Enhanced: An improved version of the TruthfulQA benchmark with updated correct answers created by the authors