📝 Paper Summary
Hallucination suppression
Inference-time intervention
END is a decoding method that mitigates hallucinations by measuring how sharply the prediction probability of specific tokens grows across hidden layers, prioritizing those that reflect emerging factual knowledge.
Core Problem
LLMs often generate hallucinations even when they possess correct knowledge, and existing mitigation methods like training or external retrieval are computationally expensive.
Why it matters:
- Hallucinations prevent LLM adoption in high-stakes industries requiring accuracy
- Existing layer-contrast methods (like DoLa) assume a single 'best' layer for all tokens, but different factual tokens exhibit different growth trends across layers
- Training-based solutions require high-quality data and compute that may not be accessible in many scenarios
Concrete Example:
When generating the name 'Sun Yat-sen', the probability of the factual token 'Sun' grows sharply in higher layers, while common functional words remain stable. Methods that pick a fixed contrast layer might miss this token-specific spike or falsely amplify non-factual tokens.
Key Novelty
Cross-Layer Entropy Enhanced Decoding (END)
- Instead of contrasting just two layers (like DoLa), END tracks the evolution of token probabilities across multiple upper layers for *each* candidate token individually
- Calculates 'cross-layer entropy' to quantify the sharpness of this probability growth; a sharp trend indicates the token is factual knowledge being actively retrieved
- Adjusts the final output distribution to boost tokens with low cross-layer entropy (high factual confidence) without any extra training
Architecture
The END decoding framework workflow. It illustrates extracting hidden states, forming cross-layer distributions for candidate tokens, calculating entropy, and re-ranking the final output.
Evaluation Highlights
- +12-21% improvement in Truth*Info scores on TruthfulQA open-ended generation compared to baselines like DoLa and greedy decoding
- Achieves highest MC1 (28.9) and MC2 scores on TruthfulQA multiple-choice, surpassing DoLa and Inference-Time Intervention
- Maintains or improves general QA performance (+10.1% accuracy on Natural Questions) while reducing rejection rates in open-ended generation
Breakthrough Assessment
7/10
Strong empirical results on standard benchmarks and a logical extension of prior layer-contrast work. Being training-free makes it highly practical, though it builds heavily on existing insights about layer-wise knowledge activation.
