HADSF: Aspect Aware Semantic Control for Explainable Recommendation

📝 Paper Summary

Review-based Recommendation LLM-enhanced Recommendation Explainable Recommendation

HADSF improves recommendation accuracy and interpretability by using a two-stage LLM framework to extract controlled, non-redundant aspect-opinion triples from reviews while managing hallucination via novel fidelity metrics.

Core Problem

Current LLM-based recommenders extract free-form reviews without scope control, leading to redundant, noisy representations and uncontrolled hallucinations that degrade downstream performance.

Why it matters:

Uncontrolled extraction creates fragmented, semantically overlapping tags that inflate redundancy and introduce statistical noise.
Existing systems lack metrics to link LLM hallucination severity directly to downstream recommendation accuracy, preventing principled model selection.
The cost-quality trade-off of using smaller vs. larger LLMs for extraction remains unexplored, hindering scalable deployment.

Concrete Example: Traditional methods might extract fragmented pairs like 'screen-good' and 'display-nice' for the same issue. Uncontrolled LLMs might decompose reviews into overly granular, low-frequency tags (as shown in Fig.1b), or hallucinate attributes not present in the text, inflating noise and redundancy.

Key Novelty

Hyper-Adaptive Dual-Stage Semantic Framework (HADSF)

**Stage I (Consensus):** Creates a compact, corpus-level aspect vocabulary by sampling reviews, generating abstracts, and merging semantically similar aspects via embedding-based clustering.
**Stage II (Constraint):** Uses this vocabulary and user/item history as explicit constraints to guide an LLM in extracting structured aspect-opinion-sentiment triples, reducing noise and drift.
**Hallucination Metrics:** Introduces Aspect Drift Rate (ADR) and Opinion Fidelity Rate (OFR) to quantify fabrication and correlates them with recommendation error.

Architecture

The overall HADSF framework consisting of Stage I (Vocabulary Induction) and Stage II (Dynamic Aspect-Aware Review Processing).

Evaluation Highlights

Demonstrates consistent reductions in rating prediction error across datasets spanning ~3 million reviews.
Uncovers a non-monotonic relationship between hallucination severity (ADR/OFR) and prediction error, aiding model selection.
Shows that smaller LLMs (e.g., 1.5B parameters) using HADSF's controlled extraction can match or exceed the performance of larger models in specific regimes.

Breakthrough Assessment

7/10

Strong contribution in formalizing 'control' for LLM extraction in RecSys and linking hallucination metrics to downstream utility. The non-monotonic finding is particularly insightful for practical deployment.

⚙️ Technical Details

Problem Definition

Setting: Review-based rating prediction using structured aspect-opinion information extracted from raw reviews

Inputs: User-item interaction history including review texts and timestamps

Outputs: Predicted rating for a target user-item pair

Pipeline Flow

Stage I: Vocabulary Induction (Sampling → Abstraction → Extraction → Clustering)
Stage II: Personalized Extraction (Historical Context Construction → Prompting → Triple Extraction)
Evaluation: Hallucination Quantification (ADR/OFR calculation)

System Modules

Vocabulary Inducer

Create a compact, non-redundant aspect vocabulary from the corpus

Model or implementation: LLM for abstraction/extraction + Embedding Model for clustering

History Builder (Stage II: Dynamic Aspect-Aware Review Processing)

Maintain dynamic user/item preference history

Model or implementation: Database/State tracker

Triple Extractor (Stage II: Dynamic Aspect-Aware Review Processing)

Extract structured (aspect, opinion, sentiment) triples using constraints

Model or implementation: LLM (1.5B to 70B parameters tested)

Novel Architectural Elements

Multi-sampling consensus mechanism for vocabulary induction to reduce noise from single-pass extraction
Dynamic history injection where the extraction prompt is conditioned on evolving user/item aspect priors

Modeling

Base Model: Evaluated across scales: 1.5B to 70B parameters (specific model names not listed in snippet, but ranges are)

Training Method: In-context learning / Prompting (Framework does not explicitly fine-tune the LLM for the extraction task in the description provided, but uses it as a frozen extractor)

Compute: Experiments conducted on models ranging from 1.5B to 70B parameters.

Comparison to Prior Work

vs. DeepCoNN/NARRE: HADSF uses LLMs for semantic extraction rather than CNNs/Attention over raw text, ensuring better semantic alignment.
vs. P5/LLaRA: HADSF focuses on 'controlled' extraction to reduce noise/redundancy and explicitly measures hallucination impact, whereas others often use raw generation or IDs.
vs. Standard LLM Extraction: HADSF introduces a global vocabulary constraint and dynamic history to prevent the generation of fragmented or hallucinated tags.

Limitations

Reliance on LLM inference costs for the extraction phase, although smaller models are shown to be viable.
The framework focuses on the extraction and representation quality; the specific rating predictor architecture used downstream is a standard choice.
No explicit mechanism to correct hallucinations in real-time, only to measure and mitigate via vocabulary constraints.

Reproducibility

Code: https://github.com/niez233/HADSF

Code, data pipelines, and metric implementations are publicly available at https://github.com/niez233/HADSF.

📊 Experiments & Results

Evaluation Setup

Review-based rating prediction and hallucination quantification

Benchmarks:

Custom Dataset (Rating Prediction / Aspect Extraction) [New]

Metrics:

Rating Prediction Error (MSE/RMSE implies)
Aspect Drift Rate (ADR)
Opinion Fidelity Rate (OFR)
Statistical methodology: Not explicitly reported in the paper

Experiment Figures

Comparison of extraction quality between Traditional Neural Methods, Mainstream LLMs, and HADSF.

Semantic similarity heatmaps before and after clustering-based normalization.

Main Takeaways

HADSF yields consistent reductions in prediction error compared to traditional neural and mainstream LLM baselines by reducing noise and redundancy.
A non-monotonic relationship exists between hallucination severity and prediction error; zero hallucination isn't always optimal, but unchecked hallucination hurts performance.
Smaller LLMs (e.g., 1.5B) using HADSF's controlled extraction can achieve competitive performance against larger models, validating the cost-effectiveness of the approach.
Global aspect constraints combined with personalized history provide the most coherent and accurate extraction results.

📚 Prerequisite Knowledge

Prerequisites

Basics of Recommender Systems (collaborative filtering, review-based methods)
Large Language Models (prompting, embeddings)
Clustering algorithms (agglomerative clustering)
Metric learning (Cosine similarity)

Key Terms

Aspect-Opinion Triple: A structured representation (aspect, opinion, sentiment) extracted from a review (e.g., 'Screen', 'Bright', 'Positive')

Hallucination: In this context, when an LLM generates aspects not in the controlled vocabulary or opinions not supported by the source text

ADR: Aspect Drift Rate—the proportion of extracted aspects that do not match the pre-defined controlled vocabulary

OFR: Opinion Fidelity Rate—the semantic consistency between the extracted opinion and the best-matching span in the original review

Agglomerative Clustering: A bottom-up clustering method used here to group semantically similar aspect terms (e.g., 'ambiance' and 'atmosphere') into a single representative concept

SemSim: A metric calculating the semantic similarity (cosine similarity of embeddings) between an extracted opinion and spans within the source review