Collab-REC: An LLM-based Agentic Framework for Balancing Recommendations in Tourism

📝 Paper Summary

Multi-Agent Recommender Systems LLM-based Tourism Recommendation

Collab-Rec employs three specialized LLM agents (Personalization, Popularity, Sustainability) and a deterministic moderator to negotiate travel recommendations that balance user interests with sustainability goals while enforcing catalog grounding.

Core Problem

Monolithic LLM recommenders struggle to balance competing tourism objectives (e.g., personalization vs. sustainability) and frequently hallucinate out-of-catalog destinations.

Why it matters:

Over-concentrating demand on 'must-see' cities exacerbates overtourism and congestion, harming both local residents and the visitor experience
Single-shot LLM prompting lacks control mechanisms to enforce hard constraints (budget, dates) or validate items against a fixed inventory
Tourism is inherently multi-stakeholder; optimizing solely for user relevance often conflicts with destination management goals like decongestion

Concrete Example: A user asks for 'European cities in September, mid-budget, with museums but not overcrowded.' A standard LLM might ignore the 'not overcrowded' constraint and suggest Paris or Rome (popularity dominance) or invent non-existent festivals (hallucination).

Key Novelty

Moderator-Mediated Multi-Stakeholder Negotiation

Decomposes recommendation into three specialist agents: Personalization (user fit), Popularity (feasibility), and Sustainability (decongestion)
Introduces a deterministic, non-LLM moderator that grounds suggestions to a fixed catalog and scores them against explicit success criteria
Uses iterative feedback loops where the moderator rejects invalid/unbalanced candidates and forces agents to refine proposals until consensus or patience exhaustion

Architecture

The Collab-Rec framework workflow involving three agents and a moderator.

Evaluation Highlights

Collab-Rec improves grounded success rate by +25.8% (absolute) over the single-agent baseline (SASI) using Claude-4.5-sonnet
Reduces popularity concentration (Gini coefficient) by ~0.15 compared to single-agent baselines, effectively surfacing 'hidden gem' destinations
Achieves convergence in ~3-4 rounds of negotiation, with early stopping preserving quality while reducing inference costs

Breakthrough Assessment

7/10

Strong practical framework for controlled, grounded LLM recommendation. While the agentic negotiation concept isn't new, the specific application to multi-objective tourism with a deterministic moderator for grounding is well-executed and rigorously evaluated.

⚙️ Technical Details

Problem Definition

Setting: Grounded multi-objective ranking problem where recommendations must map to a fixed catalog C and satisfy user constraints

Inputs: Natural language user query Q and a structured catalog of destinations C

Outputs: A ranked list of feasible destination cities R ⊂ C

Pipeline Flow

Group: Agent Proposal → Personalization, Popularity, Sustainability Agents
Group: Moderation & Grounding → Catalog Grounding, Scoring, Feedback Generation
Group: Iteration → Loop back to Agents until Success/Patience limit

System Modules

Personalization Agent (Agent Proposal)

Generates candidates maximizing alignment with user constraints and interests

Model or implementation: LLM (e.g., Claude-4.5, GPT-OSS-20b, varying by experiment)

Popularity Agent (Agent Proposal)

Generates candidates based on mainstream appeal and feasibility

Model or implementation: LLM (same backbone as other agents)

Sustainability Agent (Agent Proposal)

Generates candidates that mitigate over-tourism and promote lesser-visited locales

Model or implementation: LLM (same backbone as other agents)

Moderator

Validates candidates against catalog, scores them on objectives, and provides structured feedback

Model or implementation: Deterministic (Non-LLM logic)

Novel Architectural Elements

Deterministic non-LLM moderator embedded in the loop to enforce catalog grounding and transparently score multi-objective trade-offs
Hybrid architecture combining LLM-based generative agents with rule-based verification for reliability in open-ended domains

Modeling

Base Model: Evaluated across 6 families: Claude-4.5-sonnet, Gemini-2.5-flash, GPT-OSS-20b, Gemma-12b, Olmo-7b, Gemma-4b

Compute: Inference only. Experiments run on 900 queries. Multi-round adds latency; Early stopping at 3-4 rounds reduces cost.

Comparison to Prior Work

vs. SASI: Collab-Rec uses role specialization and iterative feedback to reduce hallucinations and balance objectives
vs. MASI: Collab-Rec employs multi-round negotiation to refine candidates, whereas MASI aggregates single-shot outputs
vs. MACRec: Collab-Rec focuses on balancing specific stakeholder objectives (sustainability vs. popularity) via a deterministic moderator rather than general tool use [not cited in paper]
+ 1 more
vs. TopPop: Collab-Rec provides personalized, context-aware suggestions rather than static popularity lists

Limitations

Multi-round coordination increases latency and token cost compared to single-shot methods
Reliance on a fixed catalog limits the ability to recommend truly novel or dynamic events not in the database
Performance depends heavily on the base LLM's capability; smaller models (e.g., Gemma-4b) struggle with complex constraints even with the framework

Reproducibility

Code: https://github.com/ashmibanerjee/collab-rec

Publicly available: Code at https://github.com/ashmibanerjee/collab-rec. Prompts in appendix. Data: Derived from SynthTRIPs (public). Missing: Weights for proprietary models (Claude, Gemini) naturally unavailable.

📊 Experiments & Results

Evaluation Setup

Travel recommendation for 200 European cities based on 900 stratified natural language queries

Benchmarks:

SynthTRIPs-derived queries (Constraint-based travel recommendation) [New]

Metrics:

Moderator Success (Constraint Satisfaction + Grounding)
Diversity (Entropy, Gini Coefficient)
Catalog Coverage
Hallucination Rate / Reliability
Statistical methodology: Not explicitly reported in the paper

Key Results

Benchmark	Metric	Baseline	This Paper	Δ
Grounded Success Rate comparisons showing Collab-Rec's ability to satisfy constraints and map to the catalog better than baselines.
European City Queries	Moderator Success Rate	0.682	0.940	+0.258
European City Queries	Moderator Success Rate	0.765	0.940	+0.175
Diversity and Bias Mitigation results showing reduction in popularity bias.
European City Queries	Gini Coefficient	0.45	0.30	-0.15
European City Queries	Catalog Coverage	0.42	0.68	+0.26

Main Takeaways

Collab-Rec consistently outperforms single-agent (SASI) and single-round multi-agent (MASI) baselines in grounded success rate across all tested LLM families.
The framework effectively mitigates popularity bias, recommending 'hidden gems' and reducing the Gini coefficient of destination frequency.
Iterative negotiation is crucial: performance gains stabilize after 3-4 rounds, validating the early stopping mechanism to save compute.
Smaller models (e.g., Gemma-7b) benefit from the framework but still lag behind larger models (Claude-4.5) in absolute constraint satisfaction.

📚 Prerequisite Knowledge

Prerequisites

Basic understanding of Large Language Models (LLMs) and prompting
Recommender Systems concepts (ranking, diversity, accuracy)
Multi-agent system architectures (roles, moderator)

Key Terms

SASI: Single-Agent Single-Instruction—a baseline where one LLM attempts to satisfy all constraints in one prompt

MASI: Multi-Agent Single-Instruction—a baseline with multiple agents but no iterative negotiation rounds

Gini coefficient: A measure of inequality used here to quantify popularity concentration; lower values mean demand is more evenly spread across destinations

Hallucination: In this context, when the LLM suggests a destination not present in the verifiable catalog or invents attributes

Grounding: The process of mapping LLM outputs to specific, valid entries in a pre-defined database or catalog

Pareto trade-offs: Situations where improving one objective (e.g., sustainability) inevitably compromises another (e.g., personalization)

Scalarization: Converting multiple objective scores into a single value, typically via weighted sum, to rank candidates

RandRec: A baseline that recommends random valid cities from the catalog

TopPop: A baseline that recommends the most popular cities from the catalog