CogRec: A Cognitive Recommender Agent Fusing Large Language Models and Soar for Explainable Recommendation

📝 Paper Summary

Neuro-Symbolic AI Cognitive Architectures Explainable Recommendation

CogRec integrates the Soar cognitive architecture with LLMs to create a recommender agent that uses symbolic reasoning for transparency and LLMs for knowledge bootstrapping, enabling online learning via rule chunking.

Core Problem

LLM-based recommenders suffer from 'black-box' hallucination and limited online learning, while cognitive architectures like Soar struggle with laborious manual knowledge acquisition.

Why it matters:

Existing LLM recommenders lack rigorous logical constraints, leading to inconsistent or factually incorrect explanations
Updating vast LLM parameters for real-time user preference changes is computationally inefficient and difficult
Pure symbolic systems (like Soar) offer interpretability but are unscalable due to the 'knowledge acquisition bottleneck' of manual rule encoding

Concrete Example: A user likes 'Star Wars' and 'Blade Runner'. A standard LLM might recommend a generic sci-fi movie with a hallucinated reason. CogRec recognizes a 'tie' impasse between candidates, queries the LLM to identify the specific 'cyberpunk' sub-genre preference, converts this insight into a permanent symbolic rule, and recommends 'Blade Runner 2049' with a verified reasoning trace.

Key Novelty

Neuro-Symbolic Bridge with Impasse-Driven Learning

Uses the Soar architecture as the 'cognitive brain' for structured reasoning and the LLM as an on-demand 'external consultant' only when symbolic rules fail (an impasse)
Implements a bridge module that translates symbolic states into text prompts and converts LLM natural language responses back into structured production rules (chunking)
Enables the agent to permanently learn new logic from the LLM, reducing future dependency on expensive LLM calls

Architecture

The Perception-Cognition-Action (PCA) feedback loop of CogRec.

Evaluation Highlights

Outperforms GPT4Rec and SASRec on Hit Rate and NDCG across MovieLens-1M, Amazon Movies, and Yelp datasets
Reduces LLM call frequency significantly over time as the agent 'chunks' new rules, whereas variants without chunking require constant querying
demonstrates superior performance on long-tail items (bottom 80%) compared to collaborative filtering baselines by leveraging common-sense knowledge

Breakthrough Assessment

8/10

Novel integration of a classic cognitive architecture (Soar) with modern LLMs. Effectively addresses the static nature of LLMs via symbolic rule learning, offering a distinct path for explainable, adaptive agents.

⚙️ Technical Details

Problem Definition

Setting: Next-item prediction with explainable reasoning traces

Inputs: User historical interaction sequence S_u and current query q_u

Outputs: A tuple (L_u, E_u) containing a ranked list of items and corresponding symbolic reasoning traces

Pipeline Flow

Perception: LLM encodes user history into symbolic state
Cognition: Soar reasons using production rules; if stuck, triggers Impasse
Bridge: Translates Impasse to Prompt -> Queries LLM -> Translates Reply to Rule
Action: Soar applies operator (learned or existing) to generate recommendation

System Modules

LLM Semantic Encoder

Initializes Working Memory by extracting entities/intents from user history

Model or implementation: Deepseek-V3

Soar Core

Main reasoning engine executing the Decision Cycle (Propose, Select, Apply)

Model or implementation: Soar (via pySoar)

Neuro-Symbolic Bridge (Learning/Bridge)

Bi-directional translator between Soar's symbols and LLM's natural language

Model or implementation: Rule-based parsers / Template fillers

LLM Knowledge Generator (Learning/Bridge)

Provides external knowledge/solutions during impasses

Model or implementation: Deepseek-V3

Novel Architectural Elements

Bridge Module enabling 'Text-to-Chunk' conversion for online rule learning
Impasse-driven LLM querying mechanism (only calling LLM when symbolic reasoning fails)

Modeling

Base Model: Deepseek-V3 (for LLM components)

Training Method: In-context learning and symbolic rule compilation (Chunking)

Training Data:

Knowledge Bootstrapping: ~10 prompt templates per dataset to generate ~500 initial rules

Key Hyperparameters:

embedding_dimension: 128
learning_rate: 1e-4
weight_decay: 0.01
+ 2 more
optimizer: Adam
epochs: 20

Compute: NVIDIA 4090 GPU

Comparison to Prior Work

vs. GPT4Rec: CogRec uses structured symbolic reasoning rather than purely generative reasoning, reducing hallucination
vs. PGPR: CogRec learns rules online from LLM knowledge rather than relying on static graph paths
vs. LLM-Direct: CogRec reduces computational cost via rule caching (chunking) and provides formal logical traces
+ 1 more
vs. ReAct [not cited in paper]: CogRec uses internal cognitive state (Soar) for persistence rather than just context window history

Limitations

Dependency on the quality of the LLM's initial common-sense knowledge
Parsing natural language into rigid symbolic rules (Text-to-Chunk) can be brittle
Latency during the initial 'learning phase' when impasses are frequent

Reproducibility

Implemented in Python using pySoar and Deepseek-V3 API. Datasets (MovieLens-1M, Amazon Review Movies, Yelp) are public. Code URL not provided in text.

📊 Experiments & Results

Evaluation Setup

Top-K Recommendation and Explainability

Benchmarks:

MovieLens-1M (Movie Recommendation)
Amazon Review (Movies) (Movie Recommendation w/ Reviews)
Yelp (Business/Service Recommendation)

Metrics:

Hit Rate (HR@10, HR@20)
NDCG (N@10, N@20)
LLM-Call Frequency (LCF)
Statistical methodology: Not explicitly reported in the paper

Key Results

Benchmark	Metric	Baseline	This Paper	Δ
CogRec consistently outperforms baselines across all three datasets.
MovieLens-1M	HR@10	0.2615	0.2842	+0.0227
Ablation studies demonstrate the critical role of the Chunking mechanism and LLM bootstrapping.
Not specified (General)	Performance (General)	Low (Drastic Drop)	High	Large positive
Not specified	LLM-Call Frequency (LCF)	Consistently High	Converges to low plateau	Significant reduction

Experiment Figures

LLM-Call Frequency (LCF) over interaction steps for CogRec vs. CogRec w/o Chunking.

Performance comparison on Head (top 20%) vs. Long-tail (bottom 80%) items.

Main Takeaways

CogRec outperforms both traditional (SASRec) and LLM-based (GPT4Rec, LLM-Direct) baselines, validating the neuro-symbolic approach.
The 'Chunking' mechanism effectively converts expensive LLM calls into cheap, permanent symbolic rules, enabling the system to become more autonomous over time.
CogRec shows superior robustness on long-tail items compared to collaborative filtering, likely due to utilizing LLM common-sense knowledge rather than just interaction statistics.

📚 Prerequisite Knowledge

Prerequisites

Understanding of production systems (IF-THEN rules)
Familiarity with Recommender Systems (collaborative filtering, sequential recommendation)
Basic knowledge of Large Language Models and Chain-of-Thought prompting

Key Terms

Soar: A cognitive architecture that simulates human cognition using production rules, working memory, and long-term memory

Chunking: Soar's learning mechanism that compiles the results of problem-solving in an impasse into new, permanent production rules

Impasse: A state in Soar where available knowledge is insufficient to select the next operator, triggering a sub-goal or external query

Production Rule: A symbolic logic unit in the format 'IF conditions THEN actions' used for reasoning

Working Memory Elements (WMEs): Dynamic data elements stored in Soar's working memory, typically as (object ^attribute value) triplets

PCA Cycle: Perception-Cognition-Action cycle, the feedback loop governing the agent's interaction with the environment

Neuro-Symbolic: AI systems combining neural networks (like LLMs) with symbolic logic (like Soar) to leverage strengths of both

Chain of Thought (CoT): Prompting technique where the LLM generates intermediate reasoning steps before the final answer

Cold-start: The difficulty of recommending items to new users or recommending new items with little interaction history

Long-tail: Items that are rarely interacted with; traditionally hard for recommenders due to data sparsity