EmotionalRAG: Enhancing role-playing agents through emotional retrieval

📝 Paper Summary

Memory recall Modularized RAG pipeline

Emotional RAG mimics human cognition by retrieving memories based on both semantic relevance and emotional congruence, enabling role-playing agents to generate more character-consistent responses.

Core Problem

Existing RAG methods for role-playing agents rely solely on semantic similarity to retrieve memories, ignoring the psychological influence of emotions on memory recall.

Why it matters:

Role-playing agents (e.g., in customer service or games) need to maintain consistent personalities, but purely semantic retrieval often misses the emotional context required for human-like responses
Current approaches fail to leverage Mood-Dependent Memory theory, which suggests memories are better recalled when current mood matches the mood at encoding

Concrete Example: If a character is currently angry, standard RAG might retrieve a semantically relevant but emotionally neutral memory (e.g., a fact about their job), whereas Emotional RAG would prioritize retrieving memories of past anger or conflict, leading to a more consistent angry response.

Key Novelty

Mood-Congruent Memory Retrieval

Encodes both the semantic meaning and the emotional state (across 8 dimensions like joy, fear, anger) for every user query and stored memory fragment
Calculates retrieval scores using a dual metric: semantic distance (Euclidean) combined with emotional distance (Cosine similarity of emotion vectors)
Applies flexible strategies (Combination or Sequential) to weight or re-rank memories, prioritizing those that match the character's current emotional state

Architecture

Overview of the Emotional RAG framework, illustrating the parallel processing of queries and memories for both semantic and emotional content.

Evaluation Highlights

Outperforms Ordinary RAG (semantic-only) on InCharacter dataset using ChatGLM-6B: +0.027 MSE improvement on BFI personality consistency
Achieves higher MBTI accuracy on CharacterEval dataset with Qwen-72B: 67.53% Acc (Full) vs 59.74% for Ordinary RAG
Consistent gains across three datasets (InCharacter, CharacterEval, Character-LLM) verify that emotional context aids personality maintenance

Breakthrough Assessment

7/10

Novel application of psychological memory theory to RAG. While the architectural changes are straightforward (dual encoding), the consistent empirical gains across personality benchmarks demonstrate the value of emotional context.

⚙️ Technical Details

Problem Definition

Setting: Role-playing conversation generation where an agent R must answer query q using memory M to maximize personality consistency

Inputs: User query q, Character role information R, Memory unit M = {m1, ..., mn}

Outputs: Generated response a that mimics the character's personality and emotional state

Pipeline Flow

Query/Memory Encoding: Generate semantic vectors (via embedding model) and emotional vectors (via LLM prompting)
Emotional Retrieval: Calculate combined similarity scores based on both semantic and emotional distances
Response Generation: Feed query + retrieved emotional memories + role profile to LLM

System Modules

Semantic Encoder (Input Processing)

Convert text into dense semantic vectors

Model or implementation: bge-base-zh-v1.5

Emotion Modeler (Input Processing)

Quantify emotional state across 8 dimensions (joy, sadness, etc.)

Model or implementation: GPT-3.5

Dual-Strategy Retriever

Select top-k memories using combined semantic and emotional scores

Model or implementation: Algorithm (Combination or Sequential)

Novel Architectural Elements

Parallel encoding pathway: Processing data through both a semantic embedder and an LLM-based emotion scorer simultaneously
Hybrid retrieval scoring function: Explicitly combining Euclidean semantic distance with Cosine emotional distance

Modeling

Base Model: Evaluated on ChatGLM3-6B, Qwen1.5-72B, and GPT-3.5-turbo

Training Method: Inference-time retrieval augmentation (no fine-tuning of the generator reported)

Compute: Not reported in the paper

Comparison to Prior Work

vs. Ordinary RAG: Incorporates explicit emotional vector dimensions into the retrieval metric
vs. Generative Agents [not cited in paper]: Focuses specifically on emotion-congruent retrieval rather than broader reflection/planning mechanisms
vs. MemGPT [not cited in paper]: MemGPT manages context window via OS-like paging; Emotional RAG focuses on the *selection criteria* (emotion) for retrieval rather than memory management mechanics

Limitations

Reliance on GPT-3.5 for emotion annotation introduces latency and cost compared to a lightweight classifier
Evaluated primarily on role-playing/personality consistency, effectiveness for factual QA is unclear
Does not model the *evolution* of emotion (how retrieving a sad memory might change the current mood)

Reproducibility

Code: https://github.com/BAI-LAB/EmotionalRAG

📊 Experiments & Results

Evaluation Setup

Role-playing agents answer psychological questionnaires (MBTI/BFI) to test if their responses align with ground-truth character personalities.

Benchmarks:

InCharacter (Role-playing personality assessment (BFI & MBTI))
CharacterEval (Role-playing personality assessment (MBTI))
Character-LLM (Role-playing personality assessment (MBTI))

Metrics:

BFI MSE (Mean Squared Error against ground truth)
MBTI Accuracy (Dim: single dimension acc, Full: full 4-letter type acc)
Statistical methodology: Not explicitly reported in the paper

Key Results

Benchmark	Metric	Baseline	This Paper	Δ
BFI results on InCharacter dataset showing Emotional RAG improves personality consistency (lower MSE is better).
InCharacter	BFI MSE	0.584	0.557	-0.027
InCharacter	BFI MSE	0.457	0.441	-0.016
MBTI Accuracy results on CharacterEval dataset showing higher accuracy in predicting full personality types.
CharacterEval	MBTI Acc (Full)	0.5974	0.6753	+0.0779
CharacterEval	MBTI Acc (Full)	0.1948	0.2727	+0.0779

Experiment Figures

The prompt template used for Emotional Modeling (extracting emotional vectors from text).

Main Takeaways

Emotional RAG consistently outperforms Ordinary RAG (semantic only) in maintaining character personality traits across different LLM backbones (ChatGLM, Qwen, GPT).
Improvements are more pronounced on smaller/weaker models (ChatGLM) or when personality traits are complex, suggesting emotional context acts as a strong guide for consistency.
The method validates the psychological Mood-Dependent Memory theory in the context of LLM agents: retrieving emotionally congruent memories yields more human-like behavior.

📚 Prerequisite Knowledge

Prerequisites

Understanding of Retrieval-Augmented Generation (RAG)
Basic knowledge of vector embeddings and similarity metrics (Cosine, Euclidean)
Familiarity with personality models (Big Five Inventory, MBTI)

Key Terms

Mood-Dependent Memory: Psychological theory stating events are better recalled if the mood during retrieval matches the mood during the original experience

BFI: Big Five Inventory—a model measuring personality across Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism

MBTI: Myers-Briggs Type Indicator—a personality taxonomy categorizing people into 16 types based on 4 dichotomies (e.g., Introversion vs. Extroversion)

Euclidean distance: Straight-line distance between two points in vector space, used here to measure semantic similarity

Cosine similarity: Measure of similarity between two non-zero vectors that measures the cosine of the angle between them, used here for emotional vectors