Social-RAG: Retrieving from Group Interactions to Socially Ground AI Generation

📝 Paper Summary

Conversational personalization Agentic RAG pipeline Group recommendation systems

Social-RAG augments LLM agents with a social knowledge base derived from group chat history to generate messages aligned with group norms and interests.

Core Problem

AI agents in group spaces often fail to align with specific group norms and interests, becoming annoying or irrelevant because they lack social context.

Why it matters:

Without social alignment, AI interventions are often ignored or abandoned by users who perceive them as intrusive.
Existing systems rely on rigid templates or explicit user feedback (polls), which imposes high user effort and disrupts natural group dynamics.
Current RAG approaches focus on factual knowledge retrieval but lack mechanisms for retrieving and utilizing 'social facts' from interaction history.

Concrete Example: A standard agent might post a generic summary of a paper to a research group. Because it doesn't know the group just discussed a specific related method yesterday, the summary feels disconnected and irrelevant, whereas Social-RAG would explicitly link the paper to that recent discussion.

Key Novelty

Social-RAG Workflow

Treats social interaction history (chats, reactions) as a 'social knowledge base' from which to retrieve 'social facts' (e.g., topical interests, communication norms).
Uses a multi-step pipeline to index group history, retrieve relevant social signals based on candidate items, and synthesize a message that socially grounds the item to the group.
Introduces a feedback loop where new group reactions to the agent's posts are indexed to update the social knowledge base dynamically.

Architecture

The Social-RAG workflow pipeline illustrating how group history is processed into a message.

Evaluation Highlights

Deployment in 18 Slack channels with 500+ researchers showed the agent successfully fostered common ground without disrupting existing practices.
Users rated Social-RAG generated messages as more contextually relevant than generic summaries.
The system successfully leveraged implicit signals (emoji reactions) to adapt to group preferences without requiring explicit configuration.

Breakthrough Assessment

7/10

Novel application of RAG to 'social facts' rather than just document facts. Strong ecological validity via real-world deployment, though the technical contribution is more about the workflow application than new architecture.

⚙️ Technical Details

Problem Definition

Setting: Generating a natural language message $m$ to recommend an item $i$ to a group $G$, given the group's interaction history $H$.

Inputs: Group interaction history $H$ (messages, reactions), Candidate item $i$ (e.g., a research paper)

Outputs: Socially grounded message $m$ explaining why $i$ is relevant to $G$

Pipeline Flow

Social Knowledge Base Construction (Indexing)
Signal Retrieval & Analysis
Message Generation
Feedback Loop

System Modules

Social Fact Indexer

Parses group chat history into a structured index

Model or implementation: Not explicitly specified (likely keyword/embedding based storage)

Signal Retriever (Retrieval)

Identifies relevant social signals based on a candidate item (e.g., a paper)

Model or implementation: Semantic Similarity / Keyword Matching

Signal Ranker (Retrieval)

Selects the most salient social signals to fit in context window

Model or implementation: Heuristic ranking (recency, relevance score)

Message Generator

Synthesizes the final post using the item and selected social signals

Model or implementation: GPT-4 (implied by context of 'LLM-based', specific model version not explicitly named in summary text)

Novel Architectural Elements

Social Knowledge Base: Unlike standard RAG which stores static documents, this stores dynamic interaction traces labeled as 'social facts'.
Feedback Loop: Explicitly indexes reactions to the agent's own previous outputs to adjust future retrieval (implicit preference learning).

Modeling

Base Model: GPT-4 (implied, typical for this class of 2024/2025 HCI work)

Reproducibility

Code: https://github.com/uw-social-computing/PaperPing

📊 Experiments & Results

Evaluation Setup

Field deployment in real-world research lab Slack channels.

Benchmarks:

Field Deployment (Real-world usage analysis) [New]

Metrics:

User Perception (Relevance, Intrusiveness)
Engagement Rates (Reactions, Replies)
Adoption/Abandonment Rates
Statistical methodology: Qualitative analysis of interview data and descriptive statistics of deployment logs.

Key Results

Benchmark	Metric	Baseline	This Paper	Δ
Field Deployment	Number of Channels	0	18	18
Field Deployment	Users Reached	0	500	500

Main Takeaways

PaperPing successfully integrated into existing group workflows without being perceived as disruptive or annoying.
The Social-RAG mechanism allowed the agent to generate explanations that felt 'grounded' by referencing specific past conversations or user interests.
Implicit feedback (reactions) was sufficient to steer the agent's topic selection over time, reducing the need for tedious manual configuration.

📚 Prerequisite Knowledge

Prerequisites

Understanding of RAG (Retrieval-Augmented Generation)
Basic knowledge of LLM prompting
Familiarity with groupware/CSCW concepts (social grounding)

Key Terms

Social-RAG: A workflow that retrieves social context (norms, interests) from group history to ground LLM generation.

Social Facts: Discrete units of social information extracted from interaction history, such as a group's specific interest in a topic or a member's expertise.

CSCW: Computer-Supported Cooperative Work—a field of study regarding how technology supports group collaboration.

RAG: Retrieval-Augmented Generation—AI systems that answer questions by first searching for relevant documents.

Common Ground: Mutual understanding and shared knowledge established between participants in a conversation.

Social Grounding: The process of linking new information to the existing shared context and social dynamics of a group.