GAP: Graph-Assisted Prompts for Dialogue-based Medication Recommendation

📝 Paper Summary

Medical Dialogue Systems Medication Recommendation

GAP improves medical dialogue recommendations by constructing a dynamic patient-centric graph to capture fine-grained history and using it to generate path-based prompts for retrieving external knowledge.

Core Problem

In multi-turn medical dialogues, LLMs often miss fine-grained historical details (like specific contraindications) and lack specialized domain knowledge, leading to unsafe or hallucinated recommendations.

Why it matters:

Safety risks: Recommending contraindicated drugs (e.g., Losartan during pregnancy) can cause severe patient harm.
Long-tail knowledge gaps: LLMs struggle to memorize rare drug interactions purely from pre-training.
Context loss: Standard RAG or ICL methods often overlook subtle state changes scattered across long dialogue histories.

Concrete Example: A patient mentions '30 weeks pregnant' early in a dialogue. Later, when asking for hypertension medication, a standard LLM recommends Losartan (unsafe for pregnancy) because it failed to track the pregnancy state across turns or check contraindications.

Key Novelty

Graph-Assisted Prompts (GAP)

Constructs an explicit 'patient-centric graph' that evolves during the dialogue, tracking medical concepts (diseases, drugs) and their specific states (e.g., pregnancy status, symptoms).
Uses 'path-based prompts' derived from this graph to formulate precise queries for external knowledge sources (KGs, Web) rather than just retrieving based on raw text similarity.

Architecture

The overall GAP framework architecture.

Evaluation Highlights

Outperforms strong baselines (like ChatGPT and specialized medical models) on dialogue-based medication recommendation tasks.
Demonstrates effectiveness in a dynamically diagnostic interviewing scenario, showing better safety and information recall.
Successfully integrates multiple knowledge sources (Knowledge Graphs and Internet search) to reduce non-factual responses.

Breakthrough Assessment

7/10

A solid application of graph-based memory to RAG in the medical domain. While the components (graphs, RAG) are known, the specific construction of patient-centric graphs for state tracking in dialogue is a practical and effective innovation for safety.

⚙️ Technical Details

Problem Definition

Setting: Dialogue-based medication recommendation

Inputs: Multi-turn dialogue history H containing patient utterances P and doctor utterances D

Outputs: Target response d_m containing diagnosis or medication recommendation

Pipeline Flow

Group 1: Graph Construction (Extraction -> Normalization -> Graph Update)
Group 2: Prompt Engineering (Neighborhood Analysis -> Path Extraction -> Query Generation)
Group 3: Generation (LLM Response)

System Modules

Concept Extractor (Graph Construction)

Identify medical concepts (diseases, symptoms) in the current utterance

Model or implementation: LLM-based extraction (Instruction tuned)

State Tracker (Slot Filling) (Graph Construction)

Determine the state/attributes of extracted concepts based on context

Model or implementation: LLM-based slot filling

Graph Manager (Graph Construction)

Update the patient-centric graph with new nodes and edges; normalize mentions to KG entities

Model or implementation: Rule-based + Linking methods (edit distance/synonyms)

Prompt Generator

Generate retrieval queries based on graph paths and neighborhood information

Model or implementation: Template-based and LLM-based rewriters

Response Generator

Generate the final medical recommendation using context and retrieved info

Model or implementation: LLM (e.g., ChatGPT)

Novel Architectural Elements

Patient-centric graph explicitly separating patient nodes from concept nodes to track dialogue states over time.
Path-based prompting mechanism that converts graph structures into natural language queries for external validation (KG/Web).

Modeling

Base Model: LLMs (Specific model versions for experiments not explicitly named in extracted text, implies usage of models like ChatGPT/GPT-4 based on context)

Training Method: Inference-time framework (Prompt Engineering / RAG)

Compute: Not reported in the paper

Comparison to Prior Work

vs. DDN: GAP uses a structured dynamic graph rather than just disease representations.
vs. GAMENet: GAP focuses on dialogue interaction details rather than static EHRs.
vs. Standard RAG (e.g., RECOMP): GAP filters retrieval using graph structure (neighborhoods/paths) rather than just semantic similarity.

Limitations

Dependency on the quality of the underlying extraction LLM; errors in entity extraction propagate to the graph.
Knowledge Graphs may be incomplete or outdated, requiring fallback to web search which introduces noise.
Constructing and querying graphs adds latency compared to simple end-to-end generation.

📊 Experiments & Results

Evaluation Setup

Dialogue-based medication recommendation and dynamic diagnostic interviewing.

Benchmarks:

Dialogue-based medication recommendation dataset (Medication Recommendation)

Metrics:

Accuracy (assumed based on 'competitive performance')
Safety/Contraindication avoidance (qualitative)
Statistical methodology: Not explicitly reported in the paper

Key Results

Benchmark	Metric	Baseline	This Paper	Δ
Detailed numeric results table was not included in the provided text snippet. The text claims competitive performance and effectiveness.

Main Takeaways

GAP effectively captures fine-grained medical information (like pregnancy status) that standard LLMs often miss in long contexts.
The integration of external knowledge (KG + Web) via graph-based prompts reduces hallucinations and unsafe recommendations.
The patient-centric graph provides an interpretable intermediate representation of the patient's state.

📚 Prerequisite Knowledge

Prerequisites

Knowledge Graphs (structure, triples)
Retrieval-Augmented Generation (RAG)
Large Language Models (In-context learning)

Key Terms

GAP: Graph-Assisted Prompts—the proposed framework that uses a patient-centric graph to guide LLM generation.

Patient-centric graph: A dynamic graph constructed during the dialogue where nodes are the patient and medical concepts, and edges represent relationships or states (e.g., 'has_symptom').

MDS: Medical Dialogue Systems—AI systems designed to converse with patients for diagnosis or consultation.

Slot-filling: A task where the model extracts specific values (e.g., 'severe') for predefined attributes (e.g., 'severity') of an entity.

RAG: Retrieval-Augmented Generation—enhancing LLMs by retrieving relevant external data.

Path-based prompts: Prompts generated by converting paths in the graph (e.g., Patient -> Disease -> Drug) into natural language queries.

ICL: In-Context Learning—providing examples or context in the prompt to guide the LLM without weight updates.