Towards a Neurosymbolic Reasoning System Grounded in Schematic Representations

📝 Paper Summary

Neurosymbolic AI Cognitive Modeling Spatial Reasoning

Embodied-LM operationalizes cognitive image schemas by using LLMs to translate natural language problems into Answer Set Programs grounded in spatial primitives like PATH and CONTAINER for executable reasoning.

Core Problem

LLMs lack robust mental representations for logical reasoning, often failing to maintain consistency in described situations or reason soundly on problems with many premises.

Why it matters:

Current LLMs purely process text statistics, missing the sensorimotor 'image schemas' (like containment or paths) that humans use to structure understanding of abstract concepts
Purely neural approaches struggle with maintaining state consistency in multi-step deductive reasoning tasks

Concrete Example: In a problem like 'Alice is older than Bill, and Charles is younger than Bill', an LLM might hallucinate relationships. Embodied-LM maps this to a spatial 'timeline' (PATH schema), where geometric constraints force 'Alice > Charles' to be true automatically (a 'free ride' inference).

Key Novelty

Neurosymbolic Reasoning Grounded in Image Schemas

Translates abstract logical problems into spatial geometry problems by identifying the underlying 'image schema' (e.g., mapping time to a spatial line)
Uses an LLM to generate Answer Set Programming (ASP) code that defines these spatial relationships, then solves them using a Declarative Spatial Reasoning solver

Architecture

The Embodied-LM pipeline transforming natural language into executable spatial logic

Evaluation Highlights

91.00% accuracy on the LogicalDeduction dataset, outperforming GPT-4 Chain-of-Thought (75.25%) and Logic-LM (87.63%)
Demonstrates capability to solve complex constraints (Zebra Puzzles) by mapping diverse relationships (ownership, attributes) to the CONTAINER image schema
Provides executable witnesses (visualizations of spatial configurations) for answers, enhancing interpretability compared to black-box LLMs

Breakthrough Assessment

7/10

Strong proof-of-concept connecting cognitive science (image schemas) with neurosymbolic execution. Achieves competitive performance with high interpretability, though currently limited to specific spatial primitives.

⚙️ Technical Details

Problem Definition

Setting: Natural Language Logical Reasoning where a context C, question Q, and options O are mapped to a spatial logic program P to find the entailed answer

Inputs: Natural language context, question, and multiple-choice options

Outputs: Selected answer option derived from the stable models of the generated ASP program

Pipeline Flow

Context Interpretation (LLM) → Question Formalization (LLM) → Symbolic Solving (Clingo+DSR)

System Modules

Schema Interpreter (Translation)

Identify applicable image schemas and translate natural language context into ASP spatial atoms

Model or implementation: GPT-4

Question Formalizer (Translation)

Translate the question and options into ASP rules corresponding to the generated context structure

Model or implementation: GPT-4

Spatial Solver

Execute the ASP program using theory propagation to enforce geometric consistency

Model or implementation: Clingo extended with DSR (using Z3 SMT backend)

Novel Architectural Elements

Integration of Declarative Spatial Reasoning (DSR) within the neurosymbolic pipeline to physically ground abstract reasoning tasks
Use of 'theory propagation' between ASP and SMT solvers to validate LLM-generated schematic interpretations

Modeling

Base Model: GPT-4 (for translation steps)

Training Method: In-context learning / Prompting only

Compute: Not reported in the paper

Comparison to Prior Work

vs. Logic-LM: Embodied-LM translates to *spatial* schemas (geometry) rather than abstract logic, leveraging cognitive 'free rides'
vs. VERUS-LM: Focuses specifically on identifying and utilizing sensorimotor primitives (PATH, CONTAINER) as the reasoning substrate
vs. GPT-4 CoT: Offloads reasoning steps to a symbolic solver rather than relying on the LLM's internal simulation

Limitations

Currently restricted to problems mappable to spatial primitives (PATH, CONTAINER)
Relying on LLM for translation introduces potential for mapping errors (e.g., inconsistent directionality)
Requires problem-specific prompting strategies to activate the correct schema

Reproducibility

Code: https://github.com/fcs-olivier/embodied-lm

Publicly available code (https://github.com/fcs-olivier/embodied-lm). Prompts provided in Appendices A-C. Uses commercial GPT-4 API.

📊 Experiments & Results

Evaluation Setup

Logical deduction tasks where text descriptions imply ordered or spatial relationships

Benchmarks:

LogicalDeduction (Deductive reasoning about ordered objects (BigBench))
Zebra Puzzle (Constraint satisfaction puzzle)

Metrics:

Accuracy (%)
Statistical methodology: Not explicitly reported in the paper

Key Results

Benchmark	Metric	Baseline	This Paper	Δ
Comparative performance on the LogicalDeduction dataset shows Embodied-LM competing with state-of-the-art neurosymbolic systems.
LogicalDeduction	Accuracy	93.00	91.00	-2.00
LogicalDeduction	Accuracy	87.63	91.00	+3.37
LogicalDeduction	Accuracy	75.25	91.00	+15.75
LogicalDeduction	Accuracy	88.67	91.00	+2.33

Experiment Figures

Bar chart comparing system accuracy on LogicalDeduction dataset against baselines (Left) and visualization of Zebra puzzle models (Right)

Main Takeaways

A single image schema (PATH) can successfully ground diverse logical domains including temporal sequences, spatial arrangements, and ordinal rankings
The system can handle complex multi-constraint scenarios like Zebra puzzles by unifying relationships (ownership, attributes) under the CONTAINER schema
The approach provides interpretability by generating executable programs and 'witnesses' (spatial visualizations) that allow for error diagnosis
Performance is competitive with or superior to other neurosymbolic approaches while offering a distinct cognitive grounding

📚 Prerequisite Knowledge

Prerequisites

Answer Set Programming (ASP)
Large Language Models (LLMs)
Cognitive Science (Image Schemas)
Qualitative Spatial Reasoning

Key Terms

Image Schema: Recurring cognitive patterns (e.g., CONTAINER, PATH) derived from sensorimotor experience that structure human understanding of abstract concepts

ASP: Answer Set Programming—a declarative programming paradigm used for difficult combinatorial search problems

DSR: Declarative Spatial Reasoning—a framework enabling reasoning about qualitative spatial relations (e.g., left_of) via analytic geometry constraints

Clingo: A state-of-the-art solver for Answer Set Programming

SMT: Satisfiability Modulo Theories—computational logic used here to verify if spatial constraints (geometric coordinates) are satisfiable

Propagator: A solver component that connects the logical search (ASP) with specific theory constraints (Spatial/Geometric) to prune invalid solutions

Free ride: An inference drawn automatically from the structural constraints of a representation (e.g., A left of B, B left of C -> A left of C) without explicit deduction steps