Reasoning with Natural Language Explanations

📝 Paper Summary

Explanation-based Natural Language Inference (NLI) Neuro-symbolic AI Philosophy of Science (Epistemology)

This tutorial bridges the gap between philosophical theories of scientific explanation and practical NLI architectures, proposing a framework where explanatory reasoning is grounded in formal epistemological accounts rather than just surface-level text generation.

Core Problem

Current explanation-based NLI research lacks theoretical grounding in epistemology, leading to models that generate fluent but logically weak explanations and evaluation metrics that fail to measure true reasoning.

Why it matters:

The disconnect between engineering practice and theories of explanation (e.g., Hempel, Salmon) results in undefined hypotheses about what constitutes a valid inference
Neural models (especially LLMs) suffer from lack of interpretability, susceptibility to shortcuts, and hallucinations when generating explanations without formal constraints
Existing benchmarks often focus on extractive tasks, failing to test abstractive reasoning capabilities required for complex domains like science or law

Concrete Example: In a multi-hop retrieval scenario, a model might suffer from 'semantic drift'—retrieving a chain of sentences that are lexically similar but logically disconnected. Because current methodologies lack a 'unificationist' or 'causal-mechanical' grounding, metrics might score this high based on word overlap, effectively masking the model's failure to reason.

Key Novelty

Epistemological-Linguistic Framework for NLI

Systematically categorizes NLI architectures (retrieval, generative, neuro-symbolic) through the lens of Philosophy of Science, mapping methods to accounts like the Deductive-Nomological or Unificationist models
Proposes a shift from purely material inference (content-based) to formal inference control via neuro-symbolic constraints and latent space geometry

Breakthrough Assessment

8/10

While a tutorial rather than a model paper, it provides a critical theoretical unification for the field, addressing the fundamental 'why' and 'how' of explanation that is often overlooked in engineering-focused papers.

⚙️ Technical Details

Problem Definition

Setting: Explanation-based Natural Language Inference (NLI)

Inputs: Natural language premise and hypothesis (or question)

Outputs: A label (e.g., entailment) and a constructed natural language explanation supporting the inference

Limitations

Methodologies are currently poorly informed by theoretical accounts of explanation (Salmon, Woodward, Ross)
Current models lack robustness and control, struggling with complex reasoning chains
Generative paradigms (LLMs) persist with issues of hallucinations and faithfulness despite prompting techniques
Retrieval-based methods face a trade-off between semantic drift and computational efficiency

Reproducibility

Code: https://sites.google.com/view/reasoning-with-explanations

This is a tutorial paper. The authors provide a website (https://sites.google.com/view/reasoning-with-explanations) which serves as the central resource for the covered material.

📚 Prerequisite Knowledge

Prerequisites

Natural Language Inference (NLI)
Deep Learning (LLMs)
Basic Epistemology (Philosophy of Science)

Key Terms

NLI: Natural Language Inference—determining if a hypothesis logically follows from a premise

Extractive NLI: Tasks where the explanation is directly derivable from the problem statement's surface form (e.g., selecting sentences)

Abstractive NLI: Tasks requiring external definitions or abstract relations not immediately available in the context to construct the explanation

Neuro-symbolic AI: Systems combining neural networks (learning from data) with symbolic logic (reasoning with rules) for better control and validity

Deductive-nomological: A philosophical account where an explanation is a logical deduction of a phenomenon from general laws and initial conditions

Semantic Drift: A failure mode in multi-hop reasoning where a model gradually loses the original context as it traverses a chain of related information

Unificationist account: A theory viewing explanation as the ability to derive many phenomena from a few argument patterns

Latent geometry: Approaches using VAEs (Variational Autoencoders) to structure the latent space so that inference steps correspond to geometric transformations