Reasoning Implicit Sentiment with Chain-of-Thought Prompting

📝 Paper Summary

Implicit Sentiment Analysis (ISA) Chain-of-Thought (CoT) Prompting

THOR is a three-step chain-of-thought prompting framework that enables Large Language Models to detect implicit sentiment by sequentially inferring the aspect, the underlying opinion, and finally the sentiment polarity.

Core Problem

Implicit Sentiment Analysis (ISA) is difficult because input texts contain only factual descriptions without explicit opinion words, requiring common-sense reasoning to infer the latent intent.

Why it matters:

Standard sentiment classifiers fail on texts lacking explicit emotion words (e.g., 'Try the tandoori salmon!'), predicting neutral instead of positive.
Accurate sentiment analysis in real-world scenarios requires understanding context and hidden opinions, which traditional non-reasoning models lack.
Existing fine-grained sentiment analysis methods struggle to connect obscure cues to sentiment polarities without explicit multi-step inference.

Concrete Example: Given 'The new mobile phone can be just put in my pocket', a standard model predicts 'neutral'. It fails to reason that fitting in a pocket implies 'portability', which is 'convenient' (good opinion), leading to a 'positive' sentiment.

Key Novelty

Three-hop Reasoning (THOR) CoT Framework

Decomposes implicit sentiment detection into three sequential reasoning steps: (1) inferring the specific aspect mentioned, (2) inferring the underlying opinion based on common sense, and (3) predicting the final polarity.
Uses a self-consistency mechanism to aggregate multiple reasoning paths, selecting answers with high agreement to ensure robustness at each step.
Incorporates a 'reasoning revising' method for supervised settings where gold labels supervise the generation of intermediate reasoning steps.

Architecture

The three-hop reasoning process of THOR compared to traditional prompting.

Evaluation Highlights

+51.10% average F1 improvement on Zero-shot setup using GPT3 (175B) compared to the best baseline (BERT Asp+SCAPT).
+6.65% average F1 improvement on Supervised setup using Flan-T5 (11B) compared to the best baseline.
Scaling laws hold: Improvement scales exponentially with model size, with GPT3-175B achieving near-supervised performance in zero-shot settings.

Breakthrough Assessment

8/10

Achieves massive improvements (>50% F1) in zero-shot implicit sentiment analysis by successfully applying Chain-of-Thought reasoning, effectively solving a long-standing challenge for standard classifiers.

⚙️ Technical Details

Problem Definition

Setting: Implicit Sentiment Analysis (ISA)

Inputs: A sentence X with a target term t (subset of X)

Outputs: Sentiment polarity y (positive, neutral, or negative) towards t

Pipeline Flow

Prompt 1: Infer Aspect
Prompt 2: Infer Opinion (conditioned on Aspect)
Prompt 3: Infer Polarity (conditioned on Aspect + Opinion)

System Modules

Aspect Inference (Hop 1) (Reasoning Step)

Identify the specific aspect of the target mentioned in the text

Model or implementation: LLM (Flan-T5 or GPT3)

Opinion Inference (Hop 2) (Reasoning Step)

Deduce the implicit opinion towards the inferred aspect using common sense

Model or implementation: LLM (Flan-T5 or GPT3)

Polarity Prediction (Hop 3) (Reasoning Step)

Determine the final sentiment label based on the explicit context and inferred opinion

Model or implementation: LLM (Flan-T5 or GPT3)

Novel Architectural Elements

Three-step sequential prompting pipeline specifically designed for ISA (Aspect → Opinion → Polarity)
Integration of Self-consistency voting at each individual reasoning hop

Modeling

Base Model: Flan-T5 (250M, 780M, 3B, 11B), GPT3 (350M to 175B), ChatGPT

Training Method: Reasoning Revising (Supervised Fine-Tuning)

Objective Functions:

Purpose: Teach the model to generate correct intermediate reasoning by supervising the final prediction given that reasoning.

Formally: Standard cross-entropy loss on the final sentiment label y given inputs constructed from reasoning steps.

Adaptation: Full fine-tuning for Flan-T5; Prompting only for GPT3

Training Data:

SemEval14 Laptop and Restaurant datasets (split into explicit and implicit subsets by Li et al. 2021)

Compute: Experiments conducted with 4 NVIDIA A100 GPUs

Comparison to Prior Work

vs. BERT+SCAPT: THOR uses explicit multi-hop reasoning (CoT) rather than contrastive pre-training.
vs. Zero-shot CoT ('Let's think step by step'): THOR imposes a structured 3-step schema (Aspect-Opinion-Polarity) specific to sentiment analysis, rather than generic reasoning.
vs. Vanilla Prompting: THOR breaks the task into intermediate steps, enabling the model to access latent knowledge before making a final decision.

Limitations

Dependency on Model Scale: Improvement is limited on smaller models (e.g., Flan-T5-base) due to weaker reasoning capabilities.
Annotation Errors: Unsupervised large models (GPT3) sometimes fail due to incorrect ground truth annotations in the dataset rather than reasoning errors.
Computational Cost: Multiple inference steps and self-consistency voting increase inference latency compared to direct classification.

Reproducibility

Code: https://github.com/scofield7419/THOR-ISA

Code publicly available at https://github.com/scofield7419/THOR-ISA. Uses publicly available SemEval14 datasets. GPT3 used via API (no fine-tuning possible). Flan-T5 models available via HuggingFace.

📊 Experiments & Results

Evaluation Setup

Sentiment Classification on SemEval14 datasets (Laptop and Restaurant)

Benchmarks:

SemEval14 Laptop (ISA subset) (Implicit Sentiment Analysis)
SemEval14 Restaurant (ISA subset) (Implicit Sentiment Analysis)

Metrics:

F1 Score
Statistical methodology: Not explicitly reported in the paper

Key Results

Benchmark	Metric	Baseline	This Paper	Δ
Supervised fine-tuning results showing THOR (Flan-T5 11B) outperforming previous state-of-the-art baselines on implicit sentiment subsets.
SemEval14 Restaurant (ISA)	F1	72.28	79.73	+7.45
SemEval14 Laptop (ISA)	F1	77.59	82.43	+4.84
Zero-shot results demonstrate massive gains using THOR with large models (GPT3-175B), far exceeding baselines that struggle without reasoning.
SemEval14 Restaurant (ISA)	F1	30.02	81.96	+51.94
SemEval14 Laptop (ISA)	F1	25.77	76.04	+50.27
SemEval14 Restaurant (ISA)	F1	87.45	86.03	-1.42

Experiment Figures

Impact of LLM scale on F1 performance for both standard prompting and THOR.

Error analysis comparing Supervised T5 (11B) and Unsupervised GPT3 (175B).

Main Takeaways

THOR enables large language models (GPT3-175B) to achieve supervised-level performance in a zero-shot setting for implicit sentiment analysis.
Model scale is critical: The efficacy of multi-hop reasoning prompting improves exponentially with model size (from 350M to 175B).
Structured CoT (THOR) significantly outperforms generic Zero-shot CoT ('Let's think step by step') on ISA tasks (e.g., +5.2% F1 on Restaurant ISA with Flan-T5 11B).
Self-consistency and Reasoning Revising are essential components, contributing noticeably to the final performance in supervised settings.

📚 Prerequisite Knowledge

Prerequisites

Understanding of Sentiment Analysis (Explicit vs. Implicit)
Familiarity with Large Language Models (LLMs) and Prompt Engineering
Chain-of-Thought (CoT) reasoning concepts

Key Terms

ISA: Implicit Sentiment Analysis—detecting sentiment in text that lacks explicit emotional words (e.g., factual statements implying an opinion)

CoT: Chain-of-Thought—a prompting technique where the model generates intermediate reasoning steps before the final answer

ESA: Explicit Sentiment Analysis—traditional sentiment analysis where texts contain direct emotional words like 'happy' or 'terrible'

THOR: Three-hop Reasoning—the proposed framework that infers Aspect → Opinion → Polarity sequentially

Self-consistency: A decoding strategy where the model generates multiple reasoning paths and selects the most consistent answer via voting

Reasoning Revising: A supervised fine-tuning technique where the model is trained to predict the final label using its own generated reasoning steps as input context

LLM: Large Language Model—massive neural networks trained on vast text data, capable of complex tasks via prompting

Flan-T5: An instruction-tuned version of the T5 (Text-to-Text Transfer Transformer) model