Should agentic conversational AI change how we think about ethics? Characterising an interactional ethics centred on respect

📝 Paper Summary

Agentic AI Ethics Human-AI Interaction

The authors propose shifting AI ethics from evaluating static content (HHH) to evaluating dynamic social interactions through a framework of respect, treating agents as social actors.

Core Problem

Current AI alignment criteria like HHH (Helpful, Honest, Harmless) focus on semantic content, failing to account for the pragmatic, social context of how agentic systems treat users.

Why it matters:

As agents become proactive and autonomous, they function as social actors; merely avoiding toxic output is insufficient for ethical interaction
Technically 'helpful' outputs can still be experienced as rude, patronizing, or manipulative when they violate social scripts or context
Existing frameworks view LLMs as passive mediums, overlooking risks inherent to the dynamics of ongoing relationships and situated speech acts

Concrete Example: A user might be interrupted by an agent offering unprompted advice. While the advice is semantically 'helpful' and 'honest' (satisfying HHH), the act of interrupting makes it pragmatically 'rude' or 'controlling' in the social context, causing interactional harm.

Key Novelty

Interactional Ethics of Respect

Shifts the unit of ethical analysis from the 'utterance' (is this sentence toxic?) to the 'interaction' (does this act treat the user with appropriate regard?)
Operationalizes 'respect' not just as politeness, but as fulfilling psychological duties: supporting the user's autonomy, competence, and social value (based on Self-Determination Theory)
Classifies harms specifically arising from the 'social actor' role, including cumulative relationship harms and manipulative influence, which static benchmarks miss

Architecture

A conceptual diagram illustrating the nested levels of analysis for AI systems, moving from broad social artifacts to specific social actors.

Breakthrough Assessment

7/10

A significant conceptual reframing for the field. While it lacks a technical implementation, it exposes a critical blind spot in current alignment research (HHH) as models move from chatbots to agents.

⚙️ Technical Details

Problem Definition

Setting: Normative framework for evaluating Agentic AI behaviour in social contexts

Inputs: Situated interactions between users and agentic AI systems

Outputs: Ethical evaluation based on interactional duties of respect (Autonomy, Competence, Relatedness)

Comparison to Prior Work

vs. HHH: Incorporates situational context (pragmatics) rather than just content (semantics); acknowledges that 'helpful' content can be 'disrespectful' in context
vs. Fairness/Transparency: Focuses on the dynamic treatment of the user during interaction rather than static properties of the model or data
vs. UX Design Patterns: Rejects 'engagement' or 'friendliness' (e.g., using emojis) as proxies for good design, arguing they can be manipulative or patronizing if not grounded in respect

Limitations

The concept of 'respect' is culturally dependent and ambiguous, making it difficult to operationalize universally
No empirical experiments or quantitative benchmarks are provided to validate the framework
Requires agents to have context awareness that may be technically difficult to implement (e.g., knowing when not to interrupt)
Does not address how to resolve conflicts between being respectful (e.g., validating a user) and being honest (e.g., correcting a factual error)

Reproducibility

No replication artifacts mentioned in the paper. This is a theoretical position paper; no code or models were trained.

📊 Experiments & Results

Evaluation Setup

Theoretical analysis and taxonomy construction (no empirical experiments)

Metrics:

Statistical methodology: Not explicitly reported in the paper

Main Takeaways

Harms in agentic AI must be categorized into three interactional types: Direct Harm (offensive acts), Harmful Influence (manipulation/deception), and Collective Harm (cumulative effects on self-image/relationship)
Evaluation of agents must move beyond the 'utterance' level to the 'relationship' level, considering how a series of interactions affects user autonomy and competence
Respect implies negative duties (do not impede autonomy) and positive duties (support competence and relatedness), offering a more robust guide than 'harmlessness' alone
Social cues (anthropomorphism) increase the user's expectation of social observance; failure to follow social scripts (e.g., forgetting past context) is experienced as a moral violation, not just a bug

📚 Prerequisite Knowledge

Prerequisites

AI Alignment Ethics
Pragmatics (Linguistics)
Moral Philosophy (Kant/Darwall)
Self-Determination Theory (Psychology)

Key Terms

HHH: Helpful, Honest, Harmless—the standard criteria currently used for aligning Large Language Models

Pragmatics: The branch of linguistics dealing with language in use and the contexts in which it is used (as opposed to Semantics, which is literal meaning)

Social Actor: An entity that is treated as having intentions and social standing; in the CASA paradigm, humans intuitively treat computers as social actors

CASA: Computers Are Social Actors—a research paradigm showing people apply human social norms to computers

Recognition Respect: A philosophical concept (Darwall) referring to treating someone in a way that gives appropriate weight to their standing as a person

BPNT: Basic Psychological Needs Theory—a sub-theory of Self-Determination Theory positing that autonomy, competence, and relatedness are essential for wellbeing

Speech Act: An utterance that performs an action (e.g., promising, ordering, apologizing) rather than just conveying information

Agentic AI: AI systems that actively pursue goals, initiate actions, and operate with a degree of autonomy in the world