User Characteristics in Explainable AI: The Rabbit Hole of Personalization?

📝 Paper Summary

Personalization (P13N) User modeling Explainable AI (XAI)

An empirical study with 149 participants reveals that most user characteristics—such as gender, experience, and personality—do not significantly impact the understanding of or trust in AI explanations.

Core Problem

XAI research increasingly attempts to personalize explanations based on fine-grained user characteristics (like personality or gender), assuming this improves user understanding and trust without sufficient empirical evidence.

Why it matters:

Designing personalized XAI systems adds significant complexity and cost; if user characteristics don't matter, this effort is wasted (the 'rabbit hole').
Prior work suggests links between traits and XAI reception, but findings are inconsistent, and no large-scale validation has confirmed which traits actually necessitate personalization.

Concrete Example: A system might try to present different explanations to a 'neurotic' user versus an 'open' user. This paper suggests such differentiation is likely useless, as the study found no link between most traits and engagement or trust.

Key Novelty

Critical Empirical Re-evaluation of XAI Personalization

Conducts a controlled user study (N=149) explicitly testing the relationship between detailed user traits (Big 5 personality, age, gender, experience) and XAI outcomes.
Provides negative evidence against the prevailing trend of 'micro-personalization' in XAI, suggesting that generic good design matters more than tailoring to specific user profiles.

Architecture

The Explanatory Debugging Interface used by participants.

Evaluation Highlights

Study included 149 participants recruited via Prolific, engaging with a toxicity classifier model (Detoxify) achieving 98 AUC.
Analysis revealed that of all characteristics tested, only Age and Openness (personality) had any statistical association with Actual Understanding.
Other factors—Gender, AI Experience, Extraversion, Agreeableness, Conscientiousness, Neuroticism—showed no significant relation to Engagement, Trust, or Understanding.

Breakthrough Assessment

5/10

While not a technical breakthrough, it provides a valuable 'correction' to the field, warning against the unproductive direction of hyper-personalized XAI based on demographics.

⚙️ Technical Details

Problem Definition

Setting: User study evaluating the impact of independent variables (User Characteristics) on dependent variables (Engagement, Trust, Understanding) in an XAI task.

Inputs: User interactions with an AI toxicity classifier interface providing LIME explanations

Outputs: Measures of Engagement (misclassifications found), Perceived/Actual Understanding, and Trust

Pipeline Flow

Task Interface (Toxic Comment Flagging)
AI Classifier (Detoxify)
Explanation Generator (LIME)
User Feedback Loop (Explanatory Debugging)

System Modules

Task Interface

Present comments, predictions, and explanations; collect user feedback

Model or implementation: React Web App

AI Classifier (Backend Inference)

Predict toxicity labels for comments

Model or implementation: Detoxify (BERT-based transformer)

Explanation Generator (Backend Inference)

Generate local explanations for predictions

Model or implementation: LIME (LimeTextExplainer)

Novel Architectural Elements

Modular prototype architecture using Serverless Cloudflare Worker Functions and MongoDB for scalable user study deployment

Modeling

Base Model: Detoxify (BERT base uncased)

Training Data:

Jigsaw toxic comment dataset (220,000 Wikipedia talk page edits)
Study subset: 100 comments filtered for length and extreme obscenity

Key Hyperparameters:

batch_size: 10 (during original Detoxify training)
loss_function: Binary cross-entropy

Compute: Not reported in the paper

Comparison to Prior Work

vs. Recommender Systems: This paper finds that unlike in recommendations, personality traits (except Openness) do NOT significantly impact XAI effectiveness.
vs. Prior XAI Personalization [Conati et al.]: Challenges the assumption that personalization is necessary, characterizing it as a potential 'rabbit hole'.

Limitations

Study limited to a single task (toxicity detection) and a single explanation type (LIME); results may differ for other domains.
Participants were crowd-sourced (Prolific) and may not represent domain experts.
The paper text provided does not include the detailed regression coefficients or p-values, only the high-level findings.

Reproducibility

The paper states the prototype is released open-source, but the URL is not included in the provided text. The study uses the public Jigsaw dataset and the open-source Detoxify model.

📊 Experiments & Results

Evaluation Setup

Online user study with 149 Prolific participants performing a comment moderation task assisted by AI explanations.

Benchmarks:

User Study (Custom) (Human-Computer Interaction / XAI Evaluation) [New]

Metrics:

User Engagement (finding misclassifications)
Perceived Understanding (Likert scale)
Actual Understanding (12-item mental model score)
Trust (Hoffman et al. scale)
Statistical methodology: Two-tailed regressions (implied by power analysis description, though full results text is missing)

Key Results

Benchmark	Metric	Baseline	This Paper	Δ
Jigsaw Dataset (Validation)	AUC	Not reported in the paper	98	Not reported in the paper

Experiment Figures

The six-step procedure of the user study.

Main Takeaways

User characteristics (Gender, AI Experience, Personality) generally do NOT correlate with Engagement, Trust, or Perceived Understanding in this XAI task.
Only Age and Openness (personality trait) showed a significant relationship with Actual Understanding.
The pursuit of personalized XAI based on fine-grained user demographics may be a 'rabbit hole' (unproductive), suggesting researchers should focus on other design aspects.

📚 Prerequisite Knowledge

Prerequisites

Basic understanding of Explainable AI (XAI) concepts
Familiarity with the 'Big 5' personality traits model
Understanding of local explanation methods like LIME

Key Terms

XAI: Explainable Artificial Intelligence—systems designed to allow humans to understand how AI decisions are made

LIME: Local Interpretable Model-Agnostic Explanations—a technique that explains individual predictions by approximating the complex model locally with a simpler one

Big 5: The five-factor model of personality: Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism

TIPI: Ten-Item Personality Inventory—a short questionnaire used to measure the Big 5 personality traits

Detoxify: A pre-trained multi-class classifier model based on BERT, used to identify toxic comments

SHAP: SHapley Additive exPlanations—a game theoretic approach to explain the output of any machine learning model

Mental Model: An individual's internal representation of how a system works, used to predict system behavior