FEAST: A Flexible Mealtime-Assistance System Towards In-the-Wild Personalization

📝 Paper Summary

Assistive Robotics Personalization (P13N)

FEAST is a modular mealtime-assistance robot that uses a large language model to safely translate user constraints into personalized behavior trees and custom gesture detectors for feeding, drinking, and wiping.

Core Problem

Existing mealtime-assistance robots rely on fixed policies or limited pre-defined customizations, failing to adapt to the diverse, changing needs of care recipients across different social contexts and physical conditions.

Why it matters:

Millions worldwide require feeding assistance due to conditions like spinal cord injuries or cerebral palsy, imposing heavy workloads on caregivers
Eating is deeply social and personal; one-size-fits-all robots can be obtrusive or socially inappropriate (e.g., blocking view of dining companions)
Users' needs fluctuate daily (energy levels) and over time (progressive conditions), requiring systems that can adapt in-the-wild without expert reprogramming

Concrete Example: A user with limited neck mobility needs food placed directly inside their mouth (inside-mouth transfer), while another user prefers leaning forward. In a social setting, one user might want the robot to retract immediately to see their friend, while another needs a custom 'long-open-mouth' gesture because their standard open-mouth gesture triggers falsely during conversation.

Key Novelty

LLM-Mediated Parameterized Behavior Trees for Safety-Critical Personalization

Uses an LLM to translate natural language user requests into structured updates for robot behavior trees, allowing flexibility while maintaining safety via static validation
Enables users to synthesize their own custom head-gesture detectors via LLM code generation (e.g., creating a 'long continuous open mouth' detector)
Integrates modular hardware to support feeding, drinking, and mouth-wiping within a single system, adaptable to diverse physical setups (wheelchair or stand)

Architecture

The FEAST hardware system components and their integration

Evaluation Highlights

Formative study with 21 care recipients identified key personalization tenets: adaptability, transparency, and safety
5-day in-home evaluation with 2 community researchers (users with disabilities) across 3 distinct contexts (personal, TV, social)
Users successfully fed themselves 6 meals each in real-world settings, reporting low cognitive workload on NASA-TLX surveys

Breakthrough Assessment

8/10

Strong contribution to assistive robotics by moving beyond fixed policies to LLM-driven in-the-wild personalization. The extensive user involvement (CBPR) and multi-day in-home testing significantly validate real-world applicability.

⚙️ Technical Details

Problem Definition

Setting: Personalizing a robotic manipulator's behavior trees and perception modules for mealtime tasks (feeding, drinking, wiping) based on user-specific constraints.

Inputs: Natural language commands (voice/text) or interface interactions specifying preferences (e.g., 'retract after bite').

Outputs: Updated parameters for Behavior Trees (BTs) or synthesized code for gesture detection.

Pipeline Flow

User Interface (Web/Voice Input)
LLM Processor (Translation & Code Synthesis)
Validation Layer (Safety Checks)
Robot Control (Behavior Tree Execution)

System Modules

Web Interface

Captures user preferences via buttons, text, or voice; provides transparency summaries

Model or implementation: Web-based UI

LLM Configurator

Translates natural language into behavior tree parameters or synthesizes python code for new gesture detectors

Model or implementation: Large Language Model (Specific model not named in text)

Safety Validator

Statically validates updates to behavior trees to ensuring safety constraints are met before execution

Model or implementation: Rule-based validation

Robot Controller

Executes physical skills (skewering, transfer, wiping) based on the personalized behavior tree

Model or implementation: Parameterized Behavior Trees

Novel Architectural Elements

LLM-in-the-loop reconfiguration of Behavior Trees where the LLM is restricted to parameter updates and specific code synthesis tasks for safety
Modular tool-change apparatus integrated into the control loop allowing dynamic switching between feeding, drinking, and wiping tools

Modeling

Base Model: Large Language Model (Specific variant not explicitly named in provided text)

Compute: Not reported in the paper

Comparison to Prior Work

vs. FLAIR: FEAST extends beyond feeding to drinking/wiping and allows modification of low-level parameters via language, not just high-level sequencing
vs. Obi: FEAST uses autonomous perception and planning rather than fixed playback, allowing adaptation to dynamic environments
vs. Nanavati et al.: FEAST allows open-ended personalization (e.g., custom gestures via code gen) rather than selecting from fixed options

Limitations

Evaluation limited to 2 community researchers for the in-home study (though 21 participated in formative study)
Long-term autonomy without expert supervision is still a goal, not fully achieved (researcher interventions were rare but present)
Reliance on LLMs introduces potential non-determinism, though mitigated by static safety validation

Reproducibility

Code: http://emprise.cs.cornell.edu/feast

All hardware (CAD models, electronics) and software are open-sourced at emprise.cs.cornell.edu/feast. Specific LLM prompt templates or weights are not explicitly detailed in the provided text snippet but implied to be in the release.

📊 Experiments & Results

Evaluation Setup

In-home user study with care recipients and formative interviews

Benchmarks:

Formative Study (User Needs Assessment (N=21)) [New]
In-Home Deployment (Real-world System Evaluation (N=2)) [New]

Metrics:

NASA-TLX (Cognitive Workload)
Technology Acceptance Model (TAM) scores
Qualitative feedback (Independence, Control)
Statistical methodology: Not explicitly reported in the paper

Main Takeaways

Users with severe motor impairments (e.g., C4-C6 Spinal Cord Injury) successfully personalized the system to feed themselves meals in diverse contexts (watching TV, social dining).
The system accommodates diverse needs: one user used standard interactions while another created a custom 'long-continuous-open-mouth' gesture to avoid false triggers during conversation.
Users reported a stronger sense of independence and control compared to human caregiving, noting that conveying preferences to the robot was easier than to a human.
An Occupational Therapist confirmed FEAST outperforms a baseline limited to fixed customizations, validating its ecological validity.

📚 Prerequisite Knowledge

Prerequisites

Behavior Trees (BTs) for robot control
Basics of Large Language Models (LLMs)
Human-Robot Interaction (HRI) principles
Assistive Robotics terminology

Key Terms

ADL: Activities of Daily Living—essential routine tasks like eating, bathing, and dressing

Behavior Tree: A hierarchical model used to control robot decision-making, consisting of nodes that execute tasks, check conditions, or sequence actions

Community Researchers (CRs): End-users (care recipients) who act as co-researchers, actively participating in the design and evaluation process

CBPR: Community-Based Participatory Research—a partnership approach that equitably involves community members (users) in all aspects of the research process

NASA-TLX: NASA Task Load Index—a widely used subjective, multidimensional assessment tool that rates perceived workload

TAM: Technology Acceptance Model—a theory that models how users come to accept and use a technology

LLM: Large Language Model—AI systems trained on vast amounts of text to understand and generate human language (and code)

ISO/TS 13482: A safety standard specifically for personal care robots, outlining requirements for safety during human-robot interaction