Gated Adaptation for Continual Learning in Human Activity Recognition

📝 Paper Summary

Continual Learning Edge AI / On-device Learning

Gated Adaptation enables stable continual learning on edge devices by modulating frozen pretrained features with lightweight channel-wise gates, restricting updates to diagonal scaling to prevent forgetting.

Core Problem

Human Activity Recognition models on edge devices must adapt to new users (plasticity) without forgetting previous ones (stability), but standard fine-tuning causes catastrophic forgetting.

Why it matters:

Privacy concerns prevent transmitting raw sensor data to the cloud for centralized retraining
Subject-specific variations in movement patterns cause significant domain shifts that degrade model performance when transferring across users
Edge devices have strict memory and compute constraints, making high-capacity replay or architectural expansion methods impractical

Concrete Example: On the PAMAP2 dataset, a standard multilayer perceptron trained sequentially on subjects suffers a 45% accuracy drop (85% to 40%) on the first subject after training on just four subsequent subjects.

Key Novelty

Channel-wise Gated Modulation on Frozen Backbones

Instead of tuning the whole model or adding dense adapter layers, the method inserts lightweight gates after frozen backbone blocks
These gates learn to scale feature channels up or down (diagonal operator) based on the input, adapting the importance of existing features without changing their geometric direction
By freezing the backbone and restricting updates to scaling factors, the method theoretically bounds how much the representation can drift, enforcing stability

Evaluation Highlights

Reduces catastrophic forgetting from 39.7% to 16.2% on PAMAP2 dataset (8 sequential subjects) compared to a trainable backbone
Improves final average accuracy from 56.7% to 77.7% on PAMAP2 compared to a trainable backbone
Requires training less than 2% of total model parameters, making it efficient for resource-constrained IoT devices

Breakthrough Assessment

7/10

Strong practical application of PEFT to the specific constraints of HAR (privacy, edge compute). Theoretical framing of gating as a diagonal operator provides good justification for stability gains.

⚙️ Technical Details

Problem Definition

Setting: Domain-incremental learning where a model learns a sequence of tasks (subjects) with shifting input distributions but fixed label space

Inputs: Time-series sensor window x (normalized, segmented)

Outputs: Activity label y

Pipeline Flow

Data Preprocessing (Normalization, Segmentation)
Frozen Backbone (Feature Extraction)
Gated Adaptation (Feature Modulation)
Classification

System Modules

Backbone Blocks

Extract hierarchical features from sensor data

Model or implementation: Residual Convolutional Network (Frozen)

Gating Modules

Compute channel-wise scaling factors to adapt features to the current subject

Model or implementation: Squeeze-and-Excitation style block (Trainable)

Classifier

Map modulated features to activity classes

Model or implementation: Linear Layer (Trainable)

Novel Architectural Elements

Integration of lightweight trainable gating modules into a frozen backbone specifically for domain-incremental continual learning
Use of gating as a constraint mechanism (diagonal operator) to bound feature drift rather than just for attention

Modeling

Base Model: Residual Convolutional Network (Custom HAR backbone)

Training Method: Continual Fine-tuning of Gates and Classifier only

Objective Functions:

Purpose: Minimize classification error on the current task.

Formally: Standard Cross-Entropy Loss.

Adaptation: Channel-wise Gated Modulation (trainable gates, frozen backbone)

Trainable Parameters: < 2% of total model parameters

Training Data:

Sequential subjects from PAMAP2, UCI-HAR, DSA datasets
No replay buffer used (privacy constraint)

Compute: Suitable for resource-constrained edge devices (low parameter count)

Comparison to Prior Work

vs. Frozen Backbone + Classifier: Adds lightweight gates to recover plasticity without destabilizing the representation
vs. Adapters: Uses multiplicative scaling (diagonal) rather than additive dense transformations, preserving feature geometry [not cited in paper, implied by 'dense transformations']
vs. Replay: Does not require storing sensitive raw sensor data
+ 1 more
vs. Full Fine-tuning: Updates <2% parameters and significantly reduces forgetting

Limitations

Relies on the assumption that diagonal scaling is sufficient for adaptation (may fail under extreme domain shifts)
Requires a well-pretrained backbone to start with
Evaluated only on sensor-based HAR datasets, not larger vision/language benchmarks

Reproducibility

No replication artifacts mentioned in the paper. Code URL not provided. Dataset splits (sequential subjects) are described for PAMAP2, UCI-HAR, and DSA.

📊 Experiments & Results

Evaluation Setup

Domain-incremental learning on sequences of subjects

Benchmarks:

PAMAP2 (Activity Recognition (8 sequential subjects))
UCI-HAR (Activity Recognition)
DSA (Activity Recognition)

Metrics:

Average Accuracy (on all seen tasks)
Forgetting (drop in performance on old tasks)
Statistical methodology: Not explicitly reported in the paper

Key Results

Benchmark	Metric	Baseline	This Paper	Δ
Results on the PAMAP2 dataset (8 subjects) comparing the proposed Gated Adaptation against a fully trainable backbone baseline.
PAMAP2	Forgetting	39.7	16.2	-23.5
PAMAP2	Final Accuracy	56.7	77.7	+21.0

Main Takeaways

Freezing the backbone and training only lightweight gates substantially reduces catastrophic forgetting compared to full fine-tuning
Gated adaptation outperforms minimal adaptation (training only the classifier) by providing necessary plasticity to handle subject-specific variations
The method achieves these gains while updating less than 2% of the parameters, validating its efficiency for edge deployment
Consistent improvements reported across PAMAP2, UCI-HAR, and DSA datasets (though specific numbers for UCI-HAR/DSA not extracted in summary)

📚 Prerequisite Knowledge

Prerequisites

Continual Learning (stability-plasticity dilemma)
Convolutional Neural Networks (CNNs)
Parameter-Efficient Fine-Tuning (PEFT)
Feature modulation / Attention mechanisms

Key Terms

HAR: Human Activity Recognition—classifying physical actions (walking, running) from wearable sensor data

Catastrophic Forgetting: A failure mode where a neural network abruptly loses knowledge of previously learned tasks upon learning new information

Plasticity: The ability of a learning system to acquire knowledge from new data or tasks

Stability: The ability of a learning system to retain previously acquired knowledge while learning new things

Domain-incremental learning: A continual learning scenario where the task definition (labels) remains constant, but the input distribution changes (e.g., different users)

Backbone: The main part of the neural network (usually a CNN) that extracts features from raw input

PEFT: Parameter-Efficient Fine-Tuning—adapting a large pretrained model by updating only a small subset of parameters

Diagonal operator: A linear transformation that scales each dimension independently without mixing dimensions (preserving feature direction)

IMU: Inertial Measurement Unit—a sensor device measuring force, angular rate, and magnetic field (accelerometer, gyroscope)