Informing Socially Appropriate Robot Behaviour under Data Distribution Shifts using Continual Learning

Abstract

This thesis addresses the challenge of enabling robots to continuously assess the social appropriateness of their actions in dynamic, changing environments, a setting where conventional machine learning suffers from catastrophic forgetting. To address this, we propose the Explicit Disentanglement DualBranch (EDD) neural network framework, which uses panoptic segmentation to split images into two explicit channels: one isolating social agents (humans, robots, animals) and another isolating environmental context. These channels are processed by parallel MobileNetV2 branches, and their features are fused to predict appropriateness scores for nine potential robot actions in a given scenario. The model incorporates a Naive Rehearsal buffer to mitigate forgetting across sequential exposure to six diverse indoor domains. Evaluated on the OFFICEDB dataset of 2,000 simulated social scenarios, EDD offers significant improvements over FedLGR and Sonar-Pro baselines, achieving an RMSE of 0.777 (vs. 1.832 and 1.038, respectively) and moderate but positive correlation with ground truth annotations (Pearson’s CC 0.285, concordance CC 0.201). Interestingly, EDD’s final performance is statistically equivalent to an oracle trained on all domains simultaneously, demonstrating that continual learning constraints do not degrade predictive accuracy after incremental training. Near elimination of catastrophic forgetting is complemented by strong forward transfer (FWT 0.797), enabling near "zero-shot" generalisation to unseen environments. This work represents, to our knowledge, the first operationalisation of an image-disentanglement architecture within a domain-incremental continual learning framework for social robotics, demonstrating that simple rehearsal mechanisms and heuristic visual masking suffice to maintain robust action appropriateness assessments under non-stationary conditions.