Personalization of Affect Recognition Models Using Biosignals and Personality

Abstract

Person-specific data for affect recognition models is scarce, and generalized models often struggle with accuracy due to individual differences. Cluster-based personalization has been proposed as a compromise between generalized and fully personalized affect recognition from biosignals. This study investigated whether grouping participants by Big-Five personality profiles can improve user-independent prediction of continuous valence and arousal ratings. Cardiac and electrodermal features were extracted from two public datasets, AMIGOS and PhyMER. The datasets consisted of both short recordings (50-250 seconds) and long recordings (14-24 minutes). Three variants were evaluated: Baseline, which used only biosignals; Clusters, which added personality-based cluster assignments; and Traits, which added personality scores directly. Each variant was implemented with the Random Forests (RF) and Support Vector Regression (SVR) algorithms. As anticipated, the Baseline models across datasets struggled with large individual differences, resulting in limited explained variance of valence and arousal levels. However, informing models of personality-based subgroups did not improve the generalization of the models. Neither personality representation produced notable or systematic improvements over the Baseline. Generally, models trained on short segments exhibited improved performance compared to those trained on long segments. Furthermore, our results confirm that recognizing continuous valence and arousal levels of unseen users is a nontrivial task, and individual variability likely diffuses the relationships between biosignals and affective labels. Future work should focus on improving the understanding of the physiological and psychological processes underlying self-reported affective states, using larger datasets and more advanced modeling techniques, such as Long Short-Term Memory.