Do we feel what we say: A Multimodal Approach to Emotional and Behavioral Responses in Social Health Campaigns Using Affective AI

Abstract

As social health concerns rapidly escalate worldwide, understanding emotional reactions that encourage behavioral engagement has become increasingly important. Contributing to the effort, this study examined how emotionally distinct social health campaigns influence individuals' emotional responses and behavior-related decisions (e.g., willingness to learn more information). An online survey was conducted with a random sample (N=103) to measure emotional responses via online explicit surveys and facial expression as an implicit indicator. Participants' facial expressions were recorded while viewing positive or somber campaign elements by FaceReader Online (FRO). Self-reported statements measured emotions, ad evaluation, and personal relevance. Results indicated that explicit emotional involvement and campaign effectiveness (i.e., ad evaluation) predicted behavioral engagement toward learning more information about the manner, while facial analysis data did not. This disconnection between internal feelings and observable facial expressions suggests that implicit indicators may not always align with explicit statements. Future studies should further explore this divergence by capturing emotional trajectories and peak values across longer-duration stimuli in emotionally charged health messaging.