Automatic Analysis of Synchrony in Dyadic Interviews

Abstract

Nonverbal synchrony has received a great deal of attention from many different scientific areas for its relatedness to the quality of interaction and interpersonal relationships, functions in early infancy, and ability to be used as a predictor for variables such as therapy outcome. This motivates a need for automated synchrony analysis in order to exclude the possibility of human error and subjectivity. In this study the different methodologies used to extract movement data from video, as well as the methodologies to measure synchrony in movement data have been investigated. The goal of this study is to find the methodologies and settings that allow for the best quantification of synchrony in dyads. Synchrony is operationalized as the ability to distinguish rapport-building trained interviewers from interviewers that did not receive this training. For motion energy time series creation OpenPose and motion energy analysis (MEA) have been compared. Using the motion energy time series generated by MEA, the ability to measure synchrony of windowed cross-lagged correlation (WCLC), windowed cross-lagged regression (WCLR) and recurrence quantification analysis (RQA) have been investigated. The parameters of each of these methods have been tweaked to investigate their influence on the output score and find optimal values. The results show that MEA provides the best motion energy time series and that WCLR most accurately quantifies synchrony. Furthermore, the results show that the output score of WCLR is not robust against frame skip, therefore frame skip should not be used.