User studies on real-time facial motion retargeting in online multiplayer games

Abstract

In most online games, communication between players is limited to text messages, voice chat and predefined animations. In this work we look at the utility of real-time face tracking to animate the faces of player avatars, i.e. facial motion retargeting, in online games. We study the effects of tracking accuracy and timing via user evaluations of facial animations with simulated errors. From the results, we derive a scheme for balancing performance parameters to maximize perceived animation quality. We present an implementation of facial motion retargeting in the form of an add-on for the multiplayer game Garry's Mod. Our data gathered from private and public game sessions with the add-on indicates that directly controllable facial expressions encourage player interaction and are useful in both competitive and social gameplay.