Adaptability in Embodied Conversational Agents: Comparing Rule-Based and Generative Models on Learning and Transfer in Conversational Skills

Abstract

Introduction: Embodied conversational agents (ECAs) are simulation-based technologies used for training professional conversational skills. Two design models with contrasting levels of adaptability are generally implemented: rule-based—with constrained and fixed decision paths—and generative—AI-powered customized responses. Their comparative effectiveness for conversational skills training remains unclear. This study explores the effects of both models on learning, transfer, and self-efficacy.

Method: Participants (n = 72) were randomly assigned to a rule-based or generative condition. To measure learning progress, participants played their (first) assigned simulation type four times. Then, both groups completed a (second) generative simulation once to assess transfer. Self-efficacy was measured before, during, and after the simulations, while perceived task complexity was measured during and after.

Results: Both models facilitated learning progression effectively, however, the generative group demonstrated significantly better transfer—with greater performance gains between the first and second simulations. Rule-based participants rated the second simulation as more complex than the first, while ratings on difficulty for the generative condition did not change between simulations. Self-efficacy remained stable throughout for both groups.

Conclusion: Generative models are apparently more effective for training participants to apply skills in real-world conversations while maintaining self-efficacy and stable complexity perceptions across simulations. Future research should examine learning transfer effects in real-world conversations and psychometric validation of the models' grading systems.