Multimodal Immersive Systems for Assembly in Augmented Reality

Abstract

This paper explores the integration of Extended Reality and multimodal Artificial Intelligence (AI) in the context of assembly tasks. The study investigates the use of dialogue agents within Mixed Reality (MR) environments to enhance assembly processes, focusing on whether training can be simplified and costs can be reduced by replacing the human trainer. Methodologically, the research includes a use case analysis, feasibility study, and system implementation, followed by a user study evaluating variables such as task performance and user satisfaction. Results indicate that integrating dialogue agents with MR systems may potentially improve assembly efficiency and user interaction, although challenges in technical development and user acceptance remain. The proposed method provides a good starting point for understanding the potential of AI-driven MR applications in training and operational settings, suggesting a course of action for further research and development.