Creation of a HoloLens Application To Support Instructors During Pilot Simulation Training

Abstract

This study investigates the product development of a HoloLens-application which enables instructors to see pilots’ gaze behaviour in real-time. Gaze behaviour is an important aspect of flying and can be difficult to learn, especially for pilots in training for their first type rating when performing an approach and landing simulation. However, the current training instruments lack objective information about the pilots’ gaze behaviour. Instructors can base their feedback only on the pilots’ movements, the results in the simulation, and by asking about flight parameters to estimate their gaze behaviour. Product development followed several iterations, in which instructors of a major Dutch airline were involved. The results of a short online questionnaire (N = 20) confirmed the claims in literature that instructors desire a tool to see the pilots’ gazes in real-time, with an optional playback during the debriefing. The pilot experiment (N = 5) yielded similar results and additional advice to enhance the tool. Answers on a common, validated questionnaire indicated a positive user experience, with ‘above average’ to ‘good’ evaluations of the application. Results from the newly constructed questionnaire regarding the experienced support for feedback, indicated that the instructors felt supported by the tool. All five instructors would implement a similar tool in their training and recommend it to others. The pilots did not have any problems with the application. Many training situations may benefit from this application. The tool needs further development and should be tested on a larger scale to prove its value.