Enriching a question-answering system with user experience concepts

Abstract

While question-answering systems are extensively researched, its user experience has not been investigated yet. Therefore, it remains unclear how a question-answering system can be enriched with user experience concepts. ExperienceBank wants to automate question-answering, and while a question-answering system could be utilized to answer frequently asked customer questions automatically, it is unclear how their desired ‘9+ customer experience’ can then still be achieved. Therefore, the present research is conducted at ExperienceBank to clarify how a question-answering system can be enriched with user experience concepts to better handle frequently asked customer questions. A literature review and an empirical investigation have been conducted to explore the body of knowledge on question-answering systems and user experience. This led to theoretical and empirical knowledge that clarified the need for and constituted the foundation for the user experience enrichment guidelines for question-answering systems. These guidelines have been designed by prescribing how to (1) manage user anticipations, (2) address human needs, and (3) measure user opinions, as these areas have been found to be most important in enabling positive user experience. A validation by expert opinions has been conducted to gather qualitative data on the adequacy and shortcomings of the guidelines. This showed that although the guidelines enable enrichment of a question-answering system with useful and effective user experience concepts, they need to be optimized. Several improvements were suggested but the main issue is the lack of instruction on balancing the enrichments, causing a too excessively enriched question-answering system. This is predicted to have a counterproductive effect on its usage and experience. More research on the effectiveness of particular (combinations of) enrichments must be conducted to determine a prioritization of enrichments and eventually an enrichment optimum.