Evaluating the usability of data preparation tools for self-service business intelligence users

Abstract

Within the field of business intelligence, casual users depend on power users to create reports to make data-driven decisions. This is due to their low technical skill, which can be defined by the tools and technologies they use. This research sought to determine which usability problems casual users face in Trifacta Wrangler when executing three data structuring tasks that are required for report creation. Furthermore, the aim was to identify how technical skill influences the usability problems they face. A usability test was conducted with 8 participants working in Sales, Marketing, and Client Services at a dutch marketing automation company. Participants were asked to fill in a survey inquiring about their technical skill and to fill in the System Usability Scale rating their interaction. As a result of categorizing identified usability problems according to the User Action Framework, it was found that most usability problems belonged to the planning and translation phase of the interaction cycle. Contrary to previous research, participants’ Excel skill influenced their capability to plan interactions negatively. The System Usability Scale revealed that this might be related to learnability as a usability criterium. In line with previous research, one participant with prior SQL experience recovered from the most severe planning issue due to their knowledge of programming concepts. It can be said that to improve the self-service level of casual users, understanding their planning of data structuring tasks is crucial. Further research is needed to verify these findings by identifying tools and technologies used by a larger sample of casual users and having them perform data structuring tasks in various market-leading data preparation tools.