Engaging Stakeholders in Scenario-Based Requirements Engineering with Gamification

Abstract

The principles behind gamification are not entirely new. The term originated in 2002, but did not gain popularity until 2010. Since then, gamification has become a major trend in many business domains to increase customer loyalty and employee engagement. In requirements engineering, academics and practitioners are also exploring new opportunities to boost stakeholder participation with the aid of game mechanics and game elements.

The purpose of this thesis is to evaluate the effectiveness of gamification in requirements engineering in order to improve stakeholder engagement. We developed an online digital platform for scenario-based RE supported with gamification. Derived from an in-depth literature study, we selected user stories complemented with scenarios from behavior-driven development (BDD) as a method to express stakeholder requirements. Points, badges and leaderboards (PBL) are very common game elements in terms of gamification and are used as a starting point for the artifact design. In total, the platform consists of 17 different game mechanics and elements, which intension is to positively affect intrinsic and extrinsic stakeholder motivation.

Subsequently, the playful prototype is tested in a controlled experiment. A conceptual framework is constructed in order to measure the effect of gamification on user engagement and performance. The experimental findings expose that it is possible to change stakeholder’s behavior effectively with gamification. Stakeholders who are exposed to the gamified platform produce more user stories, in better quality and with more creative ideas. The majority of their identified requirements are categorized as attractive, which lead to higher customer satisfaction. However, no differences concerning emotions and cognition between the experimental conditions were identified.

In contrast, the control group was likewise involved in the elicitation phase during the trial, but with collaboration as a dominating factor. Their requirements are qualitative rather poorly rated and contain less creative thinking. Furthermore, most user stories cover basic functionalities, which do not cause any positive customer satisfaction when implemented.

The reason for these different behaviors lies in the competitive environment that was simulated for the treatment group. Various game elements are designed to promote rivalries between players and therefore, diminish interpersonal communication. We hypothesize that creativity was inhibited by production blocking that emerged from the intensive discussion within the control group.

Overall, the study of gamification in RE with the utilization of user stories and scenarios as a mean to gather and document requirements yield positive results. Stakeholders interacting with a gamified version of the prototype not only outperform the control group, but are also more satisfied. Derived from the experimental results we agree that competitive game elements are advantageous for RE elicitation to mitigate collaboration and therefore, prevent production blocking. In sharp contrast, social game elements are favorable for RE analysis, specification and validation where cooperation is demanded.