Variability in Multi-tenant Environments: Usability versus flexibility in tenant-dependent reporting

Abstract

Multi-tenant applications enable maximization of economies of scale by offering one shared application and database instance to multiple customers. However, as the shared nature of multi-tenancy makes it impossible to customize by changing source code, tenant-based run-time variability needs to be implemented to satisfy as many customer requirements as possible. Since tenants became responsible for customizing the software, usability is an important aspect to consider when implementing variability. However, there is a lack of well documented techniques on how variability can be implemented and the extent to which the level of variability affects usability is unknown. In order to identify existing techniques for the implementation of variability within the functional area of reporting, a case study on Exact Online – a successful multi-tenant application serving more than 25k customers – was conducted. This resulted in a pattern language constituted by seven variability patterns providing software vendors with reusable solutions for the implementation of tenant-dependent reporting. In order to provide a detailed example on how the Data Web API, REST, Data Middleware, and Abstract Query Builder patterns can be applied, we have implemented a generic query builder web application and a data service within the RESTful OData web API on top of Exact Online. In addition, a usability experiment following the questionsuggestion protocol was performed to test the hypothesized negative relationship between flexibility and usability. Results indicate that usability – in both effectiveness and efficiency – is indeed negatively impacted by flexibility. Software vendors that want to extend their potential customer base by increasing the flexibility of their software should be aware of the adverse consequences for usability as a serious decrease in usability causes customers to cancel their subscription. In order to guide SaaS vendors in achieving an optimal balance between flexibility and usability, we propose a variability implementation approach.