Investigating the perceived value of software traceability in practice

Abstract

Many advances have been made in the research regarding software traceability, regarding the tools, methods, and techniques to create, maintain and use traceability. Nonetheless, even with the well-understood importance of traceability in the scientific software engineering community, according to researchers, traceability is still, “a sought-after, yet often elusive quality in software-intensive systems”, and the realization, awareness, and the need for software traceability varies from context to context and on a project by project basis. Therefore, multiple researchers indicated that there is still much need for empirical knowledge regarding the use of traceability within the typical domains, which include traditional software development methods and safety-critical projects, but also outside these typical domains.

In this study, we present the empirical knowledge gained regarding the perceived value, in terms of the current situation and the needs, of practitioners that use software traceability in practice. First, we investigate common perceptions and problems encountered in practice according to the current literature. Second, we deploy an online survey-questionnaire to gain the perceptions of practitioners on these problems and their needs. Third, we conduct multiple semi-structured interviews with participants of the online survey-questionnaire to gain more knowledge regarding their answers on the survey. Furthermore, the participants and the results of the questionnaire are mainly evaluated and compared based on the different development paradigms in terms of Agile, Traditional, or Mixed software development life cycles, and on the type of project in terms of safety-critical and non-safety critical. The evaluation is done based on the overall sentiment analysis of the participants. In addition, a Wilcoxon analysis is performed to see if there are any significant differences.

55 practitioners participated in the online survey questionnaire. The quantitative Wilcoxon analysis showed that there is one significant difference in the response distribution between the Agile and Traditional groups, in which the participants in the Traditional group agreed more strongly that traceability is mostly performed manually then the Agile group with an effect size margin of 0.94, and a p-value of 0.048 (p < 0.05). The sentiment analysis showed based on the collective results of all the participants that regardless of which development paradigm was followed and project type in terms of safety-critical or not, that similar perceptions and traceability situations are perceived, such as that traceability costs in money, time, and effort are the main inhibitor for adopting (more mature) traceability practices, that traceability is of high importance for the software development process, and that traceability is mostly performed manually. In addition, despite small differences in the rankings, similar needs are perceived as high priority as well, such as that there is a need to perform traceability in a more managed fashion, that there is a need for a more clear overview of the costs and benefits regarding traceability, that there is a need for more traceability automation, that there is a need to increase the awareness of the importance of traceability, and that there is a need for more collaboration and guidance regarding traceability.

These results suggest that the paradigm and the type of project in terms of safety-critical or not, do not significantly affect the software traceability situations and needs of practitioners, which indicates that the perceptions, and problems, and needs, found can be generalized to a certain extent from those perspectives. However, even though the paradigm and project type might not significantly influence the perceptions of practitioners, they might play a more subtle role. In addition, factors, such as the tools used, the tool setup, company size and structure, and more have to be taken into account and potentially play a bigger role in the differences found between traceability usage, perceptions, problems, and needs, found in practice.