Cross-team comparison in agile methods

Abstract

This study investigates which Agile metrics are most effective for tracking team performance and whether they can be applied for performance comparison across different teams. While Agile methodologies are widely used in software development, the effectiveness of associated metrics in providing meaningful performance insights remains debated.

The research followed three phases: a literature review, a quantitative analysis, and a qualitative exploration. The literature review identified key Agile metrics, including cycle time, velocity, and predictability. In the quantitative phase, data from eight Scrum teams was collected and visualized to uncover performance patterns. The qualitative phase involved semi-structured interviews with Product Owners, who evaluated the usefulness of the visualized metrics and highlighted contextual factors that affect cross-team comparability.

Findings indicate that metrics related to predictability and cycle time are particularly useful for monitoring team performance over time. However, direct comparison between teams is limited in value. Variations in outcomes often result from contextual differences such as team maturity, composition, the role of the Scrum Master, product complexity, and organizational influences like release cycles and DevOps responsibilities. Furthermore, the reliability of metrics depends on consistent data collection and disciplined use of tooling.

The conclusion is that Agile metrics are effective for identifying trends and improvements within teams but offer only constrained value for cross-team comparison. Meaningful comparisons require standardized measurement practices and careful consideration of contextual variables to avoid misinterpretation.