Navigating in Uncertainty: developing an evaluation approach for transformative innovation policies and identifying conditions for its institutionalisation

Abstract

Innovation policies are increasingly aimed at addressing grand societal challenges. These Transformative Innovation Policies (TIPs) aim to fundamentally change sociotechnical systems such as energy, transport and food. Because TIPs are generally one of many interventions aiming for transformative change and their impacts appear a long way upstream from their implementation, they pose a substantial evaluation challenge. Current evaluation frameworks are mostly focused on single measures with clear predefined goals and a focus on policy outputs rather than outcomes or final impact, making them unable to capture specific transformative-related evaluation challenges. This study acknowledges the government’s active role in steering transitions and set out to develop an evaluation approach suitable to the evaluation of TIPs and identify necessary conditions for the adoption of this approach by governments. In synthesising transition literature that attributes transition tasks to governments with policy evaluation literature, this research argues that these transition tasks should be adopted as policy objectives and therefore must be subject to evaluation. Moreover, it discusses that it is likely that tensions arise in policy evaluation practices as TIP evaluation asks for learning-based evaluations while policymakers are bound to traditional normative frameworks that favour accountability.

Four consecutive analytical steps were conducted. This research analysed whether and how transition tasks were evaluated in TIP evaluation whereafter these observations were compared with expert perspectives on current evaluation practices to propose an approach that suits the evaluation of transition tasks. An evaluation approach is proposed that evaluates whether conditions are created to successfully execute transition tasks. To do so, the evaluation approach suggests a reflexive process through stakeholder inclusion that is based on a flexible Theory of Change. Discussing challenges in evaluating transition tasks with policy evaluation experts led to the identification of barriers to the adoption and implementation of the proposed approach. Internal government structures are in place that hold civil servants accountable for their actions rather than providing room for learning. Through an illustrative case study, this research finds that current barriers to successfully executing transition tasks can be resolved when the reflexive approach is implemented early in the policy process. Moreover, the case study identified favourable conditions for learning. Several recommendations are made to legitimise learning within governments in an accountability-dominated evaluation culture.