The Disciplinary Power of Algorithms: Domination, Agency and Resistance

Abstract

In our daily lives, we are regularly unaware that we are confronted with algorithmic decision-making systems that use Big Data and machine learning to improve their efficiency and accuracy. Algorithms do not merely increase efficiency; they can also be used to mediate social processes, construct your identity, and can create the opportunity for normalization. However, they can perform these tasks without adequate transparency and accountability. The subjection of the individual to such systems raises the question of whether we are in some way dominated by those systems. Therefore, this thesis aims to answer two critical questions: whether we are dominated by algorithmic decision-making systems, and if we are, what resistance against this domination should look like. Using Foucault, a neo-republican account of freedom as non-domination as used in surveillance studies, and the concept of ‘micro-domination’, I argue that we are indeed dominated by those automated systems, but should extend the scope to the collaborative relationship between the system and the human agents involved. Resistance against this micro-domination should at least (1) uncover the asymmetrical power relations involved in the decision-making process, (2) identify the relevant agents involved, (3) incorporate democratic values and track citizen’s interests to empower them and (4) make the overall system more transparent to the individual. Furthermore, this thesis tests whether the theory of meaningful human control over automated driving systems could satisfy these conditions. However, it shows that it is unable to fully overcome the problem of distribution of responsibility amongst relevant human agents. I introduce the term ‘micro-resistance’ within the context of algorithmic decision-making, which, despite its smaller scale, may have a significant impact on the criticized system when structurally imposed.