Thinking inside the Box: The Promise and Boundaries of Transparency in Automated Decision-Making

Growing evidence suggests that human bias cannot be erased in automated decision making, at least for now. It is not clear, who is accountable. This is often referred to as ‘the black box problem’: we cannot be sure how the inputs transform into outputs. Transparency is often proposed as a solution. The call for transparency features in various AI ethics codes as well as in the EU’s GDPR. Although transparency can be approached in many of ways, its basic idea is simple. It promises legitimacy by making an object or behavior visible and, as such, controllable. In my presentation, I argue that transparency cannot solve the black box problem in ADM: transparency is a more complex an ideal that is portrayed in mainstream narratives. Transparency is inherently performative and cannot but be. This performativity goes counter the promise of unmediated visibility, vested in transparency. As I will show, in ADM, transparency’s peculiarities will come visible in a new way.