| Summary: | Human oversight has become a key mechanism for the governance of artificial intelligence
(“AI”). Human overseers are supposed to increase the accuracy and safety of AI systems,
uphold human values, and build trust in the technology. Empirical research suggests, however,
that humans are not reliable in fulfilling their oversight tasks. They may be lacking in
competence or be harmfully incentivised. This creates a challenge for human oversight to be
effective. In addressing this challenge, this article aims to make three contributions. First, it
surveys the emerging laws of oversight, most importantly the European Union’s Artificial
Intelligence Act (“AIA”). It will be shown that while the AIA is concerned with the
competence of human overseers, it does not provide much guidance on how to achieve effective
oversight and leaves oversight obligations for AI developers underdefined. Second, this article
presents a novel taxonomy of human oversight roles, differentiated along whether human
intervention is constitutive to, or corrective of a decision made or supported by an AI. The
taxonomy allows to propose suggestions for improving effectiveness tailored to the type of
oversight in question. Third, drawing on scholarship within democratic theory, this article
formulates six normative principles which institutionalise distrust in human oversight of AI.
The institutionalisation of distrust has historically been practiced in democratic governance.
Applied for the first time to AI governance, the principles anticipate the fallibility of human
overseers and seek to mitigate them at the level of institutional design. They aim to directly
increase the trustworthiness of human oversight and to indirectly inspire well-placed trust in AI
governance.
|
|---|