Aarhus University Seal / Aarhus Universitets segl

Making Metaethics Work for AI

Michał Klincewicz, Jagiellonian University, Poland

In this paper we examine the practical consequences of design and use of artificial intelligence in light of current state of art in philosophical metaethics. In particular we focus on AI that will be tasked with making explicitly moral decisions, serving as moral advisors, or contributing to morally charged decision making processes. Engineering and design decisions in this domain often come with tacit or explicit metaethical assumptions, including but not limited to: nature of moral judgment, how to characterize moral motivation, the existence of mind-independent moral properties, status of moral epistemology, and what differentiates the moral domain from all others. We analyse the systematic relationship between metaethical positions along the three main dimensions of difference: realism/anti-realism, cognitivism/non-cognitivism, and internalism/externalism and provide a hierarchy of risk and uncertainty associated with each one. We pay special attention to the limitations to human risk perception and assessment, such as the fairness and framing effects, the inverse relationship between risk and benefit, probability neglect, and unrealistic optimism which are well documented in the decision theory and psychological literature. We then offer some recommendations for the kinds of metaethics that should inform AI engineering and algorithm design.