In this paper, in line with the general framework of value-sensitive design, we aim to operationalize the general concept of “Meaningful Human Control” in order to pave the way for its translation into more specific design requirements. In particular, we focus on the operationalization of the first of the two conditions investigated: the so-called ‘tracking’ condition. Our investigation is led in relation to one specific subcase of automated system: dual-mode driving systems. First, we connect and compare (...) class='Hi'>meaningful human control with a concept of control very popular in engineering and traffic psychology, and we explain to what extent tracking resembles and differs from it. This will help clarifying the extent to which the idea of meaningful human control is connected to, but also goes beyond, current notions of control in engineering and psychology. Second, we take the systematic analysis of practical reasoning as traditionally presented in the philosophy of human action and we adapt it to offer a general framework where different types of reasons and agents are identified according to their relation to an automated system’s behaviour. This framework is meant to help explaining what reasons and what agents play a role in controlling a given system, thereby enabling policy makers to produce usable guidelines and engineers to design systems that properly respond to selected human reasons. In the final part, we discuss a practical example of how our framework could be employed in designing automated driving systems. (shrink)

Artificial intelligence is currently changing many areas of society. Especially in health, where critical decisions are made, questions of control must be renegotiated: who is in control when an automated system makes clinically relevant decisions? Increasingly, the concept of meaningful human control (MHC) is being invoked for this purpose. However, it is unclear exactly how this concept is to be understood in health. Through a systematic review, we present the current state of the concept of (...) MHC in health. The results show that there is not yet a robust MHC concept for health. We propose a broader understanding of MHC along three strands of action: enabling, exercising and evaluating control. Taking into account these strands of action and the established rules and processes in the different health sectors, the MHC concept needs to be further developed to avoid falling into two gaps, which we have described as theoretical and labelling gaps. (shrink)

The last decade has witnessed the mass distribution and adoption of smart home systems and devices powered by artificial intelligence systems ranging from household appliances like fridges and toasters to more background systems such as air and water quality controllers. The pervasiveness of these sociotechnical systems makes analyzing their ethical implications necessary during the design phases of these devices to ensure not only sociotechnical resilience, but to design them for human values in mind and thus preserve meaningful (...) class='Hi'>human control over them. This paper engages in a conceptual investigations of how meaningful human control over smart home devices can be attained through design. The value sensitive design (VSD) approach is proposed as a way of attaining this level of control. In the proposed framework, values are identified and defined, stakeholder groups are investigated and brought into the design process and the technical constraints of the technologies in question are considered. The paper concludes with some initial examples that illustrate a more adoptable way forward for both ethicists and engineers of smart home devices. (shrink)

”Meaningful human control” is a term invented in the political and legal debate on autonomous weapons system, but it is nowadays also used in many other contexts. It is supposed to specify conditions under which an artificial system is under the right kind of control to avoid responsibility gaps: that is, situations in which no moral agent is responsible. Santoni de Sio and Van den Hoven have recently suggested a framework that can be used by system (...) designers to operationalize this kind of control. It is the purpose of this paper to facilitate further operationalization of ”meaningful human control”. -/- This paper consists of two parts. In the first part I resolve an ambiguity that plagues current operationalizations of MHC. One of the design conditions says that the system should track the reasons of the relevant agents. This condition is ambiguous between the kind of reasons involved. On one interpretation it says that a system should track motivating reasons, while it is concerned with normative reasons on the other. Current participants in the debate interpret the framework as being concerned with (something in the vicinity of) motivating reasons. I argue against this interpretation by showing that meaningful human control requires that a system tracks normative reasons. Moreover, I maintain that an operationalization of meaningful human control that fails to track the right kind of reasons is morally problematic.When this is properly understood, it can be shown that the framework of MHC is committed to the agent-relativity of reasons. More precisely, I argue in the second part of this paper that if the tracking condition of MHC plays an important role in responsibility attribution (as the proponents of the view maintain), then the framework is incompatible with first-order normative theories that hold that normative reasons are agent-neutral (such as many versions of consequentialism). In the final section I present three ways forward for the proponent of MHC as reason-responsiveness. (shrink)

Meaningful human control over AI is exalted as a key tool for assuring safety, dignity, and responsibility for AI and automated decision-systems. It is a central topic especially in fields that deal with the use of AI for decisions that could cause significant harm, like AI-enabled weapons systems. This paper argues that discussions regarding meaningful human control commonly fail to identify the purpose behind the call for meaningful human control and that (...) stating that purpose is a necessary step in deciding how best to institutionalize meaningful human control over AI. The paper identifies 5 common purposes for human control and sketches how different purpose translate into different institutional design. (shrink)

The paper presents a framework to realise “meaningful human control” over Automated Driving Systems. The framework is based on an original synthesis of the results of the multidisciplinary research project “Meaningful Human Control over Automated Driving Systems” lead by a team of engineers, philosophers, and psychologists at Delft University of the Technology from 2017 to 2021. Meaningful human control aims at protecting safety and reducing responsibility gaps. The framework is based on (...) the core assumption that human persons and institutions, not hardware and software and their algorithms, should remain ultimately—though not necessarily directly—in control of, and thus morally responsible for, the potentially dangerous operation of driving in mixed traffic. We propose an Automated Driving System to be under meaningful human control if it behaves according to the relevant reasons of the relevant human actors (tracking), and that any potentially dangerous event can be related to a human actor (tracing). We operationalise the requirements for meaningful human control through multidisciplinary work in philosophy, behavioural psychology and traffic engineering. The tracking condition is operationalised via a proximal scale of reasons and the tracing condition via an evaluation cascade table. We review the implications and requirements for the behaviour and skills of human actors, in particular related to supervisory control and driver education. We show how the evaluation cascade table can be applied in concrete engineering use cases in combination with the definition of core components to expose deficiencies in traceability, thereby avoiding so-called responsibility gaps. Future research directions are proposed to expand the philosophical framework and use cases, supervisory control and driver education, real-world pilots and institutional embedding. (shrink)

The international debate on the ethics and legality of autonomous weapon systems (AWS), along with the call for a ban, primarily focus on the nebulous concept of fully autonomous AWS. These are AWS capable of target selection and engagement absent human supervision or control. This paper argues that such a conception of autonomy is divorced from both military planning and decision-making operations; it also ignores the design requirements that govern AWS engineering and the subsequent tracking and tracing of (...) moral responsibility. To show how military operations can be coupled with design ethics, this paper marries two different kinds of meaningful human control (MHC) termed levels of abstraction. Under this two-tiered understanding of MHC, the contentious notion of ‘full’ autonomy becomes unproblematic. (shrink)

When the First World War broke out in August 1914, most believed that it would rapidly be over – hopefully even by Christmas. It lasted for more than four years, with untold destruction and sufferi...

Rapid developments in Artificial Intelligence are leading to an increasing human reliance on machine decision making. Even in collaborative efforts with Decision Support Systems (DSSs), where a human expert is expected to make the final decisions, it can be hard to keep the expert actively involved throughout the decision process. DSSs suggest their own solutions and thus invite passive decision making. To keep humans actively ‘on’ the decision-making loop and counter overreliance on machines, we propose a ‘reflection machine’ (...) (RM). This system asks users questions about their decision strategy and thereby prompts them to evaluate their own decisions critically. We discuss what forms RMs can take and present a proof-of-concept implementation of a RM that can produce feedback on users’ decisions in the medical and law domains. We show that the prototype requires very little domain knowledge to create reasonably intelligent critiquing questions. With this prototype, we demonstrate the technical feasibility to develop RMs and hope to pave the way for future research into their effectiveness and value. (shrink)

Decisions in security contexts, including armed conflict, law enforcement, and disaster relief, often need to be taken under circumstances of limited information, stress, and time pressure. Since AI systems are capable of providing a certain amount of relief in such contexts, such systems will become increasingly important, be it as decision-support or decision-making systems. However, given that human life may be at stake in such situations, moral responsibility for such decisions should remain with humans. Hence the idea of “ (...) class='Hi'>meaningful human control” of intelligent systems. In this opinion paper, we outline generic configurations of control of AI and we present an alternative to human control of AI, namely the inverse idea of having AI control humans, and we discuss the normative consequences of this alternative. (shrink)

The international debate on the ethics and legality of autonomous weapon systems (AWS) as well as the call for a ban are primarily focused on the nebulous concept of fully autonomous AWS. More specifically, on AWS that are capable of target selection and engagement without human supervision or control. This thesis argues that such a conception of autonomy is divorced both from military planning and decision-making operations as well as the design requirements that govern AWS engineering and subsequently (...) the tracking and tracing of moral responsibility. To do this, this thesis marries two different levels of meaningful human control (MHC), termed levels of abstraction, to couple military operations with design ethics. In doing so, this thesis argues that the contentious notion of ‘full’ autonomy is not problematic under this two-tiered understanding of MHC. It proceeds to propose the value sensitive design (VSD) approach as a means for designing for MHC. (shrink)

The urban traffic environment is characterized by the presence of a highly differentiated pool of users, including vulnerable ones. This makes vehicle automation particularly difficult to implement, as a safe coordination among those users is hard to achieve in such an open scenario. Different strategies have been proposed to address these coordination issues, but all of them have been found to be costly for they negatively affect a range of human values (e.g. safety, democracy, accountability…). In this paper, we (...) claim that the negative value impacts entailed by each of these strategies can be interpreted as lack of what we call Meaningful Human Control over different parts of a sociotechnical system. We argue that Meaningful Human Control theory provides the conceptual tools to reduce those unwanted consequences, and show how “designing for meaningful human control” constitutes a valid strategy to address coordination issues. Furthermore, we showcase a possible application of this framework in a highly dynamic urban scenario, aiming to safeguard important values such as safety, democracy, individual autonomy, and accountability. Our meaningful human control framework offers a perspective on coordination issues that allows to keep human actors in control while minimizing the active, operational role of the drivers. This approach makes ultimately possible to promote a safe and responsible transition to full automation. (shrink)

This paper explores the role and resolution of disagreements between physicians and their diagnostic AI-based decision support systems. With an ever-growing number of applications for these independently operating diagnostic tools, it becomes less and less clear what a physician ought to do in case their diagnosis is in faultless conflict with the results of the DSS. The consequences of such uncertainty can ultimately lead to effects detrimental to the intended purpose of such machines, e.g. by shifting the burden of proof (...) towards a physician. Thus, we require normative clarity for integrating these machines without affecting established, trusted, and relied upon workflows. In reconstructing different causes of conflicts between physicians and their AI-based tools—inspired by the approach of “meaningful human control” over autonomous systems and the challenges to resolve them—we will delineate normative conditions for “meaningful disagreements”. These incorporate the potential of DSS to take on more tasks and outline how the moral responsibility of a physician can be preserved in an increasingly automated clinical work environment. (shrink)

Autonomous weapons systems, often referred to as ‘killer robots’, have been a hallmark of popular imagination for decades. However, with the inexorable advance of artificial intelligence systems (AI) and robotics, killer robots are quickly becoming a reality. These lethal technologies can learn, adapt, and potentially make life and death decisions on the battlefield with little-to-no human involvement. This naturally leads to not only legal but ethical concerns as to whether we can meaningful control such machines, and if (...) so, then how. Such concerns are made even more poignant by the ever-present fear that something may go wrong, and the machine may carry out some action(s) violating the ethics or laws of war. -/- Researchers, policymakers, and designers are caught in the quagmire of how to approach these highly controversial systems and to figure out what exactly it means to have meaningful human control over them, if at all. -/- In Designed for Death, Dr Steven Umbrello aims to not only produce a realistic but also an optimistic guide for how, with human values in mind, we can begin to design killer robots. Drawing on the value sensitive design (VSD) approach to technology innovation, Umbrello argues that context is king and that a middle path for designing killer robots is possible if we consider both ethics and design as fundamentally linked. Umbrello moves beyond the binary debates of whether or not to prohibit killer robots and instead offers a more nuanced perspective of which types of killer robots may be both legally and ethically acceptable, when they would be acceptable, and how to design for them. (shrink)

As automating technologies are increasingly integrated into workplaces, one concern is that many of the human workers who remain will be relegated to more dull and less positively impactful work. This paper considers two rival theories of meaningful work that might be used to evaluate particular implementations of automation. The first is achievementism, which says that work that culminates in achievements to workers’ credit is especially meaningful; the other is the practice view, which says that work that (...) takes the form of an open-ended practice is especially meaningful. Of the two, the practice view is the better tool for assessing the future of meaningful work, because achievementism is explanatorily inadequate in two ways. Moreover, the practice view can explain why the most meaningful forms of work cannot be automated. A procedure can only be automated as long as the steps are defined, whereas, in such work, which steps to take is indefinitely open to redefinition. This reveals the real threat to meaningful work to be a political-economic one. The relevant ethical questions there have to do with how much creative control workers retain in crafting their own jobs, when those jobs involve meaningful work in collaboration with machines; and how to liberate workers from jobs that do not involve meaningful work, which should be left to machines anyway. (shrink)

Technological outsourcing is increasingly prevalent, with AI systems taking over many tasks once performed by humans. This shift has led to various discussions within AI ethics. A question that was largely ignored until recently, but is now increasingly being discussed, concerns the meaningfulness of such a lifestyle. The literature largely features skeptical views, raising several challenges. Many of these challenges can be grouped under what I identify as the “meaningfulness gap”. Although this gap is widely acknowledged, there is a notable (...) absence of systematic exploration in the literature. This paper aims to fill this void by offering a detailed, step-by-step guide for systematically exploring the different instances of the meaningfulness gap and aids in navigating their complexities. More specifically, it proposes differentiating the gaps according to their realms and objects, normative nature, scope, and severity. To make these areas manageable, the paper takes several taxonomies and distinctions on board. Finally, the guide is summarized, and some skeptical replies are anticipated and countered by clarificatory remarks. (shrink)

Human decisions are increasingly supported by decision support systems (DSS). Humans are required to remain “on the loop,” by monitoring and approving/rejecting machine recommendations. However, use of DSS can lead to overreliance on machines, reducing human oversight. This paper proposes “reflection machines” (RM) to increase meaningful human control. An RM provides a medical expert not with suggestions for a decision, but with questions that stimulate reflection about decisions. It can refer to data points or suggest (...) counterarguments that are less compatible with the planned decision. RMs think against the proposed decision in order to increase human resistance against automation complacency. Building on preliminary research, this paper will (1) make a case for deriving a set of design requirements for RMs from EU regulations, (2) suggest a way how RMs could support decision-making, (3) describe the possibility of how a prototype of an RM could apply to the medical domain of chronic low back pain, and (4) highlight the importance of exploring an RM’s functionality and the experiences of users working with it. (shrink)

Airline pilots are attributed ultimate responsibility and final authority over their aircraft to ensure the safety and well-being of all its occupants. Yet, with the advent of automation technologies, a dissonance has emerged in that pilots have lost their actual decision-making authority as well as their ability to act in an adequate fashion towards meeting their responsibilities when unexpected circumstances or emergencies occur. Across the literature in human factor studies, we show how automated algorithmic technologies have wrestled control (...) away from airline pilots. The inter-related ethical and pragmatic consequences resulting from such a dissonance are presented. This includes pilots serving as ‘legal sponges’, being caught in ‘moral crumple zones’, as well as real impacts on safety and well-being of passengers and crew. One potential avenue presented in the literature involves a redistribution of responsibility towards all agential entities within the cockpit system. We discuss the possible pitfalls to this approach, including the reinforcement of pilot skills erosion as well as technology’s own limitations in adaptively dealing with unexpected circumstances. We justify our position by showing how algorithmic technology’s representational frames mask or severely limit the pilots’ natural reflex to ‘practically cope’ and gain an ‘optimal grip’ with a given context/situation at hand. Here, we present the concept of relevant or meaningful control, which pilots, when allowed to develop their natural brain-body reflexes, gain across gradual mastery of the aircraft. Finally, we propose a socio-technical design approach in which pilots can re-appropriate full human meaningful control of the aircraft, thus allowing them to meet their attributed responsibilities. (shrink)

Autonomous weapons systems (AWS), sometimes referred to as “killer robots”, are receiving evermore attention, both in public discourse as well as by scholars and policymakers. Much of this interest is connected with emerging ethical and legal problems linked to increasing autonomy in weapons systems, but there is a general underappreciation for the ways in which existing law might impact on these new technologies. In this paper, we argue that as AWS become more sophisticated and increasingly more capable than flesh-and-blood soldiers, (...) it will increasingly be the case that such soldiers are “in the power” of those AWS which fight against them. This implies that such soldiers ought to be considered hors de combat, and not targeted. In arguing for this point, we draw out a broader conclusion regarding hors de combat status, namely that it must be viewed contextually, with close reference to the capabilities of combatants on both sides of any discreet engagement. Given this point, and the fact that AWS may come in many shapes and sizes, and can be made for many different missions, we argue that each particular AWS will likely need its own standard for when enemy soldiers are deemed hors de combat. We conclude by examining how these nuanced views of hors de combat status might impact on meaningful human control of AWS. (shrink)

An authoethnography explores the lived experiences of patients being in control and self-managing their chronic illness among their families and friends. Findings show that the current health discourse narrows down people to mere patients and gives rise to tensions. This article indicates that people with one or several chronic illnesses or disabilities are first of all full citizens with needs, values, and drives seeking a meaningful life. Fair possibilities ought to exist to satisfy their needs to belong, to (...) care for and to matter to others since these constitute an essential aspect of being human. (shrink)

This paper demonstrates that with increased philosophical scrutiny regarding the absurdity of life and meaningful living, the development of “Emotionalized AI” falls under ethical considerations. In this context, contemporary philosophical discussions on the meaning of life—as articulated by thinkers like Thomas Nagel, Joshua Seachris, Thaddeus Metz, and Susan Wolf—intersect with recent years’ reviews of AI ethics, particularly those related to meaningful relationships. The main analysis is conducted by explaining a philosophical perspective on meaningful life and revisiting AI (...) ethics reviews based on it while accepting a superficial understanding of “Emotionalized AI” to avoid metaphysical disputes. In this regard, the questions raised about AI ethics, considering the specialized literature in recent years, are first addressed. Similarly, the discussion focuses on a specific type of ethical concern regarding AI development: human relationships. It is then explained how “love” and, more generally, emotional relationships can be an important resource of life’s meaningfulness. Subsequently, an argument is presented showing that the development of emotionalized AI, by limiting our access to “emotional relationships,” poses a threat to life’s meaningfulness and is therefore ethically unjustifiable. To reinforce this view, I also use some near-reality dystopian scenarios for the AI era. Finally, I re-evaluate the conclusion based on a four-step process for assessing the meaningfulness gap created by emotionalized AI. Applying this process also confirms the result. (shrink)

Accountability and responsibility are key concepts in the academic and societal debate on Autonomous Weapon Systems, but these notions are often used as high-level overarching constructs and are not operationalised to be useful in practice. “Meaningful Human Control” is often mentioned as a requirement for the deployment of Autonomous Weapon Systems, but a common definition of what this notion means in practice, and a clear understanding of its relation with responsibility and accountability is also lacking. In this (...) paper, we present a definition of these concepts and describe the relations between accountability, responsibility, control and oversight in order to show how these notions are distinct but also connected. We focus on accountability as a particular form of responsibility—the obligation to explain one’s action to a forum—and we present three ways in which the introduction of Autonomous Weapon Systems may create “accountability gaps”. We propose a Framework for Comprehensive Human Oversight based on an engineering, socio-technical and governance perspective on control. Our main claim is that combining the control mechanisms at technical, socio-technical and governance levels will lead to comprehensive human oversight over Autonomous Weapon Systems which may ensure solid controllability and accountability for the behaviour of Autonomous Weapon Systems. Finally, we give an overview of the military control instruments that are currently used in the Netherlands and show the applicability of the comprehensive human oversight Framework to Autonomous Weapon Systems. Our analysis reveals two main gaps in the current control mechanisms as applied to Autonomous Weapon Systems. We have identified three first options as future work for the design of a control mechanism, one in the technological layer, one in the socio-technical layer and one the governance layer, in order to achieve comprehensive human oversight and ensure accountability over Autonomous Weapon Systems. (shrink)

Artificial agents create significant moral opportunities and challenges. Over the last two decades, discourse has largely focused on the concept of a ‘responsibility gap.’ We argue that this concept is incoherent, misguided, and diverts attention from the core issue of ‘control gaps.’ Control gaps arise when there is a discrepancy between the causal control an agent exercises and the moral control it should possess or emulate. Such gaps present moral risks, often leading to harm or ethical (...) violations. We propose a second-order ‘duty of moral control’ that mandates closing these gaps to reduce risks within acceptable moral limits. Our analysis encompasses both autonomous machines and collective agents, acknowledging their similarities and key differences in constitution and moral status. We suggest four methods to close control gaps: ensuring artificial agents attain moral agency, providing meaningful human control, implementing safety engineering, and employing social control. These methods aim to responsibly integrate artificial agents into society. We conclude that a realistic approach, which addresses the practical problems posed by control gaps, is essential. This approach provides solutions to manage the risks posed by artificial agents while maintaining acceptable moral standards, ensuring we responsibly harness their potential and address the ethical challenges they present. (shrink)

John Danaher and Sven Nyholm have argued that automation, especially of the sort powered by artificial intelligence, poses a threat to meaningful work by diminishing the chances for meaning-conferring workplace achievement, what they call “achievement gaps”. In this paper, I argue that Danaher and Nyholm’s achievement gap thesis suffers from an ambiguity. The weak version of the thesis holds that automation may result in the appearance of achievement gaps, whereas the strong version holds that automation may result on balance (...) loss in possibilities for workplace achievements, i.e., in the appearance of an overall gappier work landscape. Against the strong version of the achievement gap thesis, I characterize situations where automation may result in boons to meaning-conferring workplace achievements: the appearance of what I call achievement spread and achievement swaps. Nevertheless, Danaher and Nyholm are right to worry about some uses of automation whereby human workers become subservient to AI. But these situations are better framed, I argue, as autonomy gaps rather than achievement gaps. (shrink)

Meaningful human control (MHC) is increasingly becoming an important topic in AI ethics beyond the domain of autonomous weapons systems. MHC has been conceptualized, analyzed, and applied. However, in this article, I show how all the current attempts at realizing MHC have fallen short because we have not taken the important first step of deciding what machines should and should not be doing in the first place. We must first ensure that the output we have delegated to (...) the machine is appropriate – only then do we have to do the work required to ensure that MHC is realized. Here, I argue that machines should not be evaluating – that is, we should not be delegating evaluative outputs to machines. This is practically important because machines that have evaluative outputs cannot be evaluated for efficacy. We can’t say how effective they are. This is ethically important because machines don’t have the moral agency required to make evaluations. Furthermore, machines should not be in a position to change our values – which they would be if they were evaluating. Finally, evaluations are judged based on the considerations used to reach a particular judgment. Contemporary AI cannot provide these justifying considerations – so we have no way of evaluating their evaluative outputs. (shrink)

Due to their self-learning and evolutionary character, AI (Artificial Intelligence) systems are more prone to unintended consequences and more difficult to control than traditional sociotechnical systems. To deal with this, machine ethicists have proposed to build moral (reasoning) capacities into AI systems by designing artificial moral agents. I argue that this may well lead to more, rather than less, unintended consequences and may decrease, rather than increase, human control over such systems. Instead, I suggest, we should bring (...) AI systems under meaningful human control by formulating a number of meta-values for their evolution. Amongst others, this requires responsible experimentation with AI systems, which may neither guarantee full control nor the prevention of all undesirable consequences, but nevertheless ensures that AI systems, and their evolution, do not get out of control. (shrink)

The increasing complexity and ubiquity of autonomously operating artificially intelligent (AI) systems call for a robust theoretical reconceptualization of responsibility and control. The Meaningful Human Control (MHC) approach to the design and operation of AI systems provides such a framework. However, in its focus on accountability and minimizing harms, it neglects how we may flourish in interaction with such systems. In this chapter, I show how the MHC framework can be expanded to meet this challenge by (...) drawing on the ethics of carebots and embodied design. First, I examine how discussions about flourishing with carebots invite us to consider the extent to which we control our moral character Second, I argue that we can understand the cultivation of moral character in terms of embodied virtues arising from operating in particular ecologies. Third, I demonstrate how this analysis fruitfully informs the design of carebots as supporting reciprocity and empathy. (shrink)

John Danaher and Sven Nyholm have argued that automation, especially of the sort powered by artificial intelligence, poses a threat to meaningful work by diminishing the chances for meaning-conferring workplace achievement, what they call “achievement gaps”. In this paper, I argue that Danaher and Nyholm’s achievement gap thesis suffers from an ambiguity. The weak version of the thesis holds that automation may result in the appearance of achievement gaps, whereas the strong version holds that automation may result on balance (...) loss in possibilities for workplace achievements, i.e., in the appearance of an overall gappier work landscape. Against the strong version of the achievement gap thesis, I characterize situations where automation may result in boons to meaning-conferring workplace achievements: the appearance of what I call achievement spread and achievement swaps. Nevertheless, Danaher and Nyholm are right to worry about some uses of automation whereby human workers become subservient to AI. But these situations are better framed, I argue, as autonomy gaps rather than achievement gaps. (shrink)

Within the ethical debate on Machine Learning-driven decision support systems (ML_CDSS), notions such as “human in the loop” or “meaningful human control” are often cited as being necessary for ethical legitimacy. In addition, ethical principles usually serve as the major point of reference in ethical guidance documents, stating that conflicts between principles need to be weighed and balanced against each other. Starting from a neo-Kantian viewpoint inspired by Onora O'Neill, this article makes a concrete suggestion of (...) how to interpret the role of the “human in the loop” and to overcome the perspective of rivaling ethical principles in the evaluation of AI in health care. We argue that patients should be perceived as “fellow workers” and epistemic partners in the interpretation of ML_CDSS outputs. We further highlight that a meaningful process of integrating (rather than weighing and balancing) ethical principles is most appropriate in the evaluation of medical AI. (shrink)

One of the several reasons given in calls for the prohibition of autonomous weapons systems (AWS) is that they are against human dignity (Asaro, 2012; Docherty, 2014; Heyns, 2017; Ulgen, 2016). However there have been criticisms of the reliance on human dignity in arguments against AWS (Birnbacher, 2016; Pop, 2018; Saxton, 2016). This paper critically examines the relationship between human dignity and autonomous weapons systems. Three main types of objection to AWS are identified; (i) arguments based on (...) technology and the ability of AWS to conform to International Humanitarian Law; (ii) deontological arguments based on the need for human judgement and meaningful human control, including arguments based on human dignity; (iii) consequentialist reasons about their effects on global stability and the likelihood of going to war. An account is provided of the claims made about human dignity and AWS, of the criticisms of these claims, and of the several meanings of ‘dignity’. It is concluded that although there are several ways in which AWS can be said to be against human dignity, they are not unique in this respect. There are other weapons, and other technologies, that also compromise human dignity. Given this, and the ambiguities inherent in the concept, it is wiser to draw on several types of objections in arguments against AWS, and not to rely exclusively on human dignity. (shrink)

Can AI solve the ethical, moral, and political dilemmas of warfare? How is artificial intelligence (AI)-enabled warfare changing the way we think about the ethical-political dilemmas and practice of war? This article explores the key elements of the ethical, moral, and political dilemmas of human-machine interactions in modern digitized warfare. It provides a counterpoint to the argument that AI “rational” efficiency can simultaneously offer a viable solution to human psychological and biological fallibility in combat while retaining “meaningful” (...) human control over the war machine. This Panglossian assumption neglects the psychological features of human-machine interactions, the pace at which future AI-enabled conflict will be fought, and the complex and chaotic nature of modern war. The article expounds key psychological insights of human-machine interactions to elucidate how AI shapes our capacity to think about future warfare's political and ethical dilemmas. It argues that through the psychological process of human-machine integration, AI will not merely force-multiply existing advanced weaponry but will become de facto strategic actors in warfare – the “AI commander problem.”. (shrink)

The notion of “responsibility gap” with artificial intelligence (AI) was originally introduced in the philosophical debate to indicate the concern that “learning automata” may make more difficult or impossible to attribute moral culpability to persons for untoward events. Building on literature in moral and legal philosophy, and ethics of technology, the paper proposes a broader and more comprehensive analysis of the responsibility gap. The responsibility gap, it is argued, is not one problem but a set of at least four interconnected (...) problems – gaps in culpability, moral and public accountability, active responsibility—caused by different sources, some technical, other organisational, legal, ethical, and societal. Responsibility gaps may also happen with non-learning systems. The paper clarifies which aspect of AI may cause which gap in which form of responsibility, and why each of these gaps matter. It proposes a critical review of partial and non-satisfactory attempts to address the responsibility gap: those which present it as a new and intractable problem (“fatalism”), those which dismiss it as a false problem (“deflationism”), and those which reduce it to only one of its dimensions or sources and/or present it as a problem that can be solved by simply introducing new technical and/or legal tools (“solutionism”). The paper also outlines a more comprehensive approach to address the responsibility gaps with AI in their entirety, based on the idea of designing socio-technical systems for “meaningful human control", that is systems aligned with the relevant human reasons and capacities. (shrink)

Making good decisions in extremely complex and difficult processes and situations has always been both a key task as well as a challenge in the clinic and has led to a large amount of clinical, legal and ethical routines, protocols and reflections in order to guarantee fair, participatory and up-to-date pathways for clinical decision-making. Nevertheless, the complexity of processes and physical phenomena, time as well as economic constraints and not least further endeavours as well as achievements in medicine and healthcare (...) continuously raise the need to evaluate and to improve clinical decision-making. This article scrutinises if and how clinical decision-making processes are challenged by the rise of so-called artificial intelligence-driven decision support systems (AI-DSS). In a first step, this article analyses how the rise of AI-DSS will affect and transform the modes of interaction between different agents in the clinic. In a second step, we point out how these changing modes of interaction also imply shifts in the conditions of trustworthiness, epistemic challenges regarding transparency, the underlying normative concepts of agency and its embedding into concrete contexts of deployment and, finally, the consequences for (possible) ascriptions of responsibility. Third, we draw first conclusions for further steps regarding a ‘meaningful human control’ of clinical AI-DSS. (shrink)

A number of concerns have been recently raised regarding the possibility of human agents to effectively maintain control over intelligent and (partially) autonomous artificial systems. These issues have been deemed to raise “responsibility gaps.” To address these gaps, several scholars and other public and private stakeholders converged towards the idea that, in deploying intelligent technology, a meaningful form of human control (MHC) should be at all times exercised over autonomous intelligent technology. One of the main (...) criticisms of the general idea of MHC is that it could be inherently problematic to have high degrees of control and high degrees of autonomy at the same time, as the two dimensions appear to be inversely related. Several ways to respond to this argument and deal with the dilemma between control and autonomy have been proposed in the literature. In this paper, we further contribute to the philosophical effort to overcome the trade-off between automation and human control, and to open up some space for moral responsibility. We will use the instrument of conceptual engineering to investigate whether and to what extent removing the element of direct causal intervention from the concept of control can preserve the main functions of that concept, specifically focusing on the extent it can act as a foundation of moral responsibility. We show that at least one philosophical account of MHC is indeed a conceptually viable theory to absolve the fundamental functions of control, even in the context of completely autonomous artificial systems. (shrink)

Trust is one of the big buzzwords in debates about the shaping of society, democracy, and emerging technologies. For example, one prominent idea put forward by the High‐Level Expert Group on Artificial Intelligence appointed by the European Commission is that artificial intelligence should be trustworthy. In this essay, we explore the notion of trust and argue that both proponents and critics of trustworthy AI have flawed pictures of the nature of trust. We develop an approach to understanding trust in AI (...) that does not conceive of trust merely as an accelerator for societal acceptance of AI technologies. Instead, we argue, trust is granted through leaps of faith. For this reason, trust remains precarious, fragile, and resistant to promotion through formulaic approaches. We also highlight the significance of distrust in societal deliberation, as it is relevant to trust in various and intricate ways. Among the fruitful aspects of distrust is that it enables individuals to forgo technology if desired, to constrain its power, and to exercise meaningful human control. (shrink)

Defence agencies across the globe identify artificial intelligence as a key technology to maintain an edge over adversaries. As a result, efforts to develop or acquire AI capabilities for defence are growing on a global scale. Unfortunately, they remain unmatched by efforts to define ethical frameworks to guide the use of AI in the defence domain. This article provides one such framework. It identifies five principles—justified and overridable uses, just and transparent systems and processes, human moral responsibility, meaningful (...) human control and reliable AI systems—and related recommendations to foster ethically sound uses of AI for national defence purposes. (shrink)

In this article, we critique the ways in which the people involved in the development and application of AI systems often visualize and talk about AI systems. Often, they visualize such systems as shiny humanoid robots or as free-floating electronic brains. Such images convey misleading messages; as if AI works independently of people and can reason in ways superior to people. Instead, we propose to visualize AI systems as parts of larger, sociotechnical systems. Here, we can learn, for example, from (...) cybernetics. Similarly, we propose that the people involved in the design and deployment of an algorithm would need to extend their conversations beyond the four boxes of the _Error Matrix_, for example, to critically discuss _false positives_ and _false negatives_. We present two thought experiments, with one practical example in each. We propose to understand, visualize, and talk about AI systems in relation to a larger, complex reality; this is the requirement of _requisite variety_. We also propose to enable people from diverse disciplines to collaborate around _boundary objects_, for example: a drawing of an AI system in its sociotechnical context; or an ‘extended’ Error Matrix. Such interventions can promote meaningful human control, transparency, and fairness in the design and deployment of AI systems. (shrink)

Artificially intelligent machines are different in kind from all previous machines and tools. While many are used for relatively benign purposes, the types of artificially intelligent machines that we should care about, the ones that are worth focusing on, are the machines that purport to replace humans entirely and thereby engage in what Brian Cantwell Smith calls “judgment.” As impressive as artificially intelligent machines are, their abilities are still derived from humans and as such lack the sort of normative commitments (...) that humans have. So while artificially intelligent machines possess a great capacity for “reckoning,” to use Smith’s terminology, i.e., a calculative prowess of extraordinary utility and importance, they still lack the kind of considered human judgment that accompanies the ethical commitment and responsible action we humans must ultimately aspire toward. But there is a perfect technological storm brewing. Artificially intelligent machines are analogous to a perfect storm in that such machines involve the convergence of a number of factors that threaten our ability to behave ethically and maintain meaningful human control over the outcomes of processes involving artificial intelligence. I argue that the storm in the context of artificially intelligent machines makes us vulnerable to moral complacency. That is, this perfect technological storm is capable of lulling people into a state in which they abdicate responsibility for decision-making and behaviour precipitated by the use of artificially intelligent machines, a state that I am calling “moral complacency.” I focus on three salient problems that converge to make us especially vulnerable to becoming morally complacent and losing meaningful human control. The first problem is that of transparency/opacity. The second problem is that of overtrust in machines, often referred to as the automation bias. The third problem is that of ascribing responsibility. I examine each of these problems and how together they threaten to render us morally complacent. (shrink)

The rapid progress of artificial intelligence (AI) has resulted in its integration into various stages of the media process, including information gathering, processing, and distribution. This integration has raised the possibility of AI dominating the media industry, leading to an era of “autonomous driving” within AI-driven media systems. Similar to the ethical dilemma known as the “trolley problem” (TP) in autonomous driving, a comparable problem arises in AI automated media. This study examines the emergence of the new TP in fully (...) automated AI-driven media (FAAIM), recognizing the complexity of this problem given the nature of media content and its societal impact. To address this complexity, we propose the adoption of the theory of “meaningful human control,” originally developed to address responsibility in the governance of lethal autonomous weapons systems, as a framework for governing FAAIM. By ensuring that humans can be held responsible for the operations of FAAIM, this paper aims to proactively confront the ethical challenges arising from FAAIM, identify potential solutions, and guide future research in media ethics. (shrink)

The rapid progress of artificial intelligence (AI) has resulted in its integration into various stages of the media process, including information gathering, processing, and distribution. This integration has raised the possibility of AI dominating the media industry, leading to an era of “autonomous driving” within AI-driven media systems. Similar to the ethical dilemma known as the “trolley problem” (TP) in autonomous driving, a comparable problem arises in AI automated media. This study examines the emergence of the new TP in fully (...) automated AI-driven media (FAAIM), recognizing the complexity of this problem given the nature of media content and its societal impact. To address this complexity, we propose the adoption of the theory of “meaningful human control,” originally developed to address responsibility in the governance of lethal autonomous weapons systems, as a framework for governing FAAIM. By ensuring that humans can be held responsible for the operations of FAAIM, this paper aims to proactively confront the ethical challenges arising from FAAIM, identify potential solutions, and guide future research in media ethics. (shrink)

This article describes how an empirically supported theory of human behaviour, perceptual control theory, can be used to advance nursing practice and improve health outcomes for people who are accessing nursing care. Nursing often takes a pragmatic approach to the delivery of care, with an emphasis on doing what appears to work. This focus on pragmatism can sometimes take precedence over any consideration of the underlying theoretical assumptions that inform decisions to take one particular approach over another or (...) the mechanisms through which nursing interventions have their effects. For nursing to develop as a profession, there needs to be an increased focus on the core principles that underpin the delivery of care. In addition to understanding what works, nurses must develop their understanding of how and why particular approaches work or do not work. Understanding the fundamental principles that underpin nurses’ actions will lead to more efficient and effective approaches to the delivery of nursing care. It will also enable nurses to maximize those elements of their practice that are most beneficial for people and minimize other activities that either have little effect or actually lead to worse outcomes. In this article, we will propose that the phenomenon of control is fundamental to human health. Perceptual control theory provides a coherent theoretical framework that enables us to understand the phenomenon of control through a functional model of human behaviour. People are healthy when their neurochemical, physiological, biological, psychological and social states are all controlled satisfactorily. We will explain the implications of understanding health as control throughout the paper. From this perspective, we will argue that the aim of nurses and nursing should be to support people to maintain or recover control over those aspects of their lives that are important and meaningful to them. (shrink)

Criminal justice systems have traditionally relied heavily on human decision-making, but new technologies are increasingly supplementing the human role in this sector. This paper considers what general limits need to be placed on the use of algorithms in sentencing decisions. It argues that, even once we can build algorithms that equal human decision-making capacities, strict constraints need to be placed on how they are designed and developed. The act of condemnation is a valuable element of criminal sentencing, (...) and using algorithms in sentencing – even in an advisory role – threatens to undermine this value. The paper argues that a principle of “meaningful public control” should be met in all sentencing decisions if they are to retain their condemnatory status. This principle requires that agents who have standing to act on behalf of the wider political community retain moral responsibility for all sentencing decisions. While this principle does not rule out the use of algorithms, it does require limits on how they are constructed. (shrink)

Raul Hakli & Pekka Mäkelä (2016, 2019) make a popular assumption in machine ethics explicit by arguing that artificial agents cannot be responsible because they are designed. Designed agents, they think, are analogous to manipulated humans and therefore not meaningfully in control of their actions. Contrary to this, I argue that under all mainstream theories of responsibility, designed agents can be responsible. To do so, I identify the closest parallel discussion in the literature on responsibility and free will, which (...) concerns ‘design cases’. Design cases are theoretical examples of agents that appear to lack responsibility because they were designed, philosophers use these cases to explore the relationship between design and responsibility. This paper presents several replies to design cases from the responsibility literature and uses those replies to situate the corresponding positions on the design and responsibility of artificial agents in machine ethics. I argue that each reply can support the design of responsible agents. However, each reply also entails different levels of severity in the constraints for the design of responsible agents. I offer a brief discussion of the nature of those constraints, highlighting the challenges respective to each reply. I conclude that designing responsible agents is possible, with the caveat that the difficulty of doing so will vary according to one’s favoured reply to design cases. (shrink)

A groundbreaking guide showing us how being "out of control" (and admitting it) is the first step to living a truly better, more meaningful life. Raise your hand if you've ever wanted to "self-improve" but, for some reason, you just can't follow through. Turns out, the issue isn't a lack of willpower. For centuries, we've been fed a common perspective: Explore your subconscious mind, heal your trauma, fit into your society, and happiness will follow, right? Wrong. Dr. Courtney (...) Tracy, also known as "The Truth Doctor," disrupts this outdated narrative through digestible scientific research, shockingly honest personal stories, and compassionate-yet-direct advice. Feeling out of control and helpless isn't a flaw but a universal truth of our existence. Instead of trying to change how we work as human beings (spoiler alert: you can't,) we need to embrace and make peace with our unconscious, making it work for and alongside us instead of against. Half psychology textbook written by your best friend (who's also a therapist), half comprehensive guide brimming with actionable insights for engaging with our unconscious positively and productively, Your Unconscious Is Showing is here to help us accept what we can't control, courageously change what we can, and wisely know the difference. (shrink)

The majority of animal advocacy discourse is unidirectional: Humans are regarded as stewards of animal welfare, and humans control the bestowal of rights and protections upon animals. This article offers a reversal of the typical moral reflection used in animal advocacy. I suggest that our relationship with animals participates in the development of moral faculties requisite for ethical behavior. In other words, we have a lot to learn from animals, not in this instance by documenting their behavior, but from (...) having meaningful relationships with particular animals. Quality interactions with animals can stimulate the imaginative basis for the care and empathy that are crucial for social morality. To accomplish this task, I describe “embodied care” as an extension of feminist care ethics that addresses the body’s role in morality, and argue that our relationships with animals can provide the imaginative foundation for improving human-to-human morality. (shrink)

This essay explores some concerns about the quality of informed consent in patients whose autonomy is diminished by fatal illness. It argues that patients with diminished autonomy cannot give free and voluntary consent, and that recruitment of such patients as subjects in human experimentation exploits their vulnerability in a morally objectionable way. Two options are given to overcome this objection: (i) recruit only those patients who desire to contribute to medical knowledge, rather than gain access to experimental treatment, or (...) (ii) provide prospective subjects the choice to participate in standard doubleblind study or receive the experimental treatment. Either option would guarantee that patients in desperate conditions are given a more meaningful choice and a richer freedom, and thus a higher quality of informed consent, than under standard randomized trials. Keywords: autonomy, double-blind trials, prerandomized and randomized trials, informed consent CiteULike Connotea Del.icio.us What's this? (shrink)

This essay explores some concerns about the quality of informed consent in patients whose autonomy is diminished by fatal illness. It argues that patients with diminished autonomy cannot give free and voluntary consent, and that recruitment of such patients as subjects in human experimentation exploits their vulnerability in a morally objectionable way. Two options are given to overcome this objection: (i) recruit only those patients who desire to contribute to medical knowledge, rather than gain access to experimental treatment, or (...) (ii) provide prospective subjects the choice to participate in standard doubleblind study or receive the experimental treatment. Either option would guarantee that patients in desperate conditions are given a more meaningful choice and a richer freedom, and thus a higher quality of informed consent, than under standard randomized trials. (shrink)

Individuals regaining reliable day-to-day walking function after incomplete spinal cord injury report persisting unsteadiness when confronted with walking challenges. However, quantifiable measures of walking capacity lack the sensitivity to reveal underlying impairments of supra-spinal locomotor control. This study investigates the relationship between intramuscular coherence and corticospinal dynamic balance control during a visually guided Target walking treadmill task. In thirteen individuals with iSCI and 24 controls, intramuscular coherence and cumulant densities were estimated from pairs of Tibialis anterior surface EMG (...) recordings during normal treadmill walking and a Target walking task. The approximate center of mass was calculated from pelvis markers. Spearman rank correlations were performed to evaluate the relationship between intramuscular coherence, clinical parameters, and center of mass parameters. In controls, we found that the Target walking task results in increased high-frequency intramuscular coherence, which negatively related to changes in the center of mass movement, whereas this modulation was largely reduced in individuals with iSCI. The impaired modulation of high-frequency intramuscular coherence during the Target walking task correlated with neurophysiological and functional readouts, such as motor-evoked potential amplitude and outdoor mobility score, as well as center of mass trajectory length. The Target walking effect, the difference between Target and Normal walking intramuscular coherence, was significantly higher in controls than in individuals with iSCI [F = 13.042, p < 0.001]. Intramuscular coherence obtained during challenging walking in individuals with iSCI may provide information on corticospinal gait control. The relationships between biomechanics, clinical scores, and neurophysiology suggest that intramuscular coherence assessed during challenging tasks may be meaningful for understanding impaired supra-spinal control in individuals with iSCI. (shrink)

In this article I will address the issue of the meaning of human-machine interaction as it is configured today in the light of substantial results achieved in the design and the manufacture of Artificial Intelligence (AI) systems. My starting point is a refined solution for meaningful AI recently suggested by Froese and Taguchi from the perspective of so-called augmented intelligence. Interpreted as a kind of human-machine interaction, augmented intelligence distinguishes itself by the fact that it merges the (...) interacting poles into a new hybrid entity. After having clarified the technical and the methodological background of my proposal, I will discuss a case of study of so-called bio-synthetic augmentation, where hybridisation overlaps with the cyborgisation of the human body. (shrink)