It was recently shown that the theories of generic algebraic curves converge to a limit theory as their degrees go to infinity. In this paper we give quantitative versions of this result and other similar results. In particular, we show that generic curves of degree higher than 22r cannot be distinguished by a first-order formula of quantifier rank r. A decision algorithm for the limit theory then follows easily. We also show that in this theory all formulas are equivalent (...) to boolean combinations of existential formulas, and give a quantitative version of this result. (shrink)

As is well-known, the Bernays-Schönfinkel-Ramsey class of all prenex ∃*∀* -sentences which are valid in classical first-order logic is decidable. This paper paves the way to an analogous result which the authors deem to hold when the only available predicate symbols are ∈ and =, no constants or function symbols are present, and one moves inside a (rather generic) Set Theory whose axioms yield the well-foundedness of membership and the existence of infinite sets. Here semi-decidability of the satisfiability problem (...) for the BSR class is proved by following a purely semantic approach, the remaining part of the decidability result being postponed to a forthcoming paper. (shrink)

As proved recently, the satisfaction problem for all prenex formulae in the set-theoretic Bernays-Shönfinkel-Ramsey class is semi-decidable over von Neumann's cumulative hierarchy. Here that semi-decidability result is strengthened into a decidability result for the same collection of formulae.

Algorithms used in the public sector, e.g., for allocating social benefits or predicting fraud, often require involvement from multiple stakeholders at various phases of the algorithm’s life-cycle. This paper focuses on the communication issues between diverse stakeholders that can lead to misinterpretation and misuse of algorithmic systems. Ethnographic research was conducted via 11 semi-structured interviews with practitioners working on algorithmic systems in the Dutch public sector, at local and national levels. With qualitative coding analysis, we identify key elements (...) of the communication processes that underlie fairness-related human decisions. More specifically, we analyze the division of roles and tasks, the required skills, and the challenges perceived by diverse stakeholders. Three general patterns emerge from the coding analysis: (1) Policymakers, civil servants, and domain experts are less involved compared to developers throughout a system’s life-cycle. This leads to developers taking on the role of decision-maker and policy advisor, while they potentially miss the required skills. (2) End-users and policy-makers often lack the technical skills to interpret a system’s output, and rely on actors having a developer role for making decisions concerning fairness issues. (3) Citizens are structurally absent throughout a system’s life-cycle. This may lead to unbalanced fairness assessments that do not include key input from relevant stakeholders. We formalize the underlying communication issues within such networks of stakeholders and introduce the phase-actor-role-task-skill (PARTS) model. PARTS can both (i) represent the communication patterns identified in the interviews, and (ii) explicitly outline missing elements in communication patterns such as actors who miss skills or collaborators for their tasks, or tasks that miss qualified actors. The PARTS model can be extended to other use cases and used to analyze and design the human organizations responsible for assessing fairness in algorithmic systems. It can be further extended to explore communication issues in other use cases, design potential solutions, and organize accountability with a common vocabulary. (shrink)

Should decision-making algorithms be held to higher standards of transparency than human beings? The way we answer this question directly impacts what we demand from explainable algorithms, how we govern them via regulatory proposals, and how explainable algorithms may help resolve the social problems associated with decision making supported by artificial intelligence. Some argue that algorithms and humans should be held to the same standards of transparency and that a double standard of transparency is (...) hardly justified. We give two arguments to the contrary and specify two kinds of situations for which higher standards of transparency are required from algorithmic decisions as compared to humans. Our arguments have direct implications on the demands from explainable algorithms in decision-making contexts such as automated transportation. (shrink)

This article examines the complaint that arbitrary algorithmic decisions wrong those whom they affect. It makes three contributions. First, it provides an analysis of what arbitrariness means in this context. Second, it argues that arbitrariness is not of moral concern except when special circumstances apply. However, when the same algorithm or different algorithms based on the same data are used in multiple contexts, a person may be arbitrarily excluded from a broad range of opportunities. The third contribution is to (...) explain why this systemic exclusion is of moral concern and to offer a solution to address it. (shrink)

In the study of rational solutions to polynomial equations in two-variables, we show that an algorithmic solution to the decision problem (existence of solutions) enables one to construct a search algorithm for all solutions.

Because of its practical advantages, machine learning (ML) is increasingly used for decision-making in numerous sectors. This paper demonstrates that the integral characteristics of ML, such as semi-autonomy, complexity, and non-deterministic modeling have important ethical implications. In particular, these characteristics lead to a lack of insight and lack of comprehensibility, and ultimately to the loss of human control over decision-making. Errors, which are bound to occur in any decision-making process, may lead to great harm and human (...) rights violations. It is important to have a principled way of assigning responsibility for such errors. The integral characteristics of ML, however, pose serious difficulties in defining responsibility and regulating ML decision-making. First, we elaborate on these characteristics and their epistemic and ethical implications. We then analyze possible general strategies for resolving the assignment of moral responsibility and show that, due to the specific way in which ML functions, each potential solution is problematic, whether we assign responsibility to humans, machines, or using hybrid models. Then, we shift focus on an alternative approach that bypasses moral responsibility and attempts to define legal liability independently through solutions such as informed consent and the no-fault compensation system. Both of these solutions prove unsatisfactory because they leave too much room for potential abuses of ML decision-making. We conclude that both ethical and legal solutions are fraught with serious difficulties. These difficulties prompt us to re-weigh the costs and benefits of using ML for high-stake decisions. (shrink)

We are sceptical of concerns over the opacity of algorithmic decision tools. While transparency and explainability are certainly important desiderata in algorithmic governance, we worry that automated decision-making is being held to an unrealistically high standard, possibly owing to an unrealistically high estimate of the degree of transparency attainable from human decision-makers. In this paper, we review evidence demonstrating that much human decision-making is fraught with transparency problems, show in what respects AI fares little worse or (...) better and argue that at least some regulatory proposals for explainable AI could end up setting the bar higher than is necessary or indeed helpful. The demands of practical reason require the justification of action to be pitched at the level of practical reason. Decision tools that support or supplant practical reasoning should not be expected to aim higher than this. We cast this desideratum in terms of Daniel Dennett’s theory of the “intentional stance” and argue that since the justification of action for human purposes takes the form of intentional stance explanation, the justification of algorithmic decisions should take the same form. In practice, this means that the sorts of explanations for algorithmic decisions that are analogous to intentional stance explanations should be preferred over ones that aim at the architectural innards of a decision tool. (shrink)

Algorithms silently structure our lives. Algorithms can determine whether someone is hired, promoted, offered a loan, or provided housing as well as determine which political ads and news articles consumers see. Yet, the responsibility for algorithms in these important decisions is not clear. This article identifies whether developers have a responsibility for their algorithms later in use, what those firms are responsible for, and the normative grounding for that responsibility. I conceptualize algorithms as value-laden, rather (...) than neutral, in that algorithms create moral consequences, reinforce or undercut ethical principles, and enable or diminish stakeholder rights and dignity. In addition, algorithms are an important actor in ethical decisions and influence the delegation of roles and responsibilities within these decisions. As such, firms should be responsible not only for the value-laden-ness of an algorithm but also for designing who-does-what within the algorithmic decision. As such, firms developing algorithms are accountable for designing how large a role individual will be permitted to take in the subsequent algorithmic decision. Counter to current arguments, I find that if an algorithm is designed to preclude individuals from taking responsibility within a decision, then the designer of the algorithm should be held accountable for the ethical implications of the algorithm in use. (shrink)

. Network data analysis helps in capturing node usage behavior. Existing algorithms use reduced feature set to manage high runtime complexity. Ignoring features may increase classification errors. This paper presents a model, allowing classification of network traffic, while considering all the relevant features. Learning phase partitions training sample on values of the respective features. This creates equivalence classes related to m features. During classification, each feature value of the test instance results in picking one set from equivalence class generated (...) during learning. Algorithm captures new behavior in semi-supervised incremental learning mode. For problems having m features and n training samples the model has incremental learning complexity of and average classification complexity is of the order. (shrink)

We are currently witnessing a sharp rise in the use of algorithmic decision-making tools. In these instances, a new wave of policy concerns is set forth. This article strives to map out these issues, separating the wheat from the chaff. It aims to provide policy makers and scholars with a comprehensive framework for approaching these thorny issues in their various capacities. To achieve this objective, this article focuses its attention on a general analytical framework, which will be applied to (...) a specific subset of the overall discussion. The analytical framework will reduce the discussion to two dimensions, every one of which addressing two central elements. These four factors call for a distinct discussion, which is at times absent in the existing literature. The two dimensions are the specific and novel problems the process assumedly generates and the specific attributes which exacerbate them. While the problems are articulated in a variety of ways, they most likely could be reduced to two broad categories: efficiency and fairness-based concerns. In the context of this discussion, such problems are usually linked to two salient attributes the algorithmic processes feature—its opaque and automated nature. (shrink)

In this paper, we point out that explainability is useful but not sufficient to ensure the legitimacy of algorithmic decision systems. We argue that the key requirements for high-stakes decision systems should be justifiability and contestability. We highlight the conceptual differences between explanations and justifications, provide dual definitions of justifications and contestations, and suggest different ways to operationalize justifiability and contestability.

In decision making quite often we face permanently changeable and potentially infinite databases when we cannot apply conventional algorithms for choosing a solution. A decision process on infinite databases is called troubleshooting. A decision on these databases is called creative reasoning. One of the first heuristic semi-logical means for creative decision making were proposed in the theory of inventive problem solving by Genrich Altshuller. In this paper, I show that his approach corresponds to the (...) so-called content-generic logic established by Soviet philosophers as an alternative to mathematical logic. The main assumption of content-genetic is that we cannot reduce our thinking to a mathematical combination of signs or to a language as such and our thought is ever cyclic and reflexive so that it contains ever a history. (shrink)

While principle-based ethics is well known and widely accepted in psychiatry, much less is known about how decisions are made in clinical practice, which case scenarios exist, and which challenges exist for decision-making. Protocols of the central ethics committee responsible for four psychiatric hospitals over 7 years (N=17) were analysed. While four cases concerned suicide risk in the case of intended hospital discharge, the vast majority (N=13) concerned questions of whether the responsible physician should or should not initiate the (...) use of coercion in patients lacking mental capacity. The committee’s recommendations were non-uniform. Forced feeding and electroconvulsive therapy were endorsed in each one case. In two cases of intermittent loss of capacity due to heavy drinking or intermittent severe suicidal ideation, a self-binding contract was recommended and the use of coercion was considered as justified for a very limited period. In all other cases, most of which involved involuntary treatment, the use of coercion was not endorsed. Without exception, the recommendations were accepted with relief by the physicians and their treatment teams, who feared liability in the event of harm to the patient. Eventually, a model of a decision algorithm was derived from the ethical arguments in the protocols. (shrink)

By PFM, we mean a finitely generated module over a principal ideal domain; a linear sentence is a sentence that contains no disjunctive and negative symbols. In this paper, we present an algorithm which decides the truth for linear sentences on a given PFM, and we discuss its time complexity. In particular, when the principal ideal domain is the ring of integers or a univariate polynomial ring over the field of rationals, the algorithm is polynomial-time. Finally, we consider some applications (...) to Abelian groups. (shrink)

Introduction It is increasingly considered important that people make an autonomous and informed decision concerning colorectal cancer screening. However, the realisation of autonomy within the concept of informed decision-making might be interpreted too narrowly. Additionally, relatively little is known about what the eligible population believes to be a 'good' screening decision. Therefore, we aimed to explore how the concepts of autonomous and informed decision-making relate to how the eligible CRC screening population makes their decision and (...) when they believe to have made a 'good' screening decision. Methods We conducted 27 semi-structured interviews with the eligible CRC screening population. The general topics discussed concerned how people made their CRC screening decision, how they experienced making this decision and when they considered they had made a 'good' decision. Results Most interviewees viewed a 'good' CRC screening decision as one based on both reasoning and feeling/intuition, and that is made freely. However, many CRC screening non-participants experienced a certain social pressure to participate. All CRC screening non-participants viewed making an informed decision as essential. This appeared to be the case to a lesser extent for CRC screening participants. For most, experiences and values were involved in their decision-making. Conclusion Our sample of the eligible CRC screening population viewed aspects related to the concepts of autonomous and informed decision-making as important for making a 'good' CRC screening decision. However, in particular the existence of a social norm may be affecting a true autonomous decision-making process. Additionally, the present concept of informed decision-making with its strong emphasis on making a fully informed and well-considered decision does not appear to be entirely reflective of the process in practice. More efforts could be made to attune to the diverse values and factors that are involved in deciding about CRC screening participation. (shrink)

The irreversibility in time, the multicausality on lines, and the uncertainty of feedbacks make economic systems and the predictions of economic chaotic time series possess the characteristics of high dimensionalities, multiconstraints, and complex nonlinearities. Based on genetic algorithm and fuzzy rules, the chaotic genetics combined with fuzzy decision-making can use simple, fast, and flexible means to complete the goals of automation and intelligence that are difficult to traditional predicting algorithms. Moreover, the new combined method’s ergodicity can perform nonrepetitive (...) searches in a global scope, hence improving the algorithm’s accuracy and efficiency. On the basis of summarizing and analyzing previous research works, this paper expounded the research status and significance of the prediction of economic chaotic time series, elaborated the development background, current status, and future challenges of the combined algorithm of chaotic genetics with fuzzy decision, introduced the basic principles of chaotic genetic algorithm and fuzzy decision algorithm, constructed a prediction model for economic chaotic time series, performed parameter synchronization optimization and moderate function construction, analyzed the prediction processes of economic chaotic time series, conducted phase space reconstruction and correlation dimension calculation, and finally carried out a simulation experiment with its result analysis. The study results show that the algorithm of chaotic genetics combined with fuzzy decision-making can dynamically adjust chaotic mutation operators and summarize fussy expert experiences. The phase space of its reconstructed chaotic attractor has high-precision predictability and can find orderly processes from changeable economic results, which in turn can be used to analyze and predict the complex economic chaotic time series. The study results of this paper provide a reference for further research on predictive analysis of economic chaotic time series based on chaotic genetics combined with fuzzy decision algorithm. (shrink)

Algorithmic systems are increasingly used by state agencies to inform decisions about humans. They produce scores on risks of recidivism in criminal justice, indicate the probability for a job seeker to find a job in the labor market, or calculate whether an applicant should get access to a certain university program. In this contribution, we take an interdisciplinary perspective, provide a bird’s eye view of the different key decisions that are to be taken when state actors decide to use an (...) algorithmic system, and illustrate these decisions with empirical examples from case studies. Building on these insights, we discuss the main pitfalls and promises of the use of algorithmic system by the state and focus on four levels: The most basic question whether an algorithmic system should be used at all, the regulation and governance of the system, issues of algorithm design, and, finally, questions related to the implementation of the system on the ground and the human–machine-interaction that comes with it. Based on our assessment of the advantages and challenges that arise at each of these levels, we propose a set of crucial questions to be asked when such intricate matters are addressed. (shrink)

As we increasingly delegate decision-making to algorithms, whether directly or indirectly, important questions emerge in circumstances where those decisions have direct consequences for individual rights and personal opportunities, as well as for the collective good. A key problem for policymakers is that the social implications of these new methods can only be grasped if there is an adequate comprehension of their general technical underpinnings. The discussion here focuses primarily on the case of enforcement decisions in the criminal justice (...) system, but draws on similar situations emerging from other algorithms utilised in controlling access to opportunities, to explain how machine learning works and, as a result, how decisions are made by modern intelligent algorithms or 'classifiers'. It examines the key aspects of the performance of classifiers, including how classifiers learn, the fact that they operate on the basis of correlation rather than causation, and that the term 'bias' in machine learning has a different meaning to common usage. An example of a real world 'classifier', the Harm Assessment Risk Tool (HART), is examined, through identification of its technical features: the classification method, the training data and the test data, the features and the labels, validation and performance measures. Four normative benchmarks are then considered by reference to HART: (a) prediction accuracy (b) fairness and equality before the law (c) transparency and accountability (d) informational privacy and freedom of expression, in order to demonstrate how its technical features have important normative dimensions that bear directly on the extent to which the system can be regarded as a viable and legitimate support for, or even alternative to, existing human decision-makers. (shrink)

The aim of this paper is to construct a tableau decision algorithm for the modal description logic K ALC with constant domains. More precisely, we present a tableau procedure that is capable of deciding, given an ALC-formula with extra modal operators (which are applied only to concepts and TBox axioms, but not to roles), whether is satisfiable in a model with constant domains and arbitrary accessibility relations. Tableau-based algorithms have been shown to be practical even for logics of (...) rather high complexity. This gives us grounds to believe that, although the satisfiability problem for K ALC is known to be NEXPTIME-complete, by providing a tableau decision algorithm we demonstrate that highly expressive description logics with modal operators have a chance to be implementable. The paper gives a solution to an open problem of Baader and Laux [5]. (shrink)

Using artificial intelligence (AI) to make decisions in human resource management (HRM) raises questions of how fair employees perceive these decisions to be and whether they experience respectful treatment (i.e., interactional justice). In this experimental survey study with open-ended qualitative questions, we examine decision making in six HRM functions and manipulate the decision maker (AI or human) and decision valence (positive or negative) to determine their impact on individuals’ experiences of interactional justice, trust, dehumanization, and perceptions of (...) decision-maker role appropriate- ness. In terms of decision makers, the use of human decision makers over AIs generally resulted in better perceptions of respectful treatment. In terms of decision valence, people experiencing positive over negative decisions generally resulted in better perceptions of respectful treatment. In instances where these cases conflict, on some indicators people preferred positive AI decisions over negative human decisions. Qualitative responses show how people identify justice concerns with both AI and human decision making. We outline implications for theory, practice, and future research. (shrink)

Firms use algorithms to make important business decisions. To date, the algorithmic accountability literature has elided a fundamentally empirical question important to business ethics and management: Under what circumstances, if any, are algorithmic decision-making systems considered legitimate? The present study begins to answer this question. Using factorial vignette survey methodology, we explore the impact of decision importance, governance, outcomes, and data inputs on perceptions of the legitimacy of algorithmic decisions made by firms. We find that many of (...) the procedural governance mechanisms in practice today, such as notices and impact statements, do not lead to algorithmic decisions being perceived as more legitimate in general, and, consistent with legitimacy theory, that algorithmic decisions with good outcomes are perceived as more legitimate than bad outcomes. Yet, robust governance, such as offering an appeal process, can create a legitimacy dividend for decisions with bad outcomes. However, when arbitrary or morally dubious factors are used to make decisions, most legitimacy dividends are erased. In other words, companies cannot overcome the legitimacy penalty of using arbitrary or morally dubious factors, such as race or the day of the week, with a good outcome or an appeal process for individuals. These findings add new perspectives to both the literature on legitimacy and policy discussions on algorithmic decision-making in firms. (shrink)

The use of AI algorithms in criminal trials has been the subject of very lively ethical and legal debates recently. While there are concerns over the lack of accuracy and the harmful biases that certain algorithms display, new algorithms seem more promising and might lead to more accurate legal decisions. Algorithms seem especially relevant for bail decisions, because such decisions involve statistical data to which human reasoners struggle to give adequate weight. While getting the right legal (...) outcome is a strong desideratum of criminal trials, advocates of the relational theory of procedural justice give us good reason to think that fairness and perceived fairness of legal procedures have a value that is independent from the outcome. According to this literature, one key aspect of fairness is trustworthiness. In this paper, I argue that using certain algorithms to assist bail decisions could increase three diferent aspects of judges’ trustworthiness: (1) actual trustworthiness, (2) rich trustworthiness, and (3) perceived trustworthiness. (shrink)

The danger of human operators devolving responsibility to machines and failing to detect cases where they fail has been recognised for many years by industrial psychologists and engineers studying the human operators of complex machines. We call it “the control problem”, understood as the tendency of the human within a human–machine control loop to become complacent, over-reliant or unduly diffident when faced with the outputs of a reliable autonomous system. While the control problem has been investigated for some time, up (...) to this point its manifestation in machine learning contexts has not received serious attention. This paper aims to fill that gap. We argue that, except in certain special circumstances, algorithmic decision tools should not be used in high-stakes or safety-critical decisions unless the systems concerned are significantly “better than human” in the relevant domain or subdomain of decision-making. More concretely, we recommend three strategies to address the control problem, the most promising of which involves a complementary coupling between highly proficient algorithmic tools and human agents working alongside one another. We also identify six key principles which all such human–machine systems should reflect in their design. These can serve as a framework both for assessing the viability of any such human–machine system as well as guiding the design and implementation of such systems generally. (shrink)

Doctoral students in clinical, counseling, and school psychology programs often collaborate with faculty on research projects in their training as scientist-practitioners. Yet, the determination of publications' credit and order of authorship on resulting manuscripts continues to be a major concern and challenging process for professional psychologists and student collaborators. This article describes the use of case-based learning and semi-structured interview approaches to instruct first-year clinical psychology doctoral students in publication ethics during a research seminar. The instructor models ethical (...) class='Hi'>decision-making with 1) a discussion of four cases from his own professional experiences and 2) a description of the Authorship Eligibility Assessment form, which he developed for use with junior researchers. The authors advocate for more educational strategies to supplement the APA standards in teaching the ethics of the publication process. Implications in terms of graduate student development and research collaborations in the field of professional psychology were discussed. (shrink)

Background Artificial intelligence-driven Clinical Decision Support Systems (AI-CDSS) are increasingly being integrated into healthcare for various purposes, including resource allocation. While these systems promise improved efficiency and decision-making, they also raise significant ethical concerns. This study aims to explore healthcare professionals’ perspectives on the ethical implications of using AI-CDSS for healthcare resource allocation. Methods We conducted semi-structured qualitative interviews with 23 healthcare professionals, including physicians, nurses, administrators, and medical ethicists in Turkey. Interviews focused on participants’ views regarding (...) the use of AI-CDSS in resource allocation, potential ethical challenges, and recommendations for responsible implementation. Data were analyzed using thematic analysis. Results Participant responses are clustered around five pre-determined thematic areas: (1) balancing efficiency and equity in resource allocation, (2) the importance of transparency and explicability in AI-CDSS, (3) shifting roles and responsibilities in clinical decision-making, (4) ethical considerations in data usage and algorithm development, and (5) balancing cost-effectiveness and patient-centered care. Participants acknowledged the potential of AI-CDSS to optimize resource allocation but expressed concerns about exacerbating healthcare disparities, the need for interpretable AI models, changing professional roles, data privacy, and maintaining individualized care. Conclusions The integration of AI-CDSS into healthcare resource allocation presents both opportunities and significant ethical challenges. Our findings underscore the need for robust ethical frameworks, enhanced AI literacy among healthcare professionals, interdisciplinary collaboration, and rigorous monitoring and evaluation processes. Addressing these challenges proactively is crucial for harnessing the potential of AI-CDSS while preserving the fundamental values of equity, transparency, and patient-centered care in healthcare delivery. (shrink)

While algorithmic decision-making (ADM) is projected to increase exponentially in the coming decades, the academic debate on whether people are ready to accept, trust, and use ADM as opposed to human decision-making is ongoing. The current research aims at reconciling conflicting findings on ‘algorithmic aversion’ in the literature. It does so by investigating algorithmic aversion while controlling for two important characteristics that are often associated with ADM: increased benefits (monetary and accuracy) and decreased user control. Across three high-powered (...) (Ntotal = 1192), preregistered 2 (agent: algorithm/human) × 2 (benefits: high/low) × 2 (control: user control/no control) between-subjects experiments, and two domains (finance and dating), the results were quite consistent: there is little evidence for a default aversion against algorithms and in favor of human decision makers. Instead, users accept or reject decisions and decisional agents based on their predicted benefits and the ability to exercise control over the decision. (shrink)

Decision-making assisted by algorithms developed by machine learning is increasingly determining our lives. Unfortunately, full opacity about the process is the norm. Would transparency contribute to restoring accountability for such systems as is often maintained? Several objections to full transparency are examined: the loss of privacy when datasets become public, the perverse effects of disclosure of the very algorithms themselves (“gaming the system” in particular), the potential loss of companies’ competitive edge, and the limited gains in answerability (...) to be expected since sophisticated algorithms usually are inherently opaque. It is concluded that, at least presently, full transparency for oversight bodies alone is the only feasible option; extending it to the public at large is normally not advisable. Moreover, it is argued that algorithmic decisions preferably should become more understandable; to that effect, the models of machine learning to be employed should either be interpreted ex post or be interpretable by design ex ante. (shrink)

Qualitative evidence concerning the relationship between QoL and a wide range of disabilities suggests that subjective judgments regarding other people’s QoL are wrong more often than not and that such judgments by medical practitioners in particular can be biased. Guided by their desire to do good and avoid harm, surgeons often rely on "the eyeball test" to decide whether a patient will or will not benefit from surgery. But the eyeball test can easily harbor a range of implicit judgments and (...) biases against patients with disabilities, including erroneous assumptions about QoL. We critique the use of the "eyeball" test and offer suggestions to reduce bias and ensure that patients’ values are more consistently prioritized in surgical decision-making. (shrink)

Decision-making assisted by algorithms developed by machine learning is increasingly determining our lives. Unfortunately, full opacity about the process is the norm. Would transparency contribute to restoring accountability for such systems as is often maintained? Several objections to full transparency are examined: the loss of privacy when datasets become public, the perverse effects of disclosure of the very algorithms themselves, the potential loss of companies’ competitive edge, and the limited gains in answerability to be expected since sophisticated (...) algorithms usually are inherently opaque. It is concluded that, at least presently, full transparency for oversight bodies alone is the only feasible option; extending it to the public at large is normally not advisable. Moreover, it is argued that algorithmic decisions preferably should become more understandable; to that effect, the models of machine learning to be employed should either be interpreted ex post or be interpretable by design ex ante. (shrink)

Background: Current practice frequently fails to provide care consistent with the preferences of decisionally-incapacitated patients. It also imposes significant emotional burden on their surrogates. Algorithmic-based patient preference predictors (PPPs) have been proposed as a possible way to address these two concerns. While previous research found that patients strongly support the use of PPPs, the views of surrogates are unknown. The present study thus assessed the views of experienced surrogates regarding the possible use of PPPs as a means to help make (...) treatment decisions for decisionally-incapacitated patients. -/- Methods: This qualitative study used semi-structured interviews to determine the views of experienced surrogates [n=26] who were identified from two academic medical centers and two community hospitals. The primary outcomes were respondents’ overall level of support for the idea of using PPPs and the themes related to their views on how a PPP should be used, if at all, in practice. -/- Results: Overall, 21 participants supported the idea of using PPPs. The remaining five indicated that they would not use a PPP because they made decisions based on the patient’s best interests, not based on substituted judgment. Major themes which emerged were that surrogates, not the patient’s preferences, should determine how treatment decisions are made, and concern that PPPs might be used to deny expensive care or be biased against minority groups. -/- Conclusions: Surrogates, like patients, strongly support the idea of using PPPs to help make treatment decisions for decisionally-incapacitated patients. These findings provide support for developing a PPP and assessing it in practice. At the same time, patients and surrogates disagree over whose preferences should determine how treatment decisions are made, including whether to use a PPP. These findings reveal a fundamental disagreement regarding the guiding principles for surrogate decision-making. Future research is needed to assess this disagreement and consider ways to address it. (shrink)

With the advent of automated decision-making, governments have increasingly begun to rely on artificially intelligent algorithms to inform policy decisions across a range of domains of government interest and influence. The practice has not gone unnoticed among philosophers, worried about “algocracy”, and its ethical and political impacts. One of the chief issues of ethical and political significance raised by algocratic governance, so the argument goes, is the lack of transparency of algorithms. One of the best-known examples of (...) philosophical analyses of algocracy is John Danaher’s “The threat of algocracy”, arguing that government by algorithm undermines political legitimacy. In this paper, I will treat Danaher’s argument as a springboard for raising additional questions about the connections between algocracy, comprehensibility, and legitimacy, especially in light of empirical results about what we can expect the voters and policymakers to know. The paper has the following structure: in Sect. 2, I introduce the basics of Danaher’s argument regarding algocracy. In Sect. 3 I argue that the algocratic threat to legitimacy has troubling implications for social justice. In Sect. 4, I argue that, nevertheless, there seem to be good reasons for governments to rely on algorithmic decision support systems. Lastly, I try to resolve the apparent tension between the findings of the two preceding Sections. (shrink)

ABSTRACT I argue that this examination and appreciation for the shift to abductive reasoning should be extended to the intersection of neuroscience and novel brain-computer interfaces too. This paper highlights the implications of applying abductive reasoning to personalized implantable neurotechnologies. Then, it explores whether abductive reasoning is sufficient to justify insurance coverage for devices absent widespread clinical trials, which are better applied to one-size-fits-all treatments. INTRODUCTION In contrast to the classic model of randomized-control trials, often with a large number of (...) subjects enrolled, precision medicine attempts to optimize therapeutic outcomes by focusing on the individual.[i] A recent publication highlights the strengths and weakness of both traditional evidence-based medicine and precision medicine.[ii] Plus, it outlines a tension in the shift from evidence-based medicine’s inductive reasoning style (the collection of data to postulate general theories) to precision medicine’s abductive reasoning style (the generation of an idea from the limited data available).[iii] The paper’s main example is the application of precision medicine for the treatment of cancer.[iv] I argue that this examination and appreciation for the shift to abductive reasoning should be extended to the intersection of neuroscience and novel brain-computer interfaces too. As the name suggests, brain-computer interfaces are a significant advancement in neurotechnology that directly connects someone’s brain to external or implanted devices.[v] Among the various kinds of brain-computer interfaces, adaptive deep brain stimulation devices require numerous personalized adjustments to their settings during the implantation and computation stages in order to provide adequate relief to patients with treatment-resistant disorders. What makes these devices unique is how adaptive deep brain stimulation integrates a sensory component to initiate the stimulation. While not commonly at the level of sophistication as self-supervising or generative large language models,[vi] they currently allow for a semi-autonomous form of neuromodulation. This paper highlights the implications of applying abductive reasoning to personalized implantable neurotechnologies. Then, it explores whether abductive reasoning is sufficient to justify insurance coverage for devices absent widespread clinical trials, which are better applied to one-size-fits-all treatments.[vii] ANALYSIS I. The State of Precision Medicine in Oncology and the Epistemological Shift While a thorough overview of precision medicine for the treatment of cancer is beyond the scope of this article, its practice can be roughly summarized as identifying clinically significant characteristics a patient possesses (e.g., genetic traits) to land on a specialized treatment option that, theoretically, should benefit the patient the most.[viii] However, in such a practice of stratification patients fall into smaller and smaller populations and the quality of evidence that can be applied to anyone outside these decreases in turn.[ix] As inductive logic helps to articulate, the greater the number of patients that respond to a particular therapy the higher the probability of its efficacy. By straying from this logical framework, precision medicine opens the treatment of cancer to more uncertainty about the validity of these approaches to the resulting disease subcategories.[x] Thus, while contemporary medical practices explicitly describe some treatments as “personalized”, they ought not be viewed as inherently better founded than other therapies.[xi] A relevant contemporary case of precision medicine out of Norway focuses on the care of a patient with cancer between the ventricles of the heart and esophagus, which had failed to respond to the standard regimen of therapies over four years.[xii] In a last-ditch effort, the patient elected to pay out-of-pocket for an experimental immunotherapy (nivolumab) at a private hospital. He experienced marked improvements and a reduction in the size of the tumor. Understandably, the patient tried to pursue further rounds of nivolumab at a public hospital. However, the hospital initially declined to pay for it given the “lack of evidence from randomised clinical trials for this drug relating to this [patient’s] condition.”[xiii] In rebuttal to this claim, the patient countered that he was actually similar to a subpopulation of patients who responded in “open‐label, single arm, phase 2 studies on another immune therapy drug” (pembrolizumab).[xiv] Given this interpretation of the prior studies and the patient’s response, further rounds of nivolumab were approved. Had the patient not had improvements in the tumor’s size following a round of nivolumab, then pembrolizumab’s prior empirical evidence in isolation would have been insufficient, inductively speaking, to justify his continued use of nivolumab.[xv] The case demonstrates a shift in reasoning from the traditional induction to abduction. The phenomenon of ‘cancer improvement’ is considered causally linked to nivolumab and its underlying physiological mechanisms.[xvi] However, “the weakness of abductions is that there may always be some other better, unknown explanation for an effect. The patient may for example belong to a special subgroup that spontaneously improves, or the change may be a placebo effect. This does not mean, however, that abductive inferences cannot be strong or reasonable, in the sense that they can make a conclusion probable.”[xvii] To demonstrate the limitations of relying on the abductive standard in isolation, commentators have pointed out that side effects in precision medicine are hard to rule out as being related to the initial intervention itself unless trends from a group of patients are taken into consideration.[xviii] As artificial intelligence (AI) assists the development of precision medicine for oncology, this uncertainty ought to be taken into consideration. The implementation of AI has been crucial to the development of precision medicine by providing a way to combine large patient datasets or a single patient with a large number of unique variables with machine learning to recommend matches based on statistics and probability of success upon which practitioners can base medical recommendations.[xix] The AI is usually not establishing a causal relationship[xx] – it is predicting. So, as AI bleeds into medical devices, like brain-computer interfaces, the same cautions about using abductive reasoning alone should be carried over. II. Responsive Neurostimulation, AI, and Personalized Medicine Like precision medicine in cancer treatment, computer-brain interface technology similarly focuses on the individual patient through personalized settings. In order to properly expose the intersection of AI, precision medicine, abductive reasoning, and implantable neurotechnologies, the descriptions of adaptive deep brain stimulation systems need to deepen.[xxi] As a broad summary of adaptive deep brain stimulation, to provide a patient with the therapeutic stimulation, a neural signal, typically referred to as a local field potential,[xxii] must first be detected and then interpreted by the device. The main adaptive deep brain stimulation device with premarket approval, the NeuroPace Responsive Neurostimulation system, is used to treat epilepsy by detecting and storing “programmer-defined phenomena.”[xxiii] Providers can optimize the detection settings of the device to align with the patient’s unique electrographic seizures as well as personalize the reacting stimulation’s parameters.[xxiv] The provider adjusts the technology based on trial and error. One day machine learning algorithms will be able to regularly aid this process in myriad ways, such as by identifying the specific stimulation settings a patient may respond to ahead of time based on their electrophysiological signatures.[xxv] Either way, with AI or programmers, adaptive neurostimulation technologies are individualized and therefore operate in line with precision medicine rather than standard treatments based on large clinical trials. Contemporary neurostimulation devices are not usually sophisticated enough to be prominent in AI discussions where the topics of neural networks, deep learning, generative models, and self-attention dominate the conversation. However, implantable high-density electrocorticography arrays (a much more sensitive version than adaptive deep brain stimulation systems use) have been used in combination with neural networks to help patients with neurologic deficits from a prior stroke “speak” through a virtual avatar.[xxvi] In some experimental situations, algorithms are optimizing stimulation parameters with increasing levels of independence.[xxvii] An example of neurostimulation that is analogous to the use of nivolumab in Norway surrounds a patient in the United States who was experiencing both treatment-resistant OCD and temporal lobe epilepsy.[xxviii]Given the refractory nature of her epilepsy, implantation of an adaptive deep brain stimulation system was indicated. As a form of experimental therapy, her treatment-resistant OCD was also indicated for the off-label use of an adaptive deep brain stimulation set-up. Another deep brain stimulation lead, other than the one implanted for epilepsy, was placed in the patient’s right nucleus accumbens and ventral pallidum region given the correlation these nuclei had with OCD symptoms in prior research. Following this, the patient underwent “1) ambulatory, patient-initiated magnet-swipe storage of data during moments of obsessive thoughts; (2) lab-based, naturalistic provocation of OCD-related distress (naturalistic provocation task); and (3) lab-based, VR [virtual reality] provocation of OCD-related distress (VR provocation task).”[xxix] Such signals were used to identify when to deliver the therapeutic stimulation in order to counter the OCD symptoms. Thankfully, following the procedure and calibration the patient exhibited marked improvements in their OCD symptoms and recently shared her results publicly.[xxx] In both cases, there is a similar level of abductive justification for the efficacy of the delivered therapy. In the case study in which the patient was treated with adaptive deep brain stimulation, they at least had their neural activity tested in various settings to determine the optimum parameters for treatment to avoid them being based on guesswork. Additionally, the adaptive deep brain stimulation lead was already placed before the calibration trials were conducted, meaning that the patient had already taken on the bulk of the procedural risk before the efficacy could be determined. Such an efficacy test could have been replicated in the first patient’s cancer treatment, had it been biopsied and tested against the remaining immunotherapies in vitro. Yet, in the case of cancer with few options, one previous dose of a drug that appeared to work on the patient may justify further doses. However, as the Norwegian case presents, corroboration with known responses to a similar drug (from a clinical trial) could be helpful to validate the treatment strategy. (It should be noted that both patients were resigned to these last resort options regardless of the efficacy of treatment.) There are some elements of inductive logic seen with adaptive deep brain stimulation research in general. For example, abductively the focus could be that patient X’s stimulation parameters are different from patient Y’s and patient Z’s. In contrast, when grouped as subjects who obtained personalized stimulation, patients X, Y, and Z demonstrate an inductive aspect to this approach’s safety and/or efficacy. The OCD case holds plenty of abductive characteristics in line with precision medicine’s approach to treating cancer and as more individuals try the method, there will be additional data. With the gradual integration of AI into brain-computer interfaces in the name of efficacy, this reliance on abduction will continue, if not grow, over time. Moving forward, if a responsive deep brain stimulation treatment is novel and individualized (like the dose of nivolumab) and there is some other suggestion of efficacy (like clinical similarities to other patients in the literature), then it may justify insurance coverage for the investigative intervention, absent other unrelated reasons to deny it. III. Ethical Implications and Next Steps While AI’s use in oncology and neurology is not yet as prominent as its use in other fields (e.g., radiology), it appears to be on the horizon for both.[xxxi] AI can be found in both the functioning of the neurotechnologies as well as the implementation of precision medicine. The increasing use of AI may serve to further individualize both oncologic and neurological therapies. Given these implications and the handful of publications cited in this article, it is important to have a nuanced evaluation of how these treatments, which heavily rely on abductive justification, ought to be managed. The just use an abductive approach may be difficult as AI infused precision medicine is further pursued. At baseline, such technology relies on a level of advanced technology literacy among the general public and could exclude populations who lack access to basic technological infrastructure or know-how from participation.[xxxii] Even among nations with adequate infrastructure, as more patients seek out implantable neurotechnologies, which require robust healthcare resources, the market will favor patient populations that can afford this complex care.[xxxiii] If patients already have the means to pay for an initial dose/use of a precision medicine product out of pocket, should insurance providers be required to cover subsequent treatments?[xxxiv] That is, if a first dose of a cancer drug or a deep brain stimulator over its initial battery life is successful, patients may feel justified in having the costs of further treatments covered. The Norwegian patient’s experience implies there is a precedent for the idea that some public insurance companies ought to cover successful cancer therapies, however, insurance companies may not all see themselves as obligated to cover neurotechnologies that rely on personalized settings or that are based on precision/abductive research more than on clinical trials. CONCLUSION The fact that the cases outlined above rely on abductive style of reasoning implies that there may not be as strong a justification for coverage by insurance, as they are both experimental and individualized, when compared to the more traditional large clinical trials in which groups have the same or a standardized protocol (settings/doses). If a study is examining the efficacy of a treatment with a large cohort of patients or with different experimental groups/phases, insurance companies may conclude that the resulting symptom improvements are more likely to be coming from the devices themselves. A preference for inductive justification may take priority when ruling in favor of funding someone’s continued use of an implantable neurostimulator. There are further nuances to this discussion surrounding the classifications of these interventions as research versus clinical care that warrant future exploration, since such a distinction is more of a scale[xxxv] than binary and could have significant impacts on the “right-to-try” approach to experimental therapies in the United States.[xxxvi] Namely, given the inherent limitations of conducting large cohort trials for deep brain stimulation interventions on patients with neuropsychiatric disorders, surgically innovative frameworks that blend abductive and inductive methodologies, like with sham stimulation phases, have traditionally been used.[xxxvii] Similarly, for adaptive brain-computer interface systems, if there are no large clinical trials and instead only publications that demonstrate that something similar worked for someone else, then, in addition to the evidence that the first treatment/dose worked for the patient in question, the balance of reasoning would be valid and arguably justify insurance coverage. As precision approaches to neurotechnology become more common, frameworks for evaluating efficacy will be crucial both for insurance coverage and for clinical decision making. ACKNOWLEDGEMENT This article was originally written as an assignment for Dr. Francis Shen’s “Bioethics & AI” course at Harvard’s Center for Bioethics. I would like to thank Dr. Shen for his comments as well as my colleagues in the Lázaro-Muñoz Lab for their feedback. - [i] Jonathan Kimmelman and Ian Tannock, “The Paradox of Precision Medicine,” Nature Reviews. Clinical Oncology 15, no. 6 (June 2018): 341–42, https://doi.org/10.1038/s41571-018-0016-0. [ii] Henrik Vogt and Bjørn Hofmann, “How Precision Medicine Changes Medical Epistemology: A Formative Case from Norway,” Journal of Evaluation in Clinical Practice 28, no. 6 (December 2022): 1205–12, https://doi.org/10.1111/jep.13649. [iii] David Barrett and Ahtisham Younas, “Induction, Deduction and Abduction,” Evidence-Based Nursing 27, no. 1 (January 1, 2024): 6–7, https://doi.org/10.1136/ebnurs-2023-103873. [iv] Vogt and Hofmann, “How Precision Medicine Changes Medical Epistemology,” 1208. [v] Wireko Andrew Awuah et al., “Bridging Minds and Machines: The Recent Advances of Brain-Computer Interfaces in Neurological and Neurosurgical Applications,” World Neurosurgery, May 22, 2024, S1878-8750(24)00867-2, https://doi.org/10.1016/j.wneu.2024.05.104. [vi] Mark Riedl, “A Very Gentle Introduction to Large Language Models without the Hype,” Medium (blog), May 25, 2023, https://mark-riedl.medium.com/a-very-gentle-introduction-to-large-language-models-without-the-hype-5 f67941fa59e. [vii] David E. Burdette and Barbara E. Swartz, “Chapter 4 - Responsive Neurostimulation,” in Neurostimulation for Epilepsy, ed. Vikram R. Rao (Academic Press, 2023), 97–132, https://doi.org/10.1016/B978-0-323-91702-5.00002-5. [viii] Kimmelman and Tannock, 2018. [ix] Kimmelman and Tannock, 2018. [x] Simon Lohse, “Mapping Uncertainty in Precision Medicine: A Systematic Scoping Review,” Journal of Evaluation in Clinical Practice 29, no. 3 (April 2023): 554–64, https://doi.org/10.1111/jep.13789. [xi] Kimmelman and Tannock, “The Paradox of Precision Medicine.” [xii] Vogt and Hofmann, 1206. [xiii] Vogt and Hofmann, 1206. [xiv] Vogt and Hofmann, 1206. [xv] Vogt and Hofmann, 1207. [xvi] Vogt and Hofmann, 1207. [xvii] Vogt and Hofmann, 1207. [xviii] Vogt and Hofmann, 1210. [xix] Mehar Sahu et al., “Chapter Three - Artificial Intelligence and Machine Learning in Precision Medicine: A Paradigm Shift in Big Data Analysis,” in Progress in Molecular Biology and Translational Science, ed. David B. Teplow, vol. 190, 1 vols., Precision Medicine (Academic Press, 2022), 57–100, https://doi.org/10.1016/bs.pmbts.2022.03.002. [xx] Stefan Feuerriegel et al., “Causal Machine Learning for Predicting Treatment Outcomes,” Nature Medicine 30, no. 4 (April 2024): 958–68, https://doi.org/10.1038/s41591-024-02902-1. [xxi] Sunderland Baker et al., “Ethical Considerations in Closed Loop Deep Brain Stimulation,” Deep Brain Stimulation 3 (October 1, 2023): 8–15, https://doi.org/10.1016/j.jdbs.2023.11.001. [xxii] David Haslacher et al., “AI for Brain-Computer Interfaces,” 2024, 7, https://doi.org/10.1016/bs.dnb.2024.02.003. [xxiii] Burdette and Swartz, “Chapter 4 - Responsive Neurostimulation,” 103–4; “Premarket Approval (PMA),” https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpma/pma.cfm?id=P100026. [xxiv] Burdette and Swartz, “Chapter 4 - Responsive Neurostimulation,” 104. [xxv] Burdette and Swartz, 126. [xxvi] Sean L. Metzger et al., “A High-Performance Neuroprosthesis for Speech Decoding and Avatar Control,” Nature 620, no. 7976 (August 2023): 1037–46, https://doi.org/10.1038/s41586-023-06443-4. [xxvii] Hao Fang and Yuxiao Yang, “Predictive Neuromodulation of Cingulo-Frontal Neural Dynamics in Major Depressive Disorder Using a Brain-Computer Interface System: A Simulation Study,” Frontiers in Computational Neuroscience 17 (March 6, 2023), https://doi.org/10.3389/fncom.2023.1119685; Mahsa Malekmohammadi et al., “Kinematic Adaptive Deep Brain Stimulation for Resting Tremor in Parkinson’s Disease,” Movement Disorders 31, no. 3 (2016): 426–28, https://doi.org/10.1002/mds.26482. [xxviii] Young-Hoon Nho et al., “Responsive Deep Brain Stimulation Guided by Ventral Striatal Electrophysiology of Obsession Durably Ameliorates Compulsion,” Neuron 0, no. 0 (October 20, 2023), https://doi.org/10.1016/j.neuron.2023.09.034. [xxix] Nho et al. [xxx] Nho et al.; Erik Robinson, “Brain Implant at OHSU Successfully Controls Both Seizures and OCD,” OHSU News, accessed March 3, 2024, https://news.ohsu.edu/2023/10/25/brain-implant-at-ohsu-successfully-controls-both-seizures-and-ocd. [xxxi] Awuah et al., “Bridging Minds and Machines”; Haslacher et al., “AI for Brain-Computer Interfaces.” [xxxii] Awuah et al., “Bridging Minds and Machines.” [xxxiii] Sara Green, Barbara Prainsack, and Maya Sabatello, “The Roots of (in)Equity in Precision Medicine: Gaps in the Discourse,” Personalized Medicine 21, no. 1 (January 2024): 5–9, https://doi.org/10.2217/pme-2023-0097. [xxxiv] Green, Prainsack, and Sabatello, 7. [xxxv] Robyn Bluhm and Kirstin Borgerson, “An Epistemic Argument for Research-Practice Integration in Medicine,” The Journal of Medicine and Philosophy: A Forum for Bioethics and Philosophy of Medicine 43, no. 4 (July 9, 2018): 469–84, https://doi.org/10.1093/jmp/jhy009. [xxxvi] Vijay Mahant, “‘Right-to-Try’ Experimental Drugs: An Overview,” Journal of Translational Medicine 18 (June 23, 2020): 253, https://doi.org/10.1186/s12967-020-02427-4. [xxxvii] Michael S. Okun et al., “Deep Brain Stimulation in the Internal Capsule and Nucleus Accumbens Region: Responses Observed during Active and Sham Programming,” Journal of Neurology, Neurosurgery & Psychiatry 78, no. 3 (March 1, 2007): 310–14, https://doi.org/10.1136/jnnp.2006.095315. (shrink)

Decision-making assisted by algorithms developed by machine learning is increasingly determining our lives. Unfortunately, full opacity about the process is the norm. Can transparency contribute to restoring accountability for such systems? Several objections are examined: the loss of privacy when data sets become public, the perverse effects of disclosure of the very algorithms themselves, the potential loss of competitive edge, and the limited gains in answerability to be expected since sophisticated algorithms are inherently opaque. It is (...) concluded that transparency is certainly useful, but only up to a point: extending it to the public at large is normally not to be advised. Moreover, in order to make algorithmic decisions understandable, models of machine learning to be used should either be interpreted ex post or be interpretable by design ex ante. (shrink)

Algorithm Decision Making (ADM) systems designed to augment or automate human decision-making have the potential to produce better decisions while also freeing up human time and attention for other pursuits. For this potential to be realised, however, algorithmic decisions must be sufficiently aligned with human goals and interests. We take a Principal-Agent (P-A) approach to the questions of ADM alignment and trust. In a broad sense, ADM is beneficial if and only if human principals can trust algorithmic agents (...) to act faithfully on their behalf. This mirrors the challenge of facilitating P-A relationships among humans, but the peculiar nature of human-machine interaction also raises unique issues. The problem of asymmetric information is omnipresent but takes a different form in the context of ADM. Although the decision-making machinery of an algorithmic agent can in principle be laid bare for all to see, the sheer complexity of ADM systems based on deep learning models prevents straightforward monitoring. We draw on literature from economics and political science to argue that the problem of trust in ADM systems should be addressed at the level of institutions. Although the dyadic relationship between human principals and algorithmic agents is our ultimate concern, cooperation at this level must rest against an institutional environment which allows humans to effectively evaluate and choose among algorithmic alternatives. (shrink)

Machine learning algorithms are expected to improve referral decisions. In this article I discuss the legitimacy of deferring referral decisions in primary care to recommendations from such algorithms. The standard justification for introducing algorithmic decision procedures to make referral decisions is that they are more accurate than the available practitioners. The improvement in accuracy will ensure more efficient use of scarce health resources and improve patient care. In this article I introduce a proceduralist framework for discussing the (...) legitimacy of algorithmic referral decisions and I argue that in the context of referral decisions the legitimacy of an algorithmic decision procedure can be fully accounted for in terms of the instrumental values of accuracy and fairness. I end by considering how my discussion of procedural algorithmic legitimacy relates to the debate on algorithmic fairness. (shrink)

Algorithms influence every facet of modern life: criminal justice, education, housing, entertainment, elections, social media, news feeds, work… the list goes on. Delegating important decisions to machines, however, gives rise to deep moral concerns about responsibility, transparency, freedom, fairness, and democracy. Algorithms and Autonomy connects these concerns to the core human value of autonomy in the contexts of algorithmic teacher evaluation, risk assessment in criminal sentencing, predictive policing, background checks, news feeds, ride-sharing platforms, social media, and election interference. (...) Using these case studies, the authors provide a better understanding of machine fairness and algorithmic transparency. They explain why interventions in algorithmic systems are necessary to ensure that algorithms are not used to control citizens' participation in politics and undercut democracy. This title is also available as Open Access on Cambridge Core. (shrink)

Algorithms increasingly make managerial decisions that people used to make. Perceptions of algorithms, regardless of the algorithms' actual performance, can significantly influence their adoption, yet we do not fully understand how people perceive decisions made by algorithms as compared with decisions made by humans. To explore perceptions of algorithmic management, we conducted an online experiment using four managerial decisions that required either mechanical or human skills. We manipulated the decision-maker, and measured perceived fairness, trust, and (...) emotional response. With the mechanical tasks, algorithmic and human-made decisions were perceived as equally fair and trustworthy and evoked similar emotions; however, human managers' fairness and trustworthiness were attributed to the manager's authority, whereas algorithms' fairness and trustworthiness were attributed to their perceived efficiency and objectivity. Human decisions evoked some positive emotion due to the possibility of social recognition, whereas algorithmic decisions generated a more mixed response – algorithms were seen as helpful tools but also possible tracking mechanisms. With the human tasks, algorithmic decisions were perceived as less fair and trustworthy and evoked more negative emotion than human decisions. Algorithms' perceived lack of intuition and subjective judgment capabilities contributed to the lower fairness and trustworthiness judgments. Positive emotion from human decisions was attributed to social recognition, while negative emotion from algorithmic decisions was attributed to the dehumanizing experience of being evaluated by machines. This work reveals people's lay concepts of algorithmic versus human decisions in a management context and suggests that task characteristics matter in understanding people's experiences with algorithmic technologies. (shrink)

Recently, a multinomial process tree model was developed to measure an agent’s consequence sensitivity, norm sensitivity, and generalized inaction/action preferences when making moral decisions (CNI model). However, the CNI model presupposed that an agent considersconsequences—norms—generalizedinaction/actionpreferences sequentially, which is untenable based on recent evidence. Besides, the CNI model generates parameters at the group level based on binary categorical data. Hence, theC/N/Iparameters cannot be used for correlation analyses or other conventional research designs. To solve these limitations, we developed the CAN algorithm to (...) computenormandconsequencesensitivities and overallaction/inactionpreferences algebraically in a parallel manner. We re-analyzed the raw data of the original CNI model to test the methodological predictions. Our results demonstrate that: (1) theCparameter is approximately equal between the CNI model and CAN algorithm; (2) theNparameter under the CNI model approximately equalsN/(1 −C) under the CAN algorithm; (3) theIparameter andAparameter are reversed around 0.5 – the larger theIparameter, the more the generalized inactionversusaction preference and the larger theAparameter, the more overall actionversusinaction preference; (4) tests of differences in parameters between groups with the CNI model and CAN algorithm led to almost the same statistical conclusion; (5) parameters from the CAN algorithm can be used for correlational analyses and multiple comparisons, and this is an advantage over the parameters from the CNI model. The theoretical and methodological implications of our study were also discussed. (shrink)

The stakes associated with an algorithmic decision are often said to play a role in determining whether the decision engenders a right to an explanation. More specifically, “high stakes” decisions are often said to engender such a right to explanation whereas “low stakes” or “non-high” stakes decisions do not. While the overall gist of these ideas is clear enough, the details are lacking. In this paper, we aim to provide these details through a detailed investigation of what we (...) will call the “Simple Stakes Thesis”. The Simple Stakes Thesis, as it will turn out, is too simple. For even if the stakes associated with a specific one-off decision are low—and hence does not engender a right to an explanation—such decisions may nevertheless form part of a high stakes pattern or aggregate of decisions. In such cases, we argue, even a low stakes decision may engender a right to explanation. Not only does this show that the right to explanation is more demanding than so far recognized. It also shows that the stakes thesis is significantly harder to apply in practice. (shrink)

Objectives Medical futility at the end of life is a growing challenge to medicine. The goals of the authors were to elucidate how clinicians define futility, when they perceive life-sustaining treatment (LST) to be futile, how they communicate this situation and why LST is sometimes continued despite being recognised as futile. Methods The authors reviewed ethics case consultation protocols and conducted semi-structured interviews with 18 physicians and 11 nurses from adult intensive and palliative care units at a tertiary hospital (...) in Germany. The transcripts were subjected to qualitative content analysis. Results Futility was identified in the majority of case consultations. Interviewees associated futility with the failure to achieve goals of care that offer a benefit to the patient's quality of life and are proportionate to the risks, harms and costs. Prototypic examples mentioned are situations of irreversible dependence on LST, advanced metastatic malignancies and extensive brain injury. Participants agreed that futility should be assessed by physicians after consultation with the care team. Intensivists favoured an indirect and stepwise disclosure of the prognosis. Palliative care clinicians focused on a candid and empathetic information strategy. The reasons for continuing futile LST are primarily emotional, such as guilt, grief, fear of legal consequences and concerns about the family's reaction. Other obstacles are organisational routines, insufficient legal and palliative knowledge and treatment requests by patients or families. Conclusion Managing futility could be improved by communication training, knowledge transfer, organisational improvements and emotional and ethical support systems. The authors propose an algorithm for end-of-life decision making focusing on goals of treatment. (shrink)

Algorithmic decision-making increasingly shapes people's daily lives. Given that such autonomous systems can cause severe harm to individuals and social groups, fairness concerns have arisen. A human-centric approach demanded by scholars and policymakers requires considering people's fairness perceptions when designing and implementing algorithmic decision-making. We provide a comprehensive, systematic literature review synthesizing the existing empirical insights on perceptions of algorithmic fairness from 58 empirical studies spanning multiple domains and scientific disciplines. Through thorough coding, we systemize the current empirical (...) literature along four dimensions: (1) algorithmic predictors, (2) human predictors, (3) comparative effects (human decision-making vs. algorithmic decision-making), and (4) consequences of algorithmic decision-making. While we identify much heterogeneity around the theoretical concepts and empirical measurements of algorithmic fairness, the insights come almost exclusively from Western-democratic contexts. By advocating for more interdisciplinary research adopting a society-in-the-loop framework, we hope our work will contribute to fairer and more responsible algorithmic decision-making. (shrink)

Artificial intelligence plays an increasingly important role in legal disputes, influencing not only the reality outside the court but also the judicial decision-making process itself. While it is clear why judges may generally benefit from technology as a tool for reducing effort costs or increasing accuracy, the presence of technology in the judicial process may also affect the public perception of the courts. In particular, if individuals are averse to adjudication that involves a high degree of automation, particularly given (...) fairness concerns, then judicial technology may yield lower benefits than expected. However, the degree of aversion may well depend on how technology is used, i.e., on the timing and strength of judicial reliance on algorithms. Using an exploratory survey, we investigate whether the stage in which judges turn to algorithms for assistance matters for individual beliefs about the fairness of case outcomes. Specifically, we elicit beliefs about the use of algorithms in four different stages of adjudication: (i) information acquisition, (ii) information analysis, (iii) decision selection, and (iv) decision implementation. Our analysis indicates that individuals generally perceive the use of algorithms as fairer in the information acquisition stage than in other stages. However, individuals with a legal profession also perceive automation in the decision implementation stage as less fair compared to other individuals. Our findings, hence, suggest that individuals do care about how and when algorithms are used in the courts. (shrink)

Many artificial intelligence (AI) systems currently used for decision-making are opaque, i.e., the internal factors that determine their decisions are not fully known to people due to the systems’ computational complexity. In response to this problem, several researchers have argued that human decision-making is equally opaque and since simplifying, reason-giving explanations (rather than exhaustive causal accounts) of a decision are typically viewed as sufficient in the human case, the same should hold for algorithmic decision-making. Here, I (...) contend that this argument overlooks that human decision-making is sometimes significantly more transparent and trustworthy than algorithmic decision-making. This is because when people explain their decisions by giving reasons for them, this frequently prompts those giving the reasons to govern or regulate themselves so as to think and act in ways that confirm their reason reports. AI explanation systems lack this self-regulative feature. Overlooking it when comparing algorithmic and human decision-making can result in underestimations of the transparency of human decision-making and in the development of explainable AI that may mislead people by activating generally warranted beliefs about the regulative dimension of reason-giving. (shrink)

The rapidly increasing role of automation throughout the economy, culture and our personal lives has generated a large literature on the risks of algorithmic decision-making, particularly in high-stakes legal settings. Algorithmic tools are charged with bias, shrouded in secrecy, and frequently difficult to interpret. However, these criticisms have tended to focus on particular implementations, specific predictive techniques, and the idiosyncrasies of the American legal-regulatory regime. They do not address the more fundamental unease about the prospect that we might one (...) day replace judges with algorithms, no matter how fair, transparent, and intelligible they become. The aim of this paper is to propose an account of the source of that unease, and to evaluate its plausibility. I trace foundational unease with algorithmic decision-making in the law to the powerful intuition that there is a basic moral and legal difference between showing that something is true of many people just like you and showing that it is true of you. Human judgment attends to the exception; automation insists on blindly applying the rule. I show how this intuitive thought is connected to both epistemological arguments about the value of statistical evidence, as well as to court-centered conceptions of the rule of law. Unease with algorithmic decision-making in the law thus draws on an intuitive principle that underpins a disparate range of views in legal philosophy. This suggests the principle is deeply ingrained. Nonetheless, I argue that the powerful intuition is not as decisive as it may seem, and indeed runs into significant epistemological and normative challenges. At an epistemological level, I show how concerns about statistical evidence's ability to track the truth can be resolved by adopting a probabilistic, rather than modal, conception of truth-tracking. At a normative level, commitment to highly individualized decision-making co-exists with equally ingrained and competing principles, such as consistent application of law. This suggests that the “rule of law” may not identify a discrete set of institutional arrangements, as proponents of a court-centric conception would have it, but rather a more loosely defined set of values that could potentially be operationalized in multiple ways, including through some level of algorithmic adjudication. Although the prospect of replacing judges with algorithms is indeed unsettling, it does not necessarily entail unreasonable verdicts or an attack on the rule of law. (shrink)

Decision-making assisted by algorithms developed by machine learning is increasingly determining our lives. Unfortunately, full opacity about the process is the norm. Would transparency contribute to restoring accountability for such systems as is often maintained? Several objections to full transparency are examined: the loss of privacy when datasets become public, the perverse effects of disclosure of the very algorithms themselves, the potential loss of companies’ competitive edge, and the limited gains in answerability to be expected since sophisticated (...) algorithms usually are inherently opaque. It is concluded that, at least presently, full transparency for oversight bodies alone is the only feasible option; extending it to the public at large is normally not advisable. Moreover, it is argued that algorithmic decisions preferably should become more understandable; to that effect, the models of machine learning to be employed should either be interpreted ex post or be interpretable by design ex ante. (shrink)

Background Doctors and the Sri Lanka Medical Association recognise the importance of do not attempt cardiopulmonary resuscitation decisions and disclosure; however, few previous studies exist examining these practices in Sri Lanka. Resuscitation decisions have seen significant changes in the UK in recent years, with a legal imperative for clear communication and a move to understand patients’ preferred outcomes before recommending clinical guidance. Methods Participants from two Sri Lankan hospitals were selected purposively to represent a range of specialties and seniorities for (...) semi-structured interview. Results Fifteen participants of varying seniorities were recruited. Practice of do not resuscitate and informing patients is highly variable; there is no definitive guidance published on best practice of these issues in Sri Lanka. Participants felt that inpatients were generally not aware of their medical conditions or treatments. With the poor social and palliative care service provision in Sri Lanka comes a pressure to involve families in the patient care, particularly at the end of life. This feeds into a culture where patient autonomy plays a subordinate role to family involvement. Participants understand the ethical need for do not resuscitate, however, it is viewed to be a consideration for when the patient is close to death. Conclusion Do not resuscitate does not appear to be a prominent feature of end-of-life care in Sri Lanka and thus consideration of the appropriateness of resuscitation is commonly left late in a patient’s deterioration. Education, inclination and respect for doctors are suggested barriers to effective inclusion of patients in their healthcare and end-of-life decisions. (shrink)