The use of advanced AI and data-driven automation in the public sector poses several organisational, practical, and ethical challenges. One that is easy to underestimate is automation bias, which, in turn, has underappreciated legal consequences. Automation bias is an attitude in which the operator of an autonomous system will defer to its outputs to the point where the operator overlooks or ignores evidence that the system is failing. The legal problem arises when statutory office-holders (or (...) their employees) either fetter their discretion to in-house algorithms or improperly delegate their discretion to third-party software developers—something automation bias may facilitate. A synthesis of previous research suggests an easy way to mitigate the risks of automation bias and its potential legal ramifications is for those responsible for procurement decisions to adhere to a simple checklist that ensures that the pitfalls of automation are avoided as much as possible. (shrink)

Recent advances in artificial intelligence offer many beneficial prospects. However, concerns have been raised about the opacity of decisions made by these systems, some of which have turned out to be biased in various ways. This article makes a contribution to a growing body of literature on how to make systems for automated decision-making more transparent, explainable, and fair by drawing attention to and further elaborating a distinction first made by Nozick between first-level bias in the application of standards (...) and second-level bias in the choice of standards, as well as a second distinction between discrimination and arbitrariness. Applying the typology developed, a number of illuminating observations are made. First, it is observed that some reported bias in automated decision-making is first-level arbitrariness, which can be alleviated by explainability techniques. However, such techniques have only a limited potential to alleviate first-level discrimination. Second, it is argued that second-level arbitrariness is probably quite common in automated decision-making. In contrast to first-level arbitrariness, however, second-level arbitrariness is not straightforward to detect automatically. Third, the prospects for alleviating arbitrariness are discussed. It is argued that detecting and alleviating second-level arbitrariness is a profound problem because there are many contrasting and sometimes conflicting standards from which to choose, and even when we make intentional efforts to choose standards for good reasons, some second-level arbitrariness remains. (shrink)

The use of predictive systems has become wider with the development of related computational methods, and the evolution of the sciences in which these methods are applied Solon and Selbst and Pedreschi et al.. The referred methods include machine learning techniques, face and/or voice recognition, temperature mapping, and other, within the artificial intelligence domain. These techniques are being applied to solve problems in socially and politically sensitive areas such as crime prevention and justice management, crowd management, and emotion analysis, just (...) to mention a few. However, dissimilar predictions can be found nowadays as the result of the application of these methods resulting in misclassification, for example for the case of conviction risk assessment Office of Probation and Pretrial Services or decision-making process when designing public policies Lange. The goal of this paper is to identify current gaps on fairness achievement within the context of predictive systems in artificial intelligence by analyzing available academic and scientific literature up to 2020. To achieve this goal, we have gathered available materials at the Web of Science and Scopus from last 5 years and analyzed the different proposed methods and their results in relation to the bias as an emergent issue in the Artificial Intelligence field of study. Our tentative conclusions indicate that machine learning has some intrinsic limitations which are leading to automate the bias when designing predictive algorithms. Consequently, other methods should be explored; or we should redefine the way current machine learning approaches are being used when building decision making/decision support systems for crucial institutions of our political systems such as the judicial system, just to mention one. (shrink)

Computers are used to make decisions in an increasing number of domains. There is widespread agreement that some of these uses are ethically problematic. Far less clear is where ethical problems arise, and what might be done about them. This paper expands and defends the Ethical Gravity Thesis: ethical problems that arise at higher levels of analysis of an automated decision-making system are inherited by lower levels of analysis. Particular instantiations of systems can add new problems, but not ameliorate more (...) general ones. We defend this thesis by adapting Marr’s famous 1982 framework for understanding information-processing systems. We show how this framework allows one to situate ethical problems at the appropriate level of abstraction, which in turn can be used to target appropriate interventions. (shrink)

Automated decision making for sentencing is the use of a software algorithm to analyse a convicted offender’s case and deliver a sentence. This chapter reviews the moral arguments for and against employing automated decision making for sentencing and finds that its use is in principle morally permissible. Specifically, it argues that well-designed automated decision making for sentencing will better approximate the just sentence than human sentencers. Moreover, it dismisses common concerns about transparency, privacy and bias as unpersuasive or inapplicable. (...) The chapter also notes that moral disagreement about theories of just sentencing are plausibly resolved by applying the principle of maximising expected moral choiceworthiness, and that automated decision making is better suited to the resulting ensemble model. Finally, the chapter considers the challenge posed by penal populism. The dispiriting conclusion is that although it is in theory morally desirable to use automated decision-making for criminal sentencing, it may well be the case that we ought not to try. (shrink)

By focusing on patents for recent algorithms that incorporate publicly available data to yield automated racial and ethnic classification schemes, I provide a glimpse into how engineers and programmers understand and define racial and ethnic categories. Patents provide insights into how engineers and programmers encode assumptions about identity and behavior, due to disclosure provisions required by US patent law; similar requirements are present in patent laws throughout the world. Such disclosures provide insights that are otherwise unavailable for most proprietary assets. (...) I further discuss how such classification tools, especially when combined with advertisement-based platforms such as social media, can amplify systemic racism present in the underlying data sets. (shrink)

This article probes the practical ethical implications of AI system design by reconsidering the important topic of bias in the datasets used to train autonomous intelligent systems. The discussion draws on recent work concerning behaviour-guiding technologies, and it adopts a cautious form of technological utopianism by assuming it is potentially beneficial for society at large if AI systems are designed to be comparatively free from the biases that characterise human behaviour. However, the argument presented here critiques the common well-intentioned (...) requirement that, in order to achieve this, all such datasets must be debiased prior to training. By focusing specifically on gender-bias in Neural Machine Translation systems, three automated strategies for the removal of bias are considered – downsampling, upsampling, and counterfactual augmentation – and it is shown that systems trained on datasets debiased using these approaches all achieve general translation performance that is much worse than a baseline system. In addition, most of them also achieve worse performance in relation to metrics that quantify the degree of gender bias in the system outputs. By contrast, it is shown that the technique of domain adaptation can be effectively deployed to debias existing NMT systems after they have been fully trained. This enables them to produce translations that are quantitatively far less biased when analysed using gender-based metrics, but which also achieve state-of-the-art general performance. It is hoped that the discussion presented here will reinvigorate ongoing debates about how and why bias can be most effectively reduced in state-of-the-art AI systems. (shrink)

The logical intuition hypothesis proposes a difference in the strength between logical and heuristic intuitions. The labels of logical and heuristic intuitions are exclusive to conventional reasoning research. This paper reports the result of testing intuition strength using the dot-probe methodology in a novel multiplication paradigm. Here, “logical intuition” and “heuristic intuition” were relabeled as “weaker intuition” (-1 × 5 = 5) and “stronger intuition” (1 × 5 = 5), respectively, to assess the assumptions about the difference in the strength (...) of “Chinese multiplication mnemonics” intuitions. The dot-probe paradigm was utilized in the following experiments to measure attentional bias. This was assessed by calculating the difference in reaction time between incongruent (the target location is opposite to the probe location) and congruent (the target location is the same as the probe location) conditions. Experiment 1 found attentional bias directed attention to the correct answer for both weaker and stronger intuitions. Experiment 2 replicated and extended Experiment 1 with a secondary task to manipulate high and low cognitive loads. Consequently, stronger intuitions were unaffected by cognitive load and participants were biased toward the correct answer, whereas attentional bias for weaker intuitions only occurred in low load conditions. These results revealed that although both intuitions can guide attention, stronger intuition may guide attention bias in a high-load situation due to its robust automation. Our findings extend the logical intuition hypothesis that proposes a difference in the automated strengths of intuitions to mathematical and analytic thinking. (shrink)

Purpose The online economy has not resolved the issue of racial bias in its applications. While algorithms are procedures that facilitate automated decision-making, or a sequence of unambiguous instructions, bias is a byproduct of these computations, bringing harm to historically disadvantaged populations. This paper argues that algorithmic biases explicitly and implicitly harm racial groups and lead to forms of discrimination. Relying upon sociological and technical research, the paper offers commentary on the need for more workplace diversity within high-tech (...) industries and public policies that can detect or reduce the likelihood of racial bias in algorithmic design and execution. Design/methodology/approach The paper shares examples in the US where algorithmic biases have been reported and the strategies for explaining and addressing them. Findings The findings of the paper suggest that explicit racial bias in algorithms can be mitigated by existing laws, including those governing housing, employment, and the extension of credit. Implicit, or unconscious, biases are harder to redress without more diverse workplaces and public policies that have an approach to bias detection and mitigation. Research limitations/implications The major implication of this research is that further research needs to be done. Increasing the scholarly research in this area will be a major contribution in understanding how emerging technologies are creating disparate and unfair treatment for certain populations. Practical implications The practical implications of the work point to areas within industries and the government that can tackle the question of algorithmic bias, fairness and accountability, especially African-Americans. Social implications The social implications are that emerging technologies are not devoid of societal influences that constantly define positions of power, values, and norms. Originality/value The paper joins a scarcity of existing research, especially in the area that intersects race and algorithmic development. (shrink)

This letter contains suggestions for editors and software providers to help avoid affiliation bias in the initial and concluding stages of the peer review process. Submission management systems have a responsibility to ensure protection against affiliation bias. This can be achieved by automatically withholding the author’s identity and affiliation information from all editors, including the Editor-in-Chief, until a decision about publication has been made. Journals relying on email-based submissions are in a more difficult situation. Not having external support (...) in confirming the submitting author’s identity, this dilemma can be resolved through the implementation of strict instructions and guidelines concerning manuscript submission and handling. In this case the “human factor” cannot be fully eliminated, while automated processes can only be changed intentionally and with a certain delay. Therefore, with the suggested steps, submission management systems could become a strong guard against unconscious bias while also making conscious bias more difficult to impact the peer review process. (shrink)

In the article ‘Testimonial injustice in medical machine learning’, Pozzi argues that the prescription drug monitoring programme (PDMP) leads to testimonial injustice as physicians are more inclined to trust the PDMP’s risk scores over the patient’s own account of their medication history.1 Pozzi further develops this argument by discussing how credibility shifts from patients to machine learning (ML) systems that are supposedly neutral. As a result, a sense of distrust is now formed between patients and physicians. While there are merits (...) to Pozzi’s main argument of epistemic injustice caused by PDMPs, Pozzi mentions but ultimately glosses over the problem of automation bias. In this commentary, I will discuss automation bias and the affect it has on clinical decision making as well as a technical problem exacerbated by the usage of PDMPs that can potentially cause physical harms. It is reiterated in the article that the confidence physicians have in the PDMP’s risk scores over the patient’s testimony leads to misplaced trust in the ML systems. What Pozzi describes here is known as automation bias, which occurs when there is an over-reliance on ML systems. …. (shrink)

In my paper entitled ‘Testimonial injustice in medical machine learning’,1 I argued that machine learning (ML)-based Prediction Drug Monitoring Programmes (PDMPs) could infringe on patients’ epistemic and moral standing inflicting a testimonial injustice.2 I am very grateful for all the comments the paper received, some of which expand on it while others take a more critical view. This response addresses two objections raised to my consideration of ML-induced testimonial injustice in order to clarify the position taken in the paper. The (...) first maintains that my critical stance toward ML-based PDMPs idealises standard medical practice. Moreover, it claims that the ML-induced testimonial injustice I discuss is not substantially different from situations in which it emerges in human–human interactions. The second claims that my analysis does not establish a link to issues of automation bias, even if these are to be considered the core of testimonial injustice in ML. In the following, I address each objection in turn. Gillett3 argues that my critical stance towards using risk prediction tools such as PDMPs implies the idealisation of standard (ie, non-ML-mediated) modes of clinical practice. Considering certain uses of ML in a different setting, that is, psychiatry, the author goes as far as claiming that ‘traditional models of clinical practice in psychiatry are far from a utopia, free from epistemic injustice, which Pozzi’s argument risks proposing’. Since this statement does not represent what I intend to suggest, I am glad to have the possibility to clarify …. (shrink)

Artificially intelligent machines are different in kind from all previous machines and tools. While many are used for relatively benign purposes, the types of artificially intelligent machines that we should care about, the ones that are worth focusing on, are the machines that purport to replace humans entirely and thereby engage in what Brian Cantwell Smith calls “judgment.” As impressive as artificially intelligent machines are, their abilities are still derived from humans and as such lack the sort of normative commitments (...) that humans have. So while artificially intelligent machines possess a great capacity for “reckoning,” to use Smith’s terminology, i.e., a calculative prowess of extraordinary utility and importance, they still lack the kind of considered human judgment that accompanies the ethical commitment and responsible action we humans must ultimately aspire toward. But there is a perfect technological storm brewing. Artificially intelligent machines are analogous to a perfect storm in that such machines involve the convergence of a number of factors that threaten our ability to behave ethically and maintain meaningful human control over the outcomes of processes involving artificial intelligence. I argue that the storm in the context of artificially intelligent machines makes us vulnerable to moral complacency. That is, this perfect technological storm is capable of lulling people into a state in which they abdicate responsibility for decision-making and behaviour precipitated by the use of artificially intelligent machines, a state that I am calling “moral complacency.” I focus on three salient problems that converge to make us especially vulnerable to becoming morally complacent and losing meaningful human control. The first problem is that of transparency/opacity. The second problem is that of overtrust in machines, often referred to as the automation bias. The third problem is that of ascribing responsibility. I examine each of these problems and how together they threaten to render us morally complacent. (shrink)

I argue that, given the way that AI models work and the way that ordinary human rationality works, it is very likely that people are anthropomorphizing AI, with potentially serious consequences. There are good reasons to doubt that LLMs have anything like human understanding, and even if they have representations or meaningful contents in some sense, these are unlikely to correspond to our ordinary understanding of natural language. However, there are natural, and in some ways quite rational, pressures to anthropomorphize (...) or personify LLMs and other AI systems in biased ways. This includes not only the classical or obvious ways of personifying AI — taking them to be sentient or have consciousness and understanding — but also taking AI to simulate or function like understanding, track the meaning of our language and our reasoning or logic, or have a model of the world our language is about. These more subtle or unobvious forms of anthropomorphism or personification can contribute to automation bias, AI Hype, and all their attendant risks, and encourage stronger anthropomorphisms as multi-modal AI assistants are developed further. (shrink)

The increasing use of computer-cont rolled automation systems has brought with it a bias towards a purely scientific approach to work. This tends to undermine the significance of experiential knowledge and sensory perception when working with highly automated processes. This paper argues for a recognition of the importance of subjectifying action in carrying out work practices. Without it, complex technical systems cannot be effectively operated. Moreover, the contradictory demands that arise for workers could have negative consequences in terms (...) of work burdens and health risks. (shrink)

Algorithms can now identify patterns and correlations in the (big) datasets, and predict outcomes based on those identified patterns and correlations with the use of machine learning techniques and big data, decisions can then be made by algorithms themselves in accordance with the predicted outcomes. Yet, algorithms can inherit questionable values from the datasets and acquire biases in the course of (machine) learning, and automated algorithmic decision-making makes it more difficult for people to see algorithms as biased. While researchers have (...) taken the problem of algorithmic bias seriously, but the current discussion on algorithmic fairness tends to conceptualize ‘fairness’ in algorithmic fairness primarily as a technical issue and attempts to implement pre-existing ideas of ‘fairness’ into algorithms. In this paper, I show that such a view of algorithmic fairness as technical issue is unsatisfactory for the type of problem algorithmic fairness presents. Since decisions on fairness measure and the related techniques for algorithms essentially involve choices between competing values, ‘fairness’ in algorithmic fairness should be conceptualized first and foremost as a political issue, and it should be (re)solved by democratic communication. The aim of this paper, therefore, is to explicitly reconceptualize algorithmic fairness as a political question and suggest the current discussion of algorithmic fairness can be strengthened by adopting the accountability for reasonableness framework. (shrink)

When agents insert technological systems into their decision-making processes, they can obscure moral responsibility for the results. This can give rise to a distinct moral wrong, which we call “agency laundering.” At root, agency laundering involves obfuscating one’s moral responsibility by enlisting a technology or process to take some action and letting it forestall others from demanding an account for bad outcomes that result. We argue that the concept of agency laundering helps in understanding important moral problems in a number (...) of recent cases involving automated, or algorithmic, decision-systems. We apply our conception of agency laundering to a series of examples, including Facebook’s automated advertising suggestions, Uber’s driver interfaces, algorithmic evaluation of K-12 teachers, and risk assessment in criminal sentencing. We distinguish agency laundering from several other critiques of information technology, including the so-called “responsibility gap,” “bias laundering,” and masking. (shrink)

Automated technologies populating today’s online world rely on social expectations about how “smart” they appear to be. Algorithmic processing, as well as bias and missteps in the course of their development, all come to shape a cultural realm that in turn determines what they come to be about. It is our contention that a robust analytical frame could be derived from culturally driven Science and Technology Studies while focusing on Callon’s concept of translation. Excitement and apprehensions must find a (...) specific language to move past a state of latency. Translations are thus contextual and highly performative, transforming justifications into legitimate claims, translators into discursive entrepreneurs, and power relations into new forms of governance and governmentality. In this piece, we discuss three cases in which artificial intelligence was deciphered to the public: the Montreal Declaration for a Responsible Development of Artificial Intelligence, held as a prime example of how stakeholders manage to establish the terms of the debate on ethical artificial intelligence while avoiding substantive commitment; Mark Zuckerberg’s 2018 congressional hearing, where he construed machine learning as the solution to the many problems the platform might encounter; and the normative renegotiations surrounding the gradual introduction of “killer robots” in military engagements. Of interest are not only the rational arguments put forward, but also the rhetorical maneuvers deployed. Through the examination of the ramifications of these translations, we intend to show how they are constructed in face of and in relation to forms of criticisms, thus revealing the highly cybernetic deployment of artificial intelligence technologies. (shrink)

Machine learning-based AI algorithms lack transparency. In this article, I offer an interpretation of AI’s explainability problem and highlight its ethical saliency. I try to make the case for the legal enforcement of a strong explainability requirement: human organizations which decide to automate decision-making should be legally obliged to demonstrate the capacity to explain and justify the algorithmic decisions that have an impact on the wellbeing, rights, and opportunities of those affected by the decisions. This legal duty can be derived (...) from the demands of Rawlsian public reason. In the second part of the paper, I try to show that the argument from the limitations of human cognition fails to get AI off the hook of public reason. Against a growing trend in AI ethics, my main argument is that the analogy between human minds and artificial neural networks fails because it suffers from an atomistic bias which makes it blind to the social and institutional dimension of human reasoning processes. I suggest that developing interpretive AI algorithms is not the only possible answer to the explainability problem; social and institutional answers are also available and in many cases more trustworthy than techno-scientific ones. (shrink)

Agnostic analyses of unique video material from a Mother and Baby Unit were carried out to investigate the usefulness of such analyses to the unit. The goal was to improve outcomes: the health of mothers and their babies. The method was to implement a learning machine that becomes more useful over time and over task. A feasible set-up is here described, with the purpose of producing intelligible and useful results to healthcare professionals at the unit by means of a vision (...) processing pipeline, grouped together with multi-modal capabilities of handling annotations and audio. Algorithmic bias turned out to be an obstacle that could only partly be handled by modern pipelines for automated feature analysis. The professional use of complex quantitative scoring for various mental health-related assessments further complicated the automation of laborious tasks. Activities during the MBU stay had previously been shown to decrease psychiatric symptoms across diagnostic groups. The implementation and first set of experiments on a learning machine for the unit produced the first steps toward explaining why this is so, in turn enabling decision support to staff about what to do more and what to do less of. (shrink)

Biometric technologies are becoming more pervasive in the workplace, augmenting managerial processes such as hiring, monitoring and terminating employees. Until recently, these devices consisted mainly of GPS tools that track location, software that scrutinizes browser activity and keyboard strokes, and heat/motion sensors that monitor workstation presence. Today, however, a new generation of biometric devices has emerged that can sense, read, monitor and evaluate the affective state of a worker. More popularly known by its commercial moniker, Emotional AI, the technology stems (...) from advancements in affective computing. But whereas previous generations of biometric monitoring targeted the exterior physical body of the worker, concurrent with the writings of Foucault and Hardt, we argue that emotion-recognition tools signal a far more invasive disciplinary gaze that exposes and makes vulnerable the inner regions of the worker-self. Our paper explores attitudes towards empathic surveillance by analyzing a survey of 1015 responses of future job-seekers from 48 countries with Bayesian statistics. Our findings reveal affect tools, left unregulated in the workplace, may lead to heightened stress and anxiety among disadvantaged ethnicities, gender and income class. We also discuss a stark cross-cultural discrepancy whereby East Asians, compared to Western subjects, are more likely to profess a trusting attitude toward EAI-enabled automated management. While this emerging technology is driven by neoliberal incentives to optimize the worksite and increase productivity, ultimately, empathic surveillance may create more problems in terms of algorithmic bias, opaque decisionism, and the erosion of employment relations. Thus, this paper nuances and extends emerging literature on emotion-sensing technologies in the workplace, particularly through its highly original cross-cultural study. (shrink)

Existing research shows that people can improve their decision skills by learning what experts paid attention to when faced with the same problem. However, in domains like financial education, effective instruction requires frequent, personalized feedback given at the point of decision, which makes it time‐consuming for experts to provide and thus, prohibitively costly. We address this by demonstrating an automated feedback mechanism that allows amateur decision‐makers to learn what information to attend to from one another, rather than from an expert. (...) In the first experiment, eye movements of N = 100 subjects were recorded while they repeatedly performed a standard behavioral finance investment task. Consistent with previous studies, we found that a significant proportion of subjects were affected by decision bias. In the second experiment, a different group of N = 100 subjects faced the same task but, after each choice, they received individual, machine learning‐generated feedback on whether their pre‐decision eye movements resembled those made by Experiment 1 subjects prior to good decisions. As a result, Experiment 2 subjects learned to analyze information similarly to their successful peers, which in turn reduced their decision bias. Furthermore, subjects with low Cognitive Reflection Test scores gained more from the proposed form of process feedback than from standard behavioral feedback based on decision outcomes. (shrink)

If our choices make us who we are, then what does that mean when these choices are made in the human-machine interface? Developing a clear understanding of how human decision making is influenced by automated systems in the environment is critical because, as human-machine interfaces and assistive robotics become even more ubiquitous in everyday life, many daily decisions will be an emergent result of the interactions between the human and the machine – not stemming solely from the human. For example, (...) choices can be influenced by the relative locations and motor costs of the response options, as well as by the timing of the response prompts. In drift diffusion model simulations of response-prompt timing manipulations, we find that it is only relatively equibiased choices that will be successfully influenced by this kind of perturbation. However, with drift diffusion model simulations of motor cost manipulations, we find that even relatively biased choices can still show some influence of the perturbation. We report the results of a two-alternative forced-choice experiment with a computer mouse modified to have a subtle velocity bias in a pre-determined direction for each trial, inducing an increased motor cost to move the cursor away from the pre-designated target direction. With queries that have each been normed in advance to be equibiased in people’s preferences, the participant will often begin their mouse movement before their cognitive choice has been finalized, and the directional bias in the mouse velocity exerts a small but significant influence on their final choice. With queries that are not equibiased, a similar influence is observed. By exploring the synergies that are developed between humans and machines and tracking their temporal dynamics, this work aims to provide insight into our evolving decisions. (shrink)

Artificial Intelligence technology presents a multitude of ethical concerns, many of which are being actively considered by organizations ranging from small groups in civil society to large corporations and governments. However, it also presents ethical concerns which are not being actively considered. This paper presents a broad overview of twelve topics in ethics in AI, including function, transparency, evil use, good use, bias, unemployment, socio-economic inequality, moral automation and human de-skilling, robot consciousness and rights, dependency, social-psychological effects, and (...) spiritual effects. Each of these topics will be given a brief discussion, though each deserves much deeper consideration. (shrink)

The data science era is characterized by data-driven automated decision systems (ADS) enabling, through data analytics and machine learning, automated decisions in many contexts, deeply impacting our lives. As such, their downsides and potential risks are becoming more and more evident: technical solutions, alone, are not sufficient and an interdisciplinary approach is needed. Consequently, ADS should evolve into data-informed ADS, which take humans in the loop in all the data processing steps. Data-informed ADS should deal with data responsibly, guaranteeing nondiscrimination (...) with respect to protected groups of individuals. Nondiscrimination can be characterized in terms of different types of properties, like fairness and diversity. While fairness, i.e., absence of bias against minorities, has been widely investigated in machine learning, only more recently this issue has been tackled by considering all the steps of data processing pipelines at the basis of ADS, from data acquisition to analysis. Additionally, fairness is just one point of view of nondiscrimination to be considered for guaranteeing equity: other issues, like diversity, are raising interest from the scientific community due to their relevance in society. This paper aims at critically surveying how nondiscrimination has been investigated in the context of complex data science pipelines at the basis of data-informed ADS, by focusing on the specific data processing tasks for which nondiscrimination solutions have been proposed. (shrink)

A concise but informative overview of AI ethics and policy. -/- Artificial intelligence, or AI for short, has generated a staggering amount of hype in the past several years. Is it the game-changer it's been cracked up to be? If so, how is it changing the game? How is it likely to affect us as customers, tenants, aspiring homeowners, students, educators, patients, clients, prison inmates, members of ethnic and sexual minorities, and voters in liberal democracies? Authored by experts in fields (...) ranging from computer science and law to philosophy and cognitive science, this book offers a concise overview of moral, political, legal and economic implications of AI. It covers the basics of AI's latest permutation, machine learning, and considers issues such as transparency, bias, liability, privacy, and regulation. -/- Both business and government have integrated algorithmic decision support systems into their daily operations, and the book explores the implications for our lives as citizens. For example, do we take it on faith that a machine knows best in approving a patient's health insurance claim or a defendant's request for bail? What is the potential for manipulation by targeted political ads? How can the processes behind these technically sophisticated tools ever be transparent? The book discusses such issues as statistical definitions of fairness, legal and moral responsibility, the role of humans in machine learning decision systems, “nudging” algorithms and anonymized data, the effect of automation on the workplace, and AI as both regulatory tool and target. (shrink)

The Oxford Handbook of Digital Ethics is a lively and authoritative guide to ethical issues related to digital technologies, with a special emphasis on AI. Philosophers with a wide range of expertise cover thirty-seven topics: from the right to have access to internet, to trolling and online shaming, speech on social media, fake news, sex robots and dating online, persuasive technology, value alignment, algorithmic bias, predictive policing, price discrimination online, medical AI, privacy and surveillance, automating democracy, the future of (...) work, and AI and existential risk, among others. Each chapter gives a rigorous map of the ethical terrain, engaging critically with the most notable work in the area, and pointing directions for future research. (shrink)

BackgroundCognitive impairments are prevalent in patients with unipolar and bipolar depressive disorder. Considering the fact assessing cognitive functions is increasingly feasible for clinicians and researchers, targeting these problems in treatment and using them at baseline as predictors of response to treatment can be very informative.MethodIn a naturalistic, retrospective study, data from 120 patients with UDD and BDD were analyzed. Patients received 20 sessions of bilateral rTMS and were assessed regarding their depressive symptoms, sustained attention, working memory, and executive functions, using (...) the Beck Depression Inventory and Neuropsychological Test Automated Battery Cambridge, at baseline and after the end of rTMS treatment course. Generalized estimating equations and logistic regression were used as the main statistical methods to test the hypotheses.ResultsFifty-three percentage of all patients responded to treatment. In particular, 53.1% of UDD patients and 46.9% of BDD patients responded to treatment. Bilateral rTMS improved all cognitive functions except for visual memory and resulted in more modulations in the working memory of UDD compared to BDD patients. More improvements in working memory were observed in responded patients and visual memory, age, and sex were determined as treatment response predictors. Working memory, visual memory, and age were identified as treatment response predictors in BDD and UDD patients, respectively.ConclusionBilateral rTMS improved cold cognition and depressive symptoms in UDD and BDD patients, possibly by altering cognitive control mechanisms, and processing negative emotional bias. (shrink)

A number of findings in the field of machine learning have given rise to questions about what it means for automated scoring- or decisionmaking systems to be fair. One center of gravity in this discussion is whether such systems ought to satisfy classification parity (which requires parity in accuracy across groups, defined by protected attributes) or calibration (which requires similar predictions to have similar meanings across groups, defined by protected attributes). Central to this discussion are impossibility results, owed to Kleinberg (...) et al. (2016), Chouldechova (2017), and Corbett-Davies et al. (2017), which show that classification parity and calibration are often incompatible. This paper aims to argue that classification parity, calibration, and a newer, interesting measure called counterfactual fairness are unsatisfactory measures of fairness, offer a general diagnosis of the failure of these measures, and sketch an alternative approach to understanding fairness in machine learning. (shrink)

Co-authored by a Computer Scientist and a Digital Humanist, this article examines the challenges faced by cultural heritage institutions in the digital age, which have led to the closure of the vast majority of born-digital archival collections. It focuses particularly on cultural organizations such as libraries, museums and archives, used by historians, literary scholars and other Humanities scholars. Most born-digital records held by cultural organizations are inaccessible due to privacy, copyright, commercial and technical issues. Even when born-digital data are publicly (...) available, users often need to physically travel to repositories such as the British Library or the Bibliothèque Nationale de France to consult web pages. Provided with enough sample data from which to learn and train their models, AI, and more specifically machine learning algorithms, offer the opportunity to improve and ease the access to digital archives by learning to perform complex human tasks. These vary from providing intelligent support for searching the archives to automate tedious and time-consuming tasks. In this article, we focus on sensitivity review as a practical solution to unlock digital archives that would allow archival institutions to make non-sensitive information available. This promise to make archives more accessible does not come free of warnings for potential pitfalls and risks: inherent errors, "black box" approaches that make the algorithm inscrutable, and risks related to bias, fake, or partial information. Our central argument is that AI can deliver its promise to make digital archival collections more accessible, but it also creates new challenges - particularly in terms of ethics. In the conclusion, we insist on the importance of fairness, accountability and transparency in the process of making digital archives more accessible. (shrink)

In the seventy years since AI became a field of study, the theoretical work of philosophers has played increasingly important roles in understanding many aspects of the AI project, from the metaphysics of mind and what kinds of systems can or cannot implement them, the epistemology of objectivity and algorithmic bias, the ethics of automation, drones, and specific implementations of AI, as well as analyses of AI embedded in social contexts (for example). Serious scholarship in AI ethics sometimes (...) quotes Asimov’s speculative laws of robotics as if they were genuine proposals, and yet Lem remains historically undervalued as a theorist who uses fiction as his vehicle. Here, I argue that Lem’s fiction (in particular his fiction about robots) is overlooked but highly nuanced philosophy of AI, and that we should recognize the lessons he tried to offer us, which focused on the human and social failures rather than technological breakdowns. Stories like “How the World Was Saved” and “Upside Down Evolution” ask serious philosophical questions about AI metaphysics and ethics, and offer insightful answers that deserve more attention. Highlighting some of this work from The Cyberiad and the stories in Mortal Engines in particular, I argue that the time has never been more appropriate to attend to his philosophy in light of the widespread technological and social failures brought about by the quest for artificial intelligence. In service of this argument, I discuss some of the history and philosophical debates around AI in the last decades, as well as contemporary events that illustrate Lem’s strongest claims in critique of the human side of AI. (shrink)

The current study explores how the cultural distance of ethnic outgroups relative to the ethnic ingroup is related to stereotypical news representations. It does so by drawing on a sample of more than three million Dutch newspaper articles and uses advanced methods of automated content analysis, namely word embeddings. The results show that distant ethnic outgroup members (i. e., Moroccans) are associated with negative characteristics and issues, while this is not the case for close ethnic outgroup members (i. e., Belgians). (...) The current study demonstrates the usefulness of word embeddings as a tool to study subtle aspects of ethnic bias in mass-mediated content. (shrink)

Emotions play a significant role in human relations, decision-making, and the motivation to act on those decisions. There are ongoing attempts to use artificial intelligence (AI) to read human emotions, and to predict human behavior or actions that may follow those emotions. However, a person’s emotions cannot be easily identified, measured, and evaluated by others, including automated machines and algorithms run by AI. The ethics of emotional AI is under research and this study has examined the emotional variables as well (...) as the perception of emotional AI in two large random groups of college students in an international university in Japan, with a heavy representation of Japanese, Indonesian, Korean, Chinese, Thai, Vietnamese, and other Asian nationalities. Surveys with multiple close-ended questions and an open-ended essay question regarding emotional AI were administered for quantitative and qualitative analysis, respectively. The results demonstrate how ethically questionable results may be obtained through affective computing and by searching for correlations in a variety of factors in collected data to classify individuals into certain categories and thus aggravate bias and discrimination. Nevertheless, the qualitative study of students’ essays shows a rather optimistic view over the use of emotional AI, which helps underscore the need to increase awareness about the ethical pitfalls of AI technologies in the complex field of human emotions. (shrink)

AI seems like the perfect response to the growing challenges of content moderation on social media platforms: the immense scale of the data, the relentlessness of the violations, and the need for human judgments without wanting humans to have to make them. The push toward automated content moderation is often justified as a necessary response to the scale: the enormity of social media platforms like Facebook and YouTube stands as the reason why AI approaches are desirable, even inevitable. But even (...) if we could effectively automate content moderation, it is not clear that we should. (shrink)

The rapidly increasing role of automation throughout the economy, culture and our personal lives has generated a large literature on the risks of algorithmic decision-making, particularly in high-stakes legal settings. Algorithmic tools are charged with bias, shrouded in secrecy, and frequently difficult to interpret. However, these criticisms have tended to focus on particular implementations, specific predictive techniques, and the idiosyncrasies of the American legal-regulatory regime. They do not address the more fundamental unease about the prospect that we might (...) one day replace judges with algorithms, no matter how fair, transparent, and intelligible they become. The aim of this paper is to propose an account of the source of that unease, and to evaluate its plausibility. I trace foundational unease with algorithmic decision-making in the law to the powerful intuition that there is a basic moral and legal difference between showing that something is true of many people just like you and showing that it is true of you. Human judgment attends to the exception; automation insists on blindly applying the rule. I show how this intuitive thought is connected to both epistemological arguments about the value of statistical evidence, as well as to court-centered conceptions of the rule of law. Unease with algorithmic decision-making in the law thus draws on an intuitive principle that underpins a disparate range of views in legal philosophy. This suggests the principle is deeply ingrained. Nonetheless, I argue that the powerful intuition is not as decisive as it may seem, and indeed runs into significant epistemological and normative challenges. At an epistemological level, I show how concerns about statistical evidence's ability to track the truth can be resolved by adopting a probabilistic, rather than modal, conception of truth-tracking. At a normative level, commitment to highly individualized decision-making co-exists with equally ingrained and competing principles, such as consistent application of law. This suggests that the “rule of law” may not identify a discrete set of institutional arrangements, as proponents of a court-centric conception would have it, but rather a more loosely defined set of values that could potentially be operationalized in multiple ways, including through some level of algorithmic adjudication. Although the prospect of replacing judges with algorithms is indeed unsettling, it does not necessarily entail unreasonable verdicts or an attack on the rule of law. (shrink)

Artificial Intelligence is playing an increasingly prevalent role in our lives. Whether its landing a job interview, getting a bank loan or accessing a government program, organizations are using automated systems informed by AI enabled technologies in ways that have significant consequences for people. At the same time, there is a lack of transparency around how AI technologies work and whether they are ethical, fair or accurate. This paper examines a body of literature related to the ethical considerations surrounding the (...) use of artificial intelligence and the role of ethical codes. It identifies and explores core issues including bias, fairness and transparency and looks at who is setting the agenda for AI ethics in Canada and globally. Lastly, it offers some suggestions for next steps towards a more inclusive discussion. (shrink)

The current discourse on fairness, accountability, and transparency in machine learning is driven by two competing narratives: sociotechnical dogmatism, which holds that society is full of inefficiencies and imperfections that can only be solved by better algorithms; and sociotechnical skepticism, which opposes many instances of automation on principle. Both perspectives, we argue, are reductive and unhelpful. In this chapter, we review a large, diverse body of literature in an attempt to move beyond this restrictive duality, toward a pragmatic synthesis (...) that emphasizes the central role of context and agency in evaluating new and emerging technologies. We show how epistemological and ethical considerations are inextricably intertwined in contemporary debates on algorithmic bias and explainability. We trace the dialectical interplay between dogmatic and skeptical narratives across disciplines, merging insights from social theory and philosophy. We review a number of theories of explanation, ultimately endorsing a sociotechnical pragmatism that combines elements of Floridi’s levelism and Mayo’s reliabilism to place a special emphasis on notions of agency and trust. We conclude that this hybrid does more to promote fairness, accountability, and transparency in machine learning than dogmatic or skeptical alternatives. (shrink)

This paper focuses on the Czech-born philosopher Vilém Flusser’s notion of telematic society, arguing that it implies a media-theoretical revision of Friedrich Schiller’s project for an aesthetic model of civic education, according to which aesthetic effectivity is reconsidered in light of a history of media based on a technological alternation of images and texts. After a brief overview of Schiller’s aesthetic letters, it examines the ways in which Flusser repositions and expands upon Schiller’s vision of an aesthetic education (most importantly, (...) the aesthetic condition or middle disposition), highlighting the role that Flusser assigns to the evolution of codes inherent in the most important cultural and media techniques, namely, images, texts, and technical images. It then shows how this modification results in a forecasting of a telematic model of aesthetic education, before concluding by addressing the prospects of such a model in a post-digital world of automated media bias. (shrink)

This paper considers two subjective measures of the existence of unconscious mental states - the guessing criterion, and the zero correlation criterion - and considers the assumptions underlying their application in experimental paradigms. Using higher order thought theory the impact of different types of biases on the zero correlation and guessing criteria are considered. It is argued that subjective measures of consciousness can be biased in various specified ways, some of which involve the relation between first order states and second (...) order thoughts, and hence are not errors in measurement of the conscious status of mental states; but other sorts of biases are measurement errors, involving the relation between higher order thoughts and their expression. Nonetheless, it is argued this type of bias does not preclude subjective measures - both the guessing criterion and the zero correlation criterion - as being amongst the most appropriate and useful tools for measuring the conscious status of mental states. (shrink)

According to Russell and Norvig’s (2009) classification, Artificial Intelligence (AI) is the field that aims at building systems which either think rationally, act rationally, think like humans, or...

Previous research using immersive virtual reality (VR) has shown that after a short period of embodiment of White people in a Black virtual body their implicit racial bias against Black people diminishes. Here we tested the effects of some socio-cognitive variables that could contribute to enhancing or reducing the implicit racial bias. The first aim of the study was to assess the beneficial effects of cooperation within a VR scenario, the second aim was to provide preliminary testing of (...) the hypothesis that empathy and political attitudes could contribute to implicit bias about race, while the third aim was to explore the relationship between political attitudes and empathy. We had (Caucasian) participants embodied in a Black virtual body and engaged either in a cooperative (Coop group) or in a non-cooperative (Neutral group) activity with a confederate experimenter embodying another Black avatar. Before and after VR, we measured participants’ implicit racial bias by means of Implicit Association Test (IAT) and their perceived closeness toward the confederate experimenter. Before VR we also assessed participants’ political attitudes and empathy traits. Results revealed that, as compared to the Neutral group, the Coop group showed lower IAT scores after the social interaction. Interestingly, in the Neutral but not the Coop group the perceived closeness toward the confederate experimenter was associated with the initial racial bias: the more the participants reduced their distance, the more they reduced their IAT score. Moreover, reported traits of empathy and political attitudes significantly explained the variance observed in the initial implicit bias, with perspective-taking, empathic concern, and personal distress being significant predictors of the IAT scores. Finally, there was a relationship between political attitudes and empathy: the more participants considered themselves as left-wing voters, the higher their perspective-taking and empathic concern scores. We discuss these findings within the neuroscientific and social cognition field and encourage scholars from different domains to further explore whether and under which conditions a given manipulation for reducing racial bias could be efficiently transposed in VR. (shrink)