Many have recently argued that there are weighty reasons against making high-stakes decisions solely on the basis of recommendations from artificially intelligent (AI) systems. Even if deference to a given AI system were known to reliably result in the right action being taken, the argument goes, that deference would lack morally important characteristics: the resulting decisions would not, for instance, be based on an appreciation of right-making reasons. Nor would they be performed from moral virtue; nor would they have (...) moral worth. I argue that, even if these characteristics all have intrinsic value, that intrinsic value has no practical relevance to decisions about whether to defer to AI. I make that point by drawing on a lesson from the literature on moral testimony. Once it is granted that deference to a reliable source is the policy most likely to bring about right action, a refusal to defer carries with it a heightened risk of wronging and mistreating people. And that heightened risk of wrongdoing, I argue, cannot be justified by appeal to the intrinsic value of striving for a morally exemplary decision-making process. (shrink)

Deep Neural Networks (DNNs) are epistemically opaque in the sense that their inner functioning is often unintelligible to human investigators. Inkeri Koskinen has recently argued that this poses special problems for a widespread view in social epistemology according to which thick normative trust between researchers is necessary to handle opacity: if DNNs are essentially opaque, there simply exists nobody who could be trusted to understand all the aspects a DNN picks up during training. In this paper, I present (...) a counterexample from scientific practice, AlphaFold2. I argue that for epistemic reliance on an opaque system, trust is not necessary, but reliability is. What matters is whether, for a given context, the reliability of a DNN has been compellingly established by empirical means and whether there exist trustable researchers who have performed such evaluations adequately. (shrink)

If updating with E has the same result across all epistemically possible worlds, then the agent has no uncertainty as to the behavior of the update, and we may call it a transparent update. If an agent is uncertain about the behavior of an update, we may call it opaque. In order to model the uncertainty an agent has about the result of an update, the same update must behave differently across different possible worlds. In this paper, I study (...) opaque updates using a simple system of dynamic epistemic logic suitably modified for that purpose. The paper highlights the connection between opaque updates and the dynamic-epistemic principles Perfect-Recall and No-Miracles. I argue that opaque updates are central to contemporary discussions in epistemology, in particular to externalist theories of knowledge and to the related problem of epistemic bootstrapping, or easy knowledge. Opaque updates allow us to explicitly investigate a dynamic form of uncertainty, using simple and precise logical tools. (shrink)

Opaqueness of pp collisions is evaluated at three CERN-ISR energies. Comparisons with predictions of the factorizable eikonal models and the scaling hypothesis are made. It appears that results are in favor of the factorizable models.

The increasing demand for transparency in AI has recently come under scrutiny. The question is often posted in terms of “epistemic double standards”, and whether the standards for transparency in AI ought to be higher than, or equivalent to, our standards for ordinary human reasoners. I agree that the push for increased transparency in AI deserves closer examination, and that comparing these standards to our standards of transparency for other opaque systems is an appropriate starting point. I suggest (...) that a more fruitful exploration of this question will involve a different comparison class. We routinely treat judgments made by highly trained experts in specialized fields as fair or well-grounded even though—by the nature of expert/layperson division of epistemic labor—an expert will not be able to provide an explanation of the reasoning behind these judgments that makes sense to most other people. Regardless, laypeople are thought to be acting reasonably—and ethically—in deferring to the judgments of experts that concern their areas of specialization. I suggest that we reframe our question regarding the appropriate standards of transparency in AI as one that asks when, why, and to what degree it would be ethical to accept opacity in AI. I argue that our epistemic relation to certain opaque AI technology may be relevantly similar to the layperson’s epistemic relation to the expert in certain respects, such that the successful expert/layperson division of epistemic labor can serve as a blueprint for the ethical use of opaque AI. (shrink)

In 2020, the European Union announced the award of the contract for the biometric part of the new database for border control, the Entry Exit System, to two companies: IDEMIA and Sopra Steria. Both companies had been previously involved in the development of databases for border and migration management. While there has been a growing amount of publicly available documents that show what kind of technologies are being implemented, for how much money, and by whom, there has been limited engagement (...) with digital methods in this field. Moreover, critical border and security scholarship has largely focused on qualitative and ethnographic methods. Building on a data feminist approach, we propose a transdisciplinary methodology that goes beyond binaries of qualitative/quantitative and opacity/transparency, examines power asymmetries and makes the labour of coding visible. Empirically, we build and analyse a dataset of the contracts awarded by two European Union agencies key to its border management policies – the European Agency for Large-Scale Information Systems (eu-LISA) and the European Border and Coast Guard Agency (Frontex). We supplement the digital analysis and visualisation of networks of companies with close reading of tender documents. In so doing, we show how a transdisciplinary methodology can be a device for making datafication ‘intelligible’ at the European Union borders. (shrink)

Many artificial intelligence (AI) systems currently used for decision-making are opaque, i.e., the internal factors that determine their decisions are not fully known to people due to the systems’ computational complexity. In response to this problem, several researchers have argued that human decision-making is equally opaque and since simplifying, reason-giving explanations (rather than exhaustive causal accounts) of a decision are typically viewed as sufficient in the human case, the same should hold for algorithmic decision-making. Here, I (...) contend that this argument overlooks that human decision-making is sometimes significantly more transparent and trustworthy than algorithmic decision-making. This is because when people explain their decisions by giving reasons for them, this frequently prompts those giving the reasons to govern or regulate themselves so as to think and act in ways that confirm their reason reports. AI explanation systems lack this self-regulative feature. Overlooking it when comparing algorithmic and human decision-making can result in underestimations of the transparency of human decision-making and in the development of explainable AI that may mislead people by activating generally warranted beliefs about the regulative dimension of reason-giving. (shrink)

The use of opaque, uninterpretable artificial intelligence systems in health care can be medically beneficial, but it is often viewed as potentially morally problematic on account of this opacity—because the systems are black boxes. Alex John London has recently argued that opacity is not generally problematic, given that many standard therapies are explanatorily opaque and that we can rely on statistical validation of the systems in deciding whether to implement them. But is statistical validation sufficient (...) to justify implementation of these AI systems in health care, or is it merely one of the necessary criteria? I argue that accountability, which holds an important role in preserving the patient‐physician trust that allows the institution of medicine to function, contributes further to an account of AI system justification. Hence, I endorse the vanishing accountability principle: accountability in medicine, in addition to statistical validation, must be preserved. AI systems that introduce problematic gaps in accountability should not be implemented. (shrink)

This paper examines the paradoxical transparency involved in training machine-learning models. Existing literature typically critiques the opacity of machine-learning models such as neural networks or collaborative filtering, a type of critique that parallels the black-box critique in technology studies. Accordingly, people in power may leverage the models’ opacity to justify a biased result without subjecting the technical operations to public scrutiny, in what Dan McQuillan metaphorically depicts as an “algorithmic state of exception”. This paper attempts to differentiate the black-box abstraction (...) that wraps around complex computational systems from the opacity of machine-learning models. It contends that the degree of asymmetry in knowledge is greater in the former than the latter. In the case of software systems, software codes are difficult to understand as only software experts with sufficient domain knowledge are equipped to formulate a sound critique. In contrast, the meanings of trained parameters in a machine-learning model are obscure even to the data scientists who configure and train the model. Hence, the asymmetry of knowledge lies only in how data examples are collected, the choice and configuration of machine-learning models, and the specification of features in model design. Under the trend of algorithmic decision-making proliferating with machine-learning heuristics, the paper contends that the more symmetric distribution of knowledge in machine learning could lead to a more transparent production process if proper policies are in place. (shrink)

The use of artificial intelligence (AI) in healthcare comes with opportunities but also numerous challenges. A specific challenge that remains underexplored is the lack of clear and distinct definitions of the concepts used in and/or produced by these algorithms, and how their real world meaning is translated into machine language and vice versa, how their output is understood by the end user. This “semantic” black box adds to the “mathematical” black box present in many AI systems in which the (...) underlying “reasoning” process is often opaque. In this way, whereas it is often claimed that the use of AI in medical applications will deliver “objective” information, the true relevance or meaning to the end‐user is frequently obscured. This is highly problematic as AI devices are used not only for diagnostic and decision support by healthcare professionals, but also can be used to deliver information to patients, for example to create visual aids for use in shared decision‐making. This paper provides an examination of the range and extent of this problem and its implications, on the basis of cases from the field of intensive care nephrology. We explore how the problematic terminology used in human communication about the detection, diagnosis, treatment, and prognosis of concepts of intensive care nephrology becomes a much more complicated affair when deployed in the form of algorithmic automation, with implications extending throughout clinical care, affecting norms and practices long considered fundamental to good clinical care. (shrink)

Scientists depend on complex computational systems that are often ineliminably opaque, to the detriment of our ability to give scientific explanations and detect artifacts. Some philosophers have s...

The paper illustrates the evolution of the territorial system of government in Italy. A traditionally centralized state is turning into a quasifederation. The reasons underpinning this transformation are the growing insatisfaction with the inefficiency of the central government, the quest for autonomy by the fastest growing regions and the opposition by the latter to the interterritorial redistribution of resources made by the central government with the use of nontransparent and inefficient instruments.A substantial degree of subnational tax autonomy has been reintroduced. (...) The powers of subnational governments have been expanded. The main impact on decentralization derives, however, from the direct, popular, election of mayors and of presidents of regional and provincial governments.The transformation is far from complete. The central government is still reluctant to relinquishing its powers. The increased local tax autonomy did not yet introduce substantial tax competition. More precisely, tax rates were not reduced, but rather increased, due to the fiscal stringency of the whole public sector.L’article illustre l’évolution du système territorial de gouvernement en Italie. Un Etat traditionnellement centralisé est en train de devenir quasi-fédéral. Les raisons fondamentales de cette transformation sont l’insatisfaction croissante vis-à-vis de l’inefficacité du gouvernement central, la quête de plus d’autonomie de la part des régions à croissance rapide et l’opposition de ces dernières à la redistribution interterritoriale des ressources qu’opère le gouvernement central en utilisant des instruments opaques et inefficaces.Un degré substantiel d’autonomie fiscale subnationale a été réintroduit. Les pouvoirs des gouvernements subnationaux ont été étendus. L’impact principal de la décentralisation provient, cependant, de l’élection, directe et populaire, des maires et des présidents de gouvernements régionaux et provinciaux.La transformation est loin d’être complète. Le gouvernement central reste réticent à abandonner ses pouvoirs. L’autonomie croissante des impôts locaux n’a pas introduit pour le moment de concurrence fiscale. Plus précisément, les taux d’imposition n’ont pas été réduits, mais plutôt augmentés, en raison de la rigueur budgétaire de l’ensemble du secteur public. (shrink)

This paper is an attempt to present disclosive ethics as a framework for computer and information ethics – in line with the suggestions by Brey, but also in quite a different manner. The potential of such an approach is demonstrated through a disclosive analysis of facial recognition systems. The paper argues that the politics of information technology is a particularly powerful politics since information technology is an opaque technology – i.e. relatively closed to scrutiny. It presents the design (...) of technology as a process of closure in which design and use decisions become black-boxed and progressively enclosed in increasingly complex socio-technical networks. It further argues for a disclosive ethics that aims to disclose the nondisclosure of politics by claiming a place for ethics in every actual operation of power – as manifested in actual design and use decisions and practices. It also proposes that disclosive ethics would aim to trace and disclose the intentional and emerging enclosure of politics from the very minute technical detail through to social practices and complex social-technical networks. The paper then proceeds to do a disclosive analysis of facial recognition systems. This analysis discloses that seemingly trivial biases in recognition rates of FRSs can emerge as very significant political acts when these systems become used in practice. (shrink)

Critical legal scholarship has so far been concerned primarily with trashing or deconstructing the belief clusters of "liberalism". Negative posturing of this kind is not the only feature of the movement, though. Roberto Unger has dreamt up a sociopolitical vision that presents an "empowered democracy". An important element of his "empowered democracy" is a new system of rights. Part 1 of my essay contains an analysis of the notion of a subjective right. I argue that both Hohfeld's fundamental legal conceptions (...) and Unger's various rights can be described by a simple deontic relation that I define as RIGHT. I then discuss a set of normative criteria that can help us evaluate systems of rights. Part 2 [to appear in the following issue -- ed.] contains a detailed critique of Unger's system of rights based on these normative criteria. The tenet of this part is that Unger's system of rights is contradictory, opaque, impracticable, costly, and not fully backed by what Unger offers as a background justification for it. (shrink)

There is much to learn from Durán and Jongsma’s paper.1 One particularly important insight concerns the relationship between epistemology and ethics in medical artificial intelligence. In clinical environments, the task of AI systems is to provide risk estimates or diagnostic decisions, which then need to be weighed by physicians. Hence, while the implementation of AI systems might give rise to ethical issues—for example, overtreatment, defensive medicine or paternalism2—the issue that lies at the heart is an epistemic problem: how (...) can physicians know whether to trust decisions made by AI systems? In this manner, various studies examining the interaction of AI systems and physicians have shown that without being able to evaluate their trustworthiness, especially novice physicians become over-reliant on algorithmic support—and ultimately are led astray by incorrect decisions.3–5 This leads to a second insight from the paper, namely that even if some AI system happens to be opaque, it is still not built on the moon. To assess its trustworthiness, AI developers or physicians have different sorts of higher order evidence at hand. Most importantly, …. (shrink)

As a way to address both ominous and ordinary threats of artificial intelligence, researchers have started proposing ways to stop an AI system before it has a chance to escape outside control and cause harm. A so-called “big red button” would enable human operators to interrupt or divert a system while preventing the system from learning that such an intervention is a threat. Though an emergency button for AI seems to make intuitive sense, that approach ultimately concentrates on the point (...) when a system has already “gone rogue” and seeks to obstruct interference. A better approach would be to make ongoing self-evaluation and testing an integral part of a system’s operation, diagnose how the system is in error and to prevent chaos and risk before they start. In this paper, we describe the demands that recent big red button proposals have not addressed, and we offer a preliminary model of an approach that could better meet them. We argue for an ethical core that consists of a scenario-generation mechanism and a simulation environment that are used to test a system’s decisions in simulated worlds, rather than the real world. This EC would be kept opaque to the system itself: through careful design of memory and the character of the scenario, the system’s algorithms would be prevented from learning about its operation and its function, and ultimately its presence. By monitoring and checking for deviant behavior, we conclude, a continual testing approach will be far more effective, responsive, and vigilant toward a system’s learning and action in the world than an emergency button which one might not get to push in time. (shrink)

Patients and physicians do not know the cost of medical procedures. Opaque medical billing thus contributes to exorbitant, rising medical costs, burdening the healthcare system and individuals. After criticizing two proposed solutions to the problem of opaque medical billing, I argue that the Centers for Medicare and Medicaid Services should pursue a rule requiring that patients be informed by the physician of a reasonable out-of-pocket expense estimate for non-urgent procedures prior to services rendered.

Over the past decade, significant progress in artificial intelligence (AI) has spurred the adoption of its algorithms, addressing previously daunting challenges. Alongside these remarkable strides, there has been a simultaneous increase in model complexity and reliance on opaque AI models, lacking transparency. In numerous scenarios, the systems themselves may necessitate making decisions entailing ethical dimensions. Consequently, it has become imperative to devise solutions to integrate ethical considerations into AI system development practices, facilitating broader utilization of AI systems (...) across various domains. Research endeavors should explore innovative approaches to enhance ethical principles in AI systems, fostering greater transparency, accountability, and trustworthiness. Upholding fundamental individual rights, human dignity, autonomy, privacy, equality, and fairness, while mitigating potential harm, remains paramount. Considering ethical values and ensuring compliance with ethical requirements throughout the development lifecycle of autonomous and intelligent systems nurture trust and reliability in their utilization. Ethical considerations should be ingrained within organizational procedures guiding AI research activities, establishing robust frameworks that address ethical concerns and reflect the ethical implications of AI-based systems. This paper presents an overview of ethical approaches and processes aimed at integrating ethical considerations into AI system development practices. It underscores the significance of ethical frameworks in fostering ethical AI implementation and ensuring the ethical integrity of AI technologies. (shrink)

Contents: Leszek NOWAK, Marcin PAPRZYCKI: Introduction. ON THE NATURE OF SOCIAL SYSTEM. Ulrich K. PREUSS: Political Order and Democracy. Carl Schmitt and his Influence. Katarzyna PAPRZYCKA: A Paradox in Hobbes' Philosophy of Law. Stephen L. ESQUITH: Democratic Political Dialogue. Edward JELINSKI: Democracy in Polish Reformist Socialist Thought. Katarzyna PAPRZYCKA: The Master and Slave Configuration in Hegel's System. Maurice GODELIER: Lévi-Strauss, Marx and After. A reappraisal of structuralist and Marxist tools for analyzing social logics. Krzysztof NIEDZWIADEK: On the Structure of Social (...) System. Waldemar CZAJKOWSKI: Social Being and Its Reproduction. ON RATIONALITY AND CAPTIVITY. Marek ZIO??L??KOWSKI: Power and Knowledge. Leszek NOWAK: Two Inter-Human Limits to the Rationality of Man. Marcin PAPRZYCKI: The non-Christian Model of Man. An Attempt at a Psychoanalytic Explanation. Robert EGIERT: Toward the Sophisticated Rationalistic Model of Man. ON SOCIAL REVOLUTION. Leszek NOWAK: Revolution is an Opaque Progress but a Progress Nonetheless. Katarzyna PAPRZYCKA, Marcin PAPRZYCKI: How Do Enslaved People Make Revolutions? Grzegorz TOMCZAK: Is It Worth Winning a Revolution? Krzysztof BRZECHCZYN: Civil Loop and the Absorption of Elites. Richard C. MCCLEARY: What Makes Marxist Historical Materialism Objective? Grzegorz KOTLARSKI: Classes and Masses in Social Philosophy of Rosa Luxemburg. ON REAL SOCIALISM. Ernest GELLNER: The Civil and the Sacred. Witold MARCISZEWSKI: Economics and the Idea of Information. Why socialism must have collapsed? Leszek NOWAK, Katarzyna PAPRZYCKA, Marcin PAPRZYCKI: On Multilinearity of Socialism. Achim SIEGEL: The Overrepression Cycle in the Soviet Union. An Operationalization of a Theoretical Model. Krzysztof BRZECHCZYN: The State of the Teutonic Order as a Socialist Society. DISCUSSIONS. Richard MCCLEARLY: Socioanalysis and Philosophy. W??l??odzimierz HELLER: The Public and the Private in Hannah Arendt's Political Philosophy. Methodological Remarks. Krzysztof BRZECHCZYN: Unsuccessful Conquest and Successful Subordination. A contribution to the theory of intersocial relations. (shrink)

India’s energy demand is predicted to rise by 135% within a span of 20 years. Coping up with surging energy demands requires several reforms in both renewable and non-renewable sectors. Factors such as rising population, reduction in the cost of renewable energy technology and their effect on the nation’s GDP, can make policy making a herculean task and the justification for such policies, quite opaque to the public. Artificial Intelligence technology can help decision makers to quickly draw conclusions from (...) voluminous datasets under different heads. However, AI results need to be post-processed so that they are easily understood by the layperson. This paper focuses on using Sankey diagrams as a post-processing tool for AI systems. Policy formulation often requires an overall assessment of energy sources, production pathways, end used destinations and wastage encountered—these can be easily visualized with a Sankey diagram. For this work, a Sankey protocol was written out in a form where a prompt asks the user to supply information through a spreadsheet interfaced with a customized mobile app—a first anywhere in the subcontinent. India’s mobile sector is vibrant, and the paper presents a enabling the user to have a handle on the dynamics of energy distribution locally. The app gives an instant feel of the apportioned energy across grids, the projected changes for the next decade, the efficiency and feasibility of each sector and lastly a telling visual representation showing the main strengths and weaknesses in the energy sector. (shrink)

Modern computer systems are ubiquitous in contemporary life yet many of them remain opaque. This poses significant challenges in domains where desiderata such as fairness or accountability are crucial. We suggest that the best strategy for achieving system transparency varies depending on the specific source of opacity prevalent in a given context. Synthesizing and extending existing discussions, we propose a taxonomy consisting of eight sources of opacity that fall into three main categories: architectural, analytical, and socio-technical. For each (...) source, we provide initial suggestions as to how to address the resulting opacity in practice. The taxonomy provides a starting point for requirements engineers and other practitioners to understand contextually prevalent sources of opacity, and to select or develop appropriate strategies for overcoming them. (shrink)

The prospect of including artificial intelligence (AI) in clinical decision-making is an exciting next step for some areas of healthcare. This article provides an analysis of the available kinds of AI systems, focusing on macro-level characteristics. This includes examining the strengths and weaknesses of opaque systems and fully explainable systems. Ultimately, the article argues that “grey box” systems, which include some combination of opacity and transparency, ought to be used in healthcare settings.

In their article ‘Who is afraid of black box algorithms? On the epistemological and ethical basis of trust in medical AI’, Durán and Jongsma discuss the epistemic and ethical challenges raised by black box algorithms in medical practice. The opacity of black box algorithms is an obstacle to the trustworthiness of their outcomes. Moreover, the use of opaque algorithms is not normatively justified in medical practice. The authors introduce a formalism, called computational reliabilism, which allows generating justified beliefs on (...) the algorithm reliability and trustworthy outcomes of artificial intelligence systems by means of epistemic warrants, called reliability indicators. However, they remark the need for reliability indicators specific to black box algorithms and that justified knowledge is not sufficient to justify normatively the actions of the physicians using medical AI systems. Therefore, Durán and Jongsma advocate for a more transparent design and implementation of black box algorithms, providing a series of recommendations to mitigate the epistemic and ethical challenges behind their use in medical practice. In this response, I argue that a peculiar form of black box algorithm transparency, called design publicity, may efficiently implement these recommendations. Design publicity encodes epistemic, that is, reliability indicators, and ethical recommendations for black box algorithms by means of four subtypes of transparency. These target the values and goals, their translation into design requirements, the performance and consistency of the algorithm altogether. I discuss design publicity applying it to a use case focused on the automated classification of skin lesions from medical images. (shrink)

For years, the number of opaque algorithmic decision-making systems (ADM systems) with a large impact on society has been increasing: e.g., systems that compute decisions about future recidivism of criminals, credit worthiness, or the many small decision computing systems within social networks that create rankings, provide recommendations, or filter content. Concerns that such a system makes biased decisions can be difficult to investigate: be it by people affected, NGOs, stakeholders, governmental testing and auditing authorities, or (...) other external parties. Scientific testing and auditing literature rarely focuses on the specific needs for such investigations and suffers from ambiguous terminologies. With this paper, we aim to support this investigation process by collecting, explaining, and categorizing methods of testing for bias, which are applicable to black-box systems, given that inputs and respective outputs can be observed. For this purpose, we provide a taxonomy that can be used to select suitable test methods adapted to the respective situation. This taxonomy takes multiple aspects into account, for example the effort to implement a given test method, its technical requirement (such as the need of ground truth) and social constraints of the investigation, e.g., the protection of business secrets. Furthermore, we analyze which test method can be used in the context of which black box audit concept. It turns out that various factors, such as the type of black box audit or the lack of an oracle, may limit the selection of applicable tests. With the help of this paper, people or organizations who want to test an ADM system for bias can identify which test methods and auditing concepts are applicable and what implications they entail. (shrink)

In this paper we analyse the conditions for attributing to AI autonomous systems the ontological status of “artificial moral agents”, in the context of the “distributed responsibility” between humans and machines in Machine Ethics (ME). In order to address the fundamental issue in ME of the unavoidable “opacity” of their decisions with ethical/legal relevance, we start from the neuroethical evidence in cognitive science. In humans, the “transparency” and then the “ethical accountability” of their actions as responsible moral agents is (...) not in contradiction with the unavoidable “opacity” (unawareness) of the brain process by which they perform their moral judgements on the right action to execute. In fact, the moral accountability of our actions depends on what is immediately before and after our “moral judgements” on the right action to execute (formally, deontic first order logic (FOL) decisions). I.e., our moral accountability depends on the “ethical constraints” we imposed to our judgement before performing it in an opaque way. Anyway, our moral accountability depends overall on the “ethical assessment” or explicit “moral reasoning” after and over the moral judgement before executing our actions (deontic higher order logic (HOL) assessment). In this way, in the light of the AI “imitation game”, the consistent attribution of the status of ethically accountable artificial moral agents to autonomous AI systems depends on two similar conditions. Firstly, it depends on the presence of “ethical constraints” to be satisfied in their Machine Learning (ML) supervised optimization algorithm during its training phase, to give the system ethical skills (“competences”) in its decisions. Secondly – and definitely—, it depends on the presence in an AI autonomous system of a deontic HOL “ethical reasoner” to perform an automatic, and fully transparent assessment (metalogical deontic valuation) about the decisions taken by the ethically skilled ML algorithm about the right action to execute, before executing it. Finally, we show that the proper deontic FOL and HOL for this class of artificial moral agents is Kripke’s modal relational logic, in its algebraic topological formalization. This is naturally implemented in the dissipative QFT unsupervised deep learning of our brains, based on the “doubling of the degrees of freedom” (DDF), and then in the so-called “deep-belief” artificial neural networks for the statistical data pre-processing. This unsupervised learning procedure is also compliant with the usage of the “maximin fairness principle”, used as a balancing aggregation principle of the statistical variables in Sen’s formal theory of fairness. (shrink)

Current research in Explainable AI includes post-hoc explanation methods that focus on building transparent explaining agents able to emulate opaque ones. Such agents are naturally required to be accurate and trustworthy. However, what it means for an explaining agent to be accurate and trustworthy is far from being clear. We characterize accuracy and trustworthiness as measures of the distance between the formal properties of a given opaque system and those of its transparent explanantes. To this aim, we extend (...) Probabilistic Computation Tree Logic with operators to specify degrees of accuracy and trustworthiness of explaining agents. We also provide a semantics for this logic, based on a multi-agent structure and relative model-checking algorithms. The paper concludes with a simple example of a possible application. (shrink)

The ongoing explosion of interest in artificial intelligence is fueled in part by recently developed techniques in machine learning. Those techniques allow automated systems to process huge amounts of data, utilizing mathematical methods that depart from traditional statistical approaches, and resulting in impressive advancements in our ability to make predictions and uncover correlations across a host of interesting domains. But as is now widely discussed, the way that those systems arrive at their outputs is often opaque, even (...) to the experts who design and deploy them. Is it morally problematic to make use of opaque automated methods when making high-stakes decisions, like whether to issue a loan to an applicant, or whether to approve a parole request? Many scholars answer in the affirmative. However, there is no widely accepted explanation for why transparent systems are morally preferable to opaque systems. We argue that the use of automated decision-making systems sometimes violates duties of consideration that are owed by decision-makers to decision-subjects, duties that are both epistemic and practical in character. Violations of that kind generate a weighty consideration against the use of opaque decision systems. In the course of defending our approach, we show that it is able to address three major challenges sometimes leveled against attempts to defend the moral import of transparency in automated decision-making. (shrink)

As opaque algorithmic systems take up a larger and larger role in shaping our lives, calls for explainability in various algorithmic systems have increased. Many moral and political philosophers have sought to vindicate these calls for explainability by developing theories on which decision-subjects—that is, individuals affected by decisions—have a moral right to the explanation of the systems that affect them. Existing theories tend to suggest that the right to explanation arises solely in virtue of facts about (...) how decision-subjects are affected by opaque systems. But this ignores an important part of the normative landscape: sometimes, facts about a decision-maker’s rights and freedoms can make a difference to whether she owes anyone explanations of her decisions. I illustrate the point by calling attention to examples of artistic, romantic, and supererogatory altruistic decisions; in at least some cases of this sort, even highly influential decisions can legitimately be kept entirely private and opaque. (shrink)

Deepfakes pose a multi-faceted threat to the acquisition of knowledge. It is widely hoped that technological solutions—in the form of artificially intelligent systems for detecting deepfakes—will help to address this threat. I argue that the prospects for purely technological solutions to the problem of deepfakes are dim. Especially given the evolving nature of the threat, technological solutions cannot be expected to prevent deception at the hands of deepfakes, or to preserve the authority of video footage. Moreover, the success of (...) such technologies depends on institutional trust that is in short supply. Finally, outsourcing the discrimination between the real and the fake to automated, largely opaque systems runs the risk of undermining epistemic autonomy. (shrink)

In Spinoza’s system, the identity of mental modes and extended modes is suggested, but a formal argument for its truth is difficult to extract. One prima facie difficulty for the claim that mental and extended modes are identical is that substitution of co-referential terms in contexts which are specific to thought or extension fails to preserve truth value. Della Rocca has answered this challenge by claiming that Spinoza relies upon referentially opaque contexts. In this essay, I defend this solution (...) by analyzing what is required to establish that Spinoza recognizes referentially opaque contexts as part of his system One objection that has been made to Della Rocca’s account is that he only establishes the intelligibility, not the actuality, of such mode-identity. In this essay, I argue that the intelligibility of mode-identity is sufficient to establish the existence of opaque attribute-specific contexts. (shrink)

Assistive systems based on Artificial Intelligence (AI) are bound to reshape decision-making in all areas of society. One of the most intricate challenges arising from their implementation in high-stakes environments such as medicine concerns their frequently unsatisfying levels of explainability, especially in the guise of the so-called black-box problem: highly successful models based on deep learning seem to be inherently opaque, resisting comprehensive explanations. This may explain why some scholars claim that research should focus on rendering AI (...) class='Hi'>systems understandable, rather than explainable. Yet, there is a grave lack of agreement concerning these terms in much of the literature on AI. We argue that the seminal distinction made by the philosopher and physician Karl Jaspers between different types of explaining and understanding in psychopathology can be used to promote greater conceptual clarity in the context of Machine Learning (ML). Following Jaspers, we claim that explaining and understanding constitute multi-faceted epistemic approaches that should not be seen as mutually exclusive, but rather as complementary ones as in and of themselves they are necessarily limited. Drawing on the famous example of Watson for Oncology we highlight how Jaspers’ methodology translates to the case of medical AI. Classical considerations from the philosophy of psychiatry can therefore inform a debate at the centre of current AI ethics, which in turn may be crucial for a successful implementation of ethically and legally sound AI in medicine. (shrink)

The ongoing explosion of interest in artificial intelligence is fueled in part by recently developed techniques in machine learning. Those techniques allow automated systems to process huge amounts of data, utilizing mathematical methods that depart from traditional statistical approaches, and resulting in impressive advancements in our ability to make predictions and uncover correlations across a host of interesting domains. But as is now widely discussed, the way that those systems arrive at their outputs is often opaque, even (...) to the experts who design and deploy them. Is it morally problematic to make use of opaque automated methods when making high-stakes decisions, like whether to issue a loan to an applicant, or whether to approve a parole request? Many scholars answer in the affirmative. However, there is no widely accepted explanation for why transparent systems are morally preferable to opaque systems. We argue that the use of automated decision-making systems sometimes violates duties of consideration that are owed by decision-makers to decision-subjects, duties that are both epistemic and practical in character. Violations of that kind generate a weighty consideration against the use of opaque decision systems. In the course of defending our approach, we show that it is able to address three major challenges sometimes leveled against attempts to defend the moral import of transparency in automated decision-making. (shrink)

The system of clinical ethics committees (CECs) in the United Kingdom is based on goodwill. No formal requirements exist as to constitution, membership, range of expertise or the status of their recommendations. Healthcare professionals are not obliged to use CECs where they exist, nor to follow any advice received. In addition, the make-up of CECs suggests that ethics itself may be under-represented. In most cases, there is one member with a training in ethics – the rest are healthcare professionals or (...) administrators, although a lawyer is generally also included in the membership. This begs the question as whether CECs can ‘do ethics’, as well whether or not they can take seriously the requirements of due process, formal justice and human rights. Moreover, the role of the patient in this system is opaque. (shrink)

As systems based on opaque Artificial Intelligence (AI) continue to flourish in diverse real-world applications, understanding these black box models has become paramount. In response, Explainable AI (XAI) has emerged as a field of research with practical and ethical benefits across various domains. This paper not only highlights the advancements in XAI and its application in real-world scenarios but also addresses the ongoing challenges within XAI, emphasizing the need for broader perspectives and collaborative efforts. We bring together experts (...) from diverse fields to identify open problems, striving to synchronize research agendas and accelerate XAI in practical applications. By fostering collaborative discussion and interdisciplinary cooperation, we aim to propel XAI forward, contributing to its continued success. Our goal is to put forward a comprehensive proposal for advancing XAI. To achieve this goal, we present a manifesto of 27 open problems categorized into nine categories. These challenges encapsulate the complexities and nuances of XAI and offer a road map for future research. For each problem, we provide promising research directions in the hope of harnessing the collective intelligence of interested stakeholders. (shrink)

Opaque value chains as well as environmental, ethical and health issues and food scandals are decreasing consumer trust in conventional agriculture and the dominant food system. As a result, critical consumers are increasingly turning to community-supported agriculture (CSA) to reconnect with producers and food. CSA is often perceived as a more sustainable, localized mode of food production, providing transparent production or social interaction between consumers and producers. This enables consumers to observe where their food is coming from, which means (...) CSA is considered suitable for building trust in food (production). However, it remains unclear how exactly members’ trust in ‘their’ farmers is built. To determine the factors that predict members’ trust in CSA and its farmers, and the importance of these factors when compared to each other, we conducted a quantitative study among CSA members in Germany and applied a multiple regression model (n = 790). The analysis revealed that trust in CSA and its farmers is influenced by “reputation”, “supply of information”, “direct social interaction” and the “duration of CSA membership”. Other factors such as the “certification status of the CSA farm” and “attitudes toward organic certification” did not significantly predict trust. We conclude that producers’ willingness to be transparent already signals trustworthiness to CSA members and is more important to members than formal signals. Other actors within the food system could learn from CSA principles and foster a transition toward a more regionalized value-based food system to help restore agriculture’s integrity. (shrink)

Artificial intelligent (AI) systems that perform image classification tasks are being used to great success in many application contexts. However, many of these systems are opaque, even to experts. This lack of understanding can be problematic for ethical, legal, or practical reasons. The research field Explainable AI (XAI) has therefore developed several approaches to explain image classifiers. The hope is to bring about understanding, e.g., regarding why certain images are classified as belonging to a particular target class. (...) Most of these approaches use visual explanations. Drawing on Elgin’s work (True enough. MIT Press, Cambridge, 2017), I argue that analyzing what those explanations exemplify can help to assess their suitability for producing understanding. More specifically, I suggest to distinguish between two forms of examples according to their suitability for producing understanding. I call these forms samples and exemplars, respectively. Samples are prone to misinterpretation and thus carry the risk of leading to misunderstanding. Exemplars, by contrast, are intentionally designed or chosen to meet contextual requirements and to mitigate the risk of misinterpretation. They are thus preferable for bringing about understanding. By reviewing several XAI approaches directed at image classifiers, I show that most of them explain with samples. If my analysis is correct, it will be beneficial if such explainability methods use explanations that qualify as exemplars. (shrink)

This paper reports on an experimental study investigating the acquisition of grammatical gender in Russian by heritage speakers living in Norway. The participants are 54 Norwegian-Russian bilingual children (4;0-10;2) as well as 107 Russian monolingual controls (3;0-7;0). Previous research has shown that grammatical gender is problematic for bilingual speakers, especially in cases where gender assignment is opaque (Schwartz et al., 2015; Polinsky, 2008; Rodina and Westergaard, 2017). Furthermore, factors such as proficiency and family type (one or two Russian-speaking parents) (...) have been argued to be important. Interestingly, previous findings differ with respect to the kind of errors children make: restructuring to a two-gender system (masculine-feminine, see Polinsky, 2008) or defaulting to masculine (see Rodina and Westergaard, 2017). It is also not clear to what extent children are sensitive to gender cues or whether certain agreement patterns are simply memorized. To investigate this, we used both existing nouns and nonce words and tested both transparent and opaque gender cues. The results were checked against a number of background factors measuring exposure, proficiency, and dominance. Our findings show that bilingual children are clearly sensitive to morphophonological cues for gender assignment. The most common and robust error pattern for all bilinguals involved overgeneralization to masculine (especially affecting neuter and opaque nouns). At the same time, children from families with two Russian-speaking parents and monolinguals also occasionally overused feminine with vowel-final nouns. The following variables were found to be the most reliable predictors of accuracy on grammatical gender tasks: cumulative length of exposure (CLoE) and consistency of input in Russian, as well as the presence of older siblings, with CLoE to Russian being by far the most robust and important predictor. Furthermore, we show that a lexical diversity measure (number of different words in a Russian narrative) is also correlated significantly with the children’s performance on the gender tasks. At the same time, our results indicate that relative measures of dominance (e.g., the difference in exposure between the two languages or the difference in narrative scores) may be redundant when more robust absolute measures are present (CLoE and lexical diversity in the heritage language). (shrink)

Many of the computing systems programmed using Machine Learning are opaque: it is difficult to know why they do what they do or how they work. Explainable Artificial Intelligence aims to develop analytic techniques that render opaque computing systems transparent, but lacks a normative framework with which to evaluate these techniques’ explanatory successes. The aim of the present discussion is to develop such a framework, paying particular attention to different stakeholders’ distinct explanatory requirements. Building on an (...) analysis of “opacity” from philosophy of science, this framework is modeled after accounts of explanation in cognitive science. The framework distinguishes between the explanation-seeking questions that are likely to be asked by different stakeholders, and specifies the general ways in which these questions should be answered so as to allow these stakeholders to perform their roles in the Machine Learning ecosystem. By applying the normative framework to recently developed techniques such as input heatmapping, feature-detector visualization, and diagnostic classification, it is possible to determine whether and to what extent techniques from Explainable Artificial Intelligence can be used to render opaque computing systems transparent and, thus, whether they can be used to solve the Black Box Problem. (shrink)

Many AI systems that make important decisions are black boxes: how they function is opaque even to their developers. This is due to their high complexity and to the fact that they are trained rather than programmed. Efforts to alleviate the opacity of black box systems are typically discussed in terms of transparency, interpretability, and explainability. However, there is little agreement about what these key concepts mean, which makes it difficult to adjudicate the success or promise of (...) opacity alleviation methods. I argue for a unified account of these key concepts that treats the concept of understanding as fundamental. This allows resources from the philosophy of science and the epistemology of understanding to help guide opacity alleviation efforts. A first significant benefit of this understanding account is that it defuses one of the primary, in-principle objections to post hoc explainable AI (XAI) methods. This “rationalization objection” argues that XAI methods provide mere rationalizations rather than genuine explanations. This is because XAI methods involve using a separate “explanation” system to approximate the original black box system. These explanation systems function in a completely different way than the original system, yet XAI methods make inferences about the original system based on the behavior of the explanation system. I argue that, if we conceive of XAI methods as idealized scientific models, this rationalization worry is dissolved. Idealized scientific models misrepresent their target phenomena, yet are capable of providing significant and genuine understanding of their targets. (shrink)

Decision-making assisted by algorithms developed by machine learning is increasingly determining our lives. Unfortunately, full opacity about the process is the norm. Would transparency contribute to restoring accountability for such systems as is often maintained? Several objections to full transparency are examined: the loss of privacy when datasets become public, the perverse effects of disclosure of the very algorithms themselves, the potential loss of companies’ competitive edge, and the limited gains in answerability to be expected since sophisticated algorithms usually (...) are inherently opaque. It is concluded that, at least presently, full transparency for oversight bodies alone is the only feasible option; extending it to the public at large is normally not advisable. Moreover, it is argued that algorithmic decisions preferably should become more understandable; to that effect, the models of machine learning to be employed should either be interpreted ex post or be interpretable by design ex ante. (shrink)

This book challenges the standard view that modern hermeneutics begins with Friedrich Ast and Friedrich Schleiermacher, arguing instead that it is the dialectic of reflective and teleological reason in Kant’s Critique of Judgment that provides the actual proto-hermeneutic foundation. It is revolutionary in doing so by replacing interpretive truth claims by the more appropriate claim of rendering opaque contexts intelligible. Taking Gadamer’s comprehensive analysis of hermeneutics in Truth and Method (1960) as its point of departure, the book turns to (...) Kant’s Critiques, reviewing his major concepts as a coherent system in relation to his sensus communis. At the heart of the book is the interaction between reflective, bottom-up search and teleological, top-down interpretative projection as provided in Part II of the third Critique. This text contends that Kant’s broad definition of nature invites the liberation of the reflective-teleological judgment from its biological exemplifications and so permits us to establish its generalised status as a path-breaking, methodological tool. Kant’s dialectic of reflective search and meaning bestowing, stipulated teleology is asserted to anticipate a series of motifs commonly associated with hermeneutics. Figures covered include Dilthey, Husserl, Ingarden, Heidegger, Gadamer, Apel, Habermas, Ricoeur, Derrida, Foucault, Lyotard, Deleuze, Vattimo, Nancy and Caputo. Their collective contributions to interpretation allow for a review of the evolution of hermeneutics from the perspective of the Kantian critique of the limitations of human cognition. The book is written for the informed, general reader, but will likewise appeal to advanced undergraduate and graduate students as well as researchers in the humanities and social sciences. (shrink)

The apparent downsides of opaque algorithms has led to a demand for explainable AI (XAI) methods by which a user might come to understand why an algorithm produced the particular output it did, given its inputs. Patients, for example, might find that the lack of explanation of the process underlying the algorithmic recommendations for diagnosis and treatment hinders their ability to provide informed consent. This paper examines the impact of two factors on user perceptions of explanations for AI (...) class='Hi'>systems in medical contexts. The factors considered were the stakes of the decision—high versus low—and the decision source—human versus AI. 484 participants were presented with vignettes in which medical diagnosis and treatment plan recommendations were made by humans or by AI. Separate vignettes were used for high stakes scenarios involving life-threatening diseases, and low stakes scenarios involving mild diseases. In each vignette, an explanation for the decision was given. Four explanation types were tested across separate vignettes: no explanation, counterfactual, causal and a novel ‘narrative-based’ explanation, not previously considered. This yielded a total of 16 conditions, of which each participant saw only one. Individuals were asked to evaluate the explanations they received based on helpfulness, understanding, consent, reliability, trust, interests and likelihood of undergoing treatment. We observed a main effect for stakes on all factors and a main effect for decision source on all factors except for helpfulness and likelihood to undergo treatment. While we observed effects for explanation on helpfulness, understanding, consent, reliability, trust and interests, we by and large did not see any differences between the effects of explanation types. This suggests that the effectiveness of explanations may not depend on type of explanation but instead, on the stakes and decision source. (shrink)

Artificial intelligence (AI) systems are increasingly adopted to make decisions in domains such as business, education, health care, and criminal justice. However, such algorithmic decision systems can have prevalent biases against marginalized social groups and undermine social justice. Explainable artificial intelligence (XAI) is a recent development aiming to make an AI system’s decision processes less opaque and to expose its problematic biases. This paper argues against technical XAI, according to which the detection and interpretation of algorithmic bias (...) can be handled more or less independently by technical experts who specialize in XAI methods. Drawing on resources from feminist epistemology, we show why technical XAI is mistaken. Specifically, we demonstrate that the proper detection of algorithmic bias requires relevant interpretive resources, which can only be made available, in practice, by actively involving a diverse group of stakeholders. Finally, we suggest how feminist theories can help shape integrated XAI: an inclusive social-epistemic process that facilitates the amelioration of algorithmic bias. (shrink)

What is it to interpret the outputs of an opaque machine learning model? One approach is to develop interpretable machine learning techniques. These techniques aim to show how machine learning models function by providing either model-centric local or global explanations, which can be based on mechanistic interpretations (revealing the inner working mechanisms of models) or non-mechanistic approximations (showing input feature–output data relationships). In this paper, we draw on social philosophy to argue that interpreting machine learning outputs in certain normatively (...) salient domains could require appealing to a third type of explanation that we call “socio-structural” explanation. The relevance of this explanation type is motivated by the fact that machine learning models are not isolated entities but are embedded within and shaped by social structures. Socio-structural explanations aim to illustrate how social structures contribute to and partially explain the outputs of machine learning models. We demonstrate the importance of socio-structural explanations by examining a racially biased healthcare allocation algorithm. Our proposal highlights the need for transparency beyond model interpretability: understanding the outputs of machine learning systems could require a broader analysis that extends beyond the understanding of the machine learning model itself. (shrink)

When is it justified to use opaque artificial intelligence (AI) output in medical decision-making? Consideration of this question is of central importance for the responsible use of opaque machine learning (ML) models, which have been shown to produce accurate and reliable diagnoses, prognoses, and treatment suggestions in medicine. In this article, I discuss the merits of two answers to the question. According to the Explanation View, clinicians must have access to an explanation of why an output was produced. (...) According to the Validation View, it is sufficient that the AI system has been validated using established standards for safety and reliability. I defend the Explanation View against two lines of criticism, and I argue that within the framework of evidence-based medicine mere validation seems insufficient for the use of AI output. I end by characterizing the epistemic responsibility of clinicians and point out how a mere AI output cannot in itself ground a practical conclusion about what to do. (shrink)

Research under real-world conditions is crucial to the development and deployment of robust AI systems. Exposing large language models to complex use settings yields knowledge about their performance and impact, which cannot be obtained under controlled laboratory conditions or through anticipatory methods. This epistemic need for real-world research is exacerbated by large-language models’ opaque internal operations and potential for emergent behavior. However, despite its epistemic value and widespread application, the ethics of real-world AI research has received little scholarly (...) attention. To address this gap, this paper provides an analysis of real-world research with LLMs and generative AI, assessing both its epistemic value and ethical concerns such as the potential for interpersonal and societal research harms, the increased privatization of AI learning, and the unjust distribution of benefits and risks. This paper discusses these concerns alongside four moral principles influencing research ethics standards: non-maleficence, beneficence, respect for autonomy, and distributive justice. I argue that real-world AI research faces challenges in meeting these principles and that these challenges are exacerbated by absent or imperfect current ethical governance. Finally, I chart two distinct but compatible ways forward: through ethical compliance and regulation and through moral education and cultivation. (shrink)

Although decision‐making algorithms are not new to medicine, the availability of vast stores of medical data, gains in computing power, and breakthroughs in machine learning are accelerating the pace of their development, expanding the range of questions they can address, and increasing their predictive power. In many cases, however, the most powerful machine learning techniques purchase diagnostic or predictive accuracy at the expense of our ability to access “the knowledge within the machine.” Without an explanation in terms of reasons or (...) a rationale for particular decisions in individual cases, some commentators regard ceding medical decision‐making to black box systems as contravening the profound moral responsibilities of clinicians. I argue, however, that opaque decisions are more common in medicine than critics realize. Moreover, as Aristotle noted over two millennia ago, when our knowledge of causal systems is incomplete and precarious—as it often is in medicine—the ability to explain how results are produced can be less important than the ability to produce such results and empirically verify their accuracy. (shrink)

Decision-making assisted by algorithms developed by machine learning is increasingly determining our lives. Unfortunately, full opacity about the process is the norm. Would transparency contribute to restoring accountability for such systems as is often maintained? Several objections to full transparency are examined: the loss of privacy when datasets become public, the perverse effects of disclosure of the very algorithms themselves, the potential loss of companies’ competitive edge, and the limited gains in answerability to be expected since sophisticated algorithms usually (...) are inherently opaque. It is concluded that, at least presently, full transparency for oversight bodies alone is the only feasible option; extending it to the public at large is normally not advisable. Moreover, it is argued that algorithmic decisions preferably should become more understandable; to that effect, the models of machine learning to be employed should either be interpreted ex post or be interpretable by design ex ante. (shrink)

The initial successes in recent years in harnessing machine learning technologies to improve medical practice and benefit patients have attracted attention in a wide range of healthcare fields. Particularly, it should be achieved by providing automated decision recommendations to the treating clinician. Some hopes placed in such ML-based systems for healthcare, however, seem to be unwarranted, at least partially because of their inherent lack of transparency, although their results seem convincing in accuracy and reliability. Skepticism arises when the physician (...) as the agent responsible for the implementation of diagnosis, therapy, and care is unable to access the generation of findings and recommendations. There is widespread agreement that, generally, a complete traceability is preferable to opaque recommendations; however, there are differences about addressing ML-based systems whose functioning seems to remain opaque to some degree—even if so-called explicable or interpretable systems gain increasing amounts of interest. This essay approaches the epistemic foundations of ML-generated information specifically and medical knowledge generally to advocate differentiations of decision-making situations in clinical contexts regarding their necessary depth of insight into the process of information generation. Empirically accurate or reliable outcomes are sufficient for some decision situations in healthcare, whereas other clinical decisions require extensive insight into ML-generated outcomes because of their inherently normative implications. (shrink)

Decision-making assisted by algorithms developed by machine learning is increasingly determining our lives. Unfortunately, full opacity about the process is the norm. Would transparency contribute to restoring accountability for such systems as is often maintained? Several objections to full transparency are examined: the loss of privacy when datasets become public, the perverse effects of disclosure of the very algorithms themselves (“gaming the system” in particular), the potential loss of companies’ competitive edge, and the limited gains in answerability to be (...) expected since sophisticated algorithms usually are inherently opaque. It is concluded that, at least presently, full transparency for oversight bodies alone is the only feasible option; extending it to the public at large is normally not advisable. Moreover, it is argued that algorithmic decisions preferably should become more understandable; to that effect, the models of machine learning to be employed should either be interpreted ex post or be interpretable by design ex ante. (shrink)