A morally acceptable course of AI development should avoid two dangers: creating unaligned AI systems that pose a threat to humanity and mistreating AI systems that merit moral consideration in their own right. This paper argues these two dangers interact and that if we create AI systems that merit moral consideration, simultaneously avoiding both of these dangers would be extremely challenging. While our argument is straightforward and supported by a wide range of pretheoretical moral judgments, it has far-reaching moral implications (...) for AI development. Although the most obvious way to avoid the tension between alignment and ethical treatment would be to avoid creating AI systems that merit moral consideration, this option may be unrealistic and is perhaps fleeting. So, we conclude by offering some suggestions for other ways of mitigating mistreatment risks associated with alignment. (shrink)

Roughly, empirical AI alignment research (AIA) is an area of AI research which investigates empirically how to design AI systems in line with human goals. This paper examines the role of non-epistemic values in AIA. It argues that: (1) Sciences differ in the degree to which values influence them. (2) AIA is strongly value-laden. (3) This influence of values is managed inappropriately and thus threatens AIA’s epistemic integrity and ethical beneficence. (4) AIA should strive to achieve value transparency, critical (...) scrutiny from inside and outside the discipline – involving the public –, and to empower actors without strong commercial interests. (shrink)

The AI and AI alignment communities have been instrumental in addressing existential risks, developing alignment methodologies, and promoting rationalist problem-solving approaches. However, as AI research ventures into increasingly uncertain domains, there is a risk of premature epistemic convergence, where prevailing methodologies influence not only the evaluation of ideas but also determine which ideas are considered within the discourse. This paper examines critical epistemic blind spots in AI alignment research, particularly the lack of predictive frameworks to differentiate problems (...) necessitating general intelligence, decentralized intelligence, or innovative alignment methodologies. We analyze how heuristic-based epistemic filtering—favoring ideas that conform to established norms over those assessed purely on logical merit—may inadvertently constrain the field's potential for groundbreaking solutions. The semblance of rigor, wherein social consensus and stylistic conformity replace foundational reasoning, further intensifies this concern. To mitigate these issues, we advocate for an epistemically adaptive research environment that emphasizes structured evaluation of paradigm-challenging ideas, cross-disciplinary collaboration, and strategies to identify and reduce epistemic overfitting. By broadening the epistemic frameworks governing AI alignment discourse, the field can progress beyond incremental improvements toward discovering entirely new solution paradigms. Maintaining epistemic inclusivity in AI safety research is not only an intellectual necessity but also a crucial step in preparing alignment efforts for the challenges posed by rapid AI advancements. (shrink)

New AI technologies have the potential to cause unintended harms in diverse domains including warfare, judicial sentencing, biomedicine and governance. One strategy for realising the benefits of AI whilst avoiding its potential dangers is to ensure that new AIs are properly ‘aligned’ with some form of ‘alignment target.’ One danger of this strategy is that – dependent on the alignment target chosen – our AIs might optimise for objectives that reflect the values only of a certain subset of (...) society, and that do not take into account alternative views about what constitutes desirable and safe behaviour for AI agents. In response to this problem, several AI ethicists have suggested alignment targets that are designed to be sensitive to widespread normative disagreement amongst the relevant stakeholders. Authors inspired by voting theory have suggested that AIs should be aligned with the verdicts of actual or simulated ‘moral parliaments’ whose members represent the normative views of the relevant stakeholders. Other authors inspired by decision theory and the philosophical literature on moral uncertainty have suggested that AIs should maximise socially expected choiceworthiness. In this paper, I argue that both of these proposals face several important problems. In particular, they fail to select attractive ‘compromise options’ in cases where such options are available. I go on to propose and defend an alternative, bargaining-theoretic alignment target, which avoids the problems associated with the voting- and decision-theoretic approaches. (shrink)

The dominant practice of AI alignment assumes (1) that preferences are an adequate representation of human values, (2) that human rationality can be understood in terms of maximizing the satisfaction of preferences, and (3) that AI systems should be aligned with the preferences of one or more humans to ensure that they behave safely and in accordance with our values. Whether implicitly followed or explicitly endorsed, these commitments constitute what we term apreferentistapproach to AI alignment. In this paper, (...) we characterize and challenge the preferentist approach, describing conceptual and technical alternatives that are ripe for further research. We first survey the limits of rational choice theory as a descriptive model, explaining how preferences fail to capture the thick semantic content of human values, and how utility representations neglect the possible incommensurability of those values. We then critique the normativity of expected utility theory (EUT) for humans and AI, drawing upon arguments showing how rational agents need not comply with EUT, while highlighting how EUT is silent on which preferences are normatively acceptable. Finally, we argue that these limitations motivate a reframing of the targets of AI alignment: Instead of alignment with the preferences of a human user, developer, or humanity-writ-large, AI systems should be aligned with normative standards appropriate to their social roles, such as the role of a general-purpose assistant. Furthermore, these standards should be negotiated and agreed upon by all relevant stakeholders. On this alternative conception of alignment, a multiplicity of AI systems will be able to serve diverse ends, aligned with normative standards that promote mutual benefit and limit harm despite our plural and divergent values. (shrink)

We argue that the later Wittgenstein’s philosophy of language and mathematics, substantially focused on rule-following, is relevant to understand and improve on the Artificial Intelligence (AI) alignment problem: his discussions on the categories that influence alignment between humans can inform about the categories that should be controlled to improve on the alignment problem when creating large data sets to be used by supervised and unsupervised learning algorithms, as well as when introducing hard coded guardrails for AI models. (...) We cast these considerations in a model of human–human and human–machine alignment and sketch basic alignment strategies based on these categories and further reflections on rule-following like the notion of meaning as use. To sustain the validity of these considerations, we also show that successful techniques employed by AI safety researchers to better align new AI systems with our human goals are congruent with the stipulations that we derive from the later Wittgenstein’s philosophy. However, their application may benefit from the added specificities and stipulations of our framework: it extends on the current efforts and provides further, specific AI alignment techniques. Thus, we argue that the categories of the model and the core alignment strategies presented in this work can inform further AI alignment techniques. (shrink)

Tae Wan Kim, John Hooker, and Thomas Donaldson make an attempt, in recent articles, to solve the alignment problem. As they define the alignment problem, it is the issue of how to give AI systems moral intelligence. They contend that one might program machines with a version of Kantian ethics cast in deontic modal logic. On their view, machines can be aligned with human values if such machines obey principles of universalization and autonomy, as well as a deontic (...) utilitarian principle. Programming machines to do so might be useful, in their view, for applications such as future autonomous vehicles. Their proposal draws both on traditional logic-based and contemporary connectionist approaches, to fuse factual information with normative principles. I will argue that this approach makes demands of machines that go beyond what is currently feasible, and may extend past the limits of the possible for AI. I also argue that a deontological ethics for machines should place greater stress on the formula of humanity of the Kantian categorical imperative. On this principle, one ought never treat a person as a mere means. Recognition of what makes a person a person requires ethical insight. Similar insight is needed to tell treatment as a means from treatment as a mere means. The resources in Kim, Hooker, and Donaldson’s approach is insufficient for this reason. Hesitation regarding deployment of autonomous machines is warranted in light of these alignment concerns. (shrink)

Foundation models such as GPT-4 are fine-tuned to avoid unsafe or otherwise problematic behavior, such as helping to commit crimes or producing racist text. One approach to fine-tuning, called reinforcement learning from human feedback, learns from humans' expressed preferences over multiple outputs. Another approach is constitutional AI, in which the input from humans is a list of high-level principles. But how do we deal with potentially diverging input from humans? How can we aggregate the input into consistent data about "collective" (...) preferences or otherwise use it to make collective choices about model behavior? In this paper, we argue that the field of social choice is well positioned to address these questions, and we discuss ways forward for this agenda, drawing on discussions in a recent workshop on Social Choice for AI Ethics and Safety held in Berkeley, CA, USA in December 2023. (shrink)

The problem of AI alignment has parallels in Kantian ethics and can benefit from its concepts and arguments. The Kantian framework allows us to better answer the question of what exactly AI is being aligned to, what are the problems of alignment of rational agents in general, and what are the prospects for achieving a state of alignment. Having described the state of discussions about alignment in AI, I will reformulate them in Kantian terms. Thus, the (...) process of alignment is captured by the concept of enlightenment, and for the final state of alignment in Kant’s lexicon there is the concept of the “kingdom of ends.” I will argue that the discourse of alignment and the Kantian ethical program 1) are devoted to the same general end of harmonizing the thinking and acting of rational agents, 2) encounter similar difficulties, well known in the Kantian discussions with its comparatively longer history, and 3) for a number of reasons lying on the side of humanity, do not have and, despite the hopes and attitudes of some participants in the AI discussions, will not have a theoretically rigorous, harmonious and practically implementable, conflict-free solution – alignment will remain a regulative idea in the Kantian sense, but will not become a reality. (shrink)

The Alignment Problem in artificial intelligence concerns how to insure that artificial general intelligence (AGI) conforms to human goals and values and remains under human control. The concept of general intelligence, modelled on human and animal behavior, lacks coherence. The ideal of autonomy inherent in AGI conflicts with the ideal of external control. Truly autonomous agents are necessarily _embodied,_ but embodiment implies more than physical instantiation or sensory input. It means being an autopoietic system (like a natural organism), with (...) its own priorities and values, which may compete and conflict with those of humans. Ambiguous terms and concepts, and problematic notions such as ‘orthogonality’, are critically examined, as well as inconsistencies in thought and expectation concerning AGI. Two paradigmatic approaches to the Alignment Problem are compared: Stuart Russell’s and Eric Drexler’s. Large Language Models are discussed in the context of the Turing Test. It is concluded that task-oriented tools, not autonomous agents, should be the goal of AI research. (shrink)

The threat from possible future superintelligent AI has given rise to discussion of the so-called “value alignment problem”. This is the problem of how to ensure artificially intelligent systems align with human values, and thus (hopefully) mitigate risks associated with them. Naturally, AI value alignment is often discussed in relation to morally relevant values, such as the value of human lives or human wellbeing. However, solutions to the value alignment problem target all human values, not only morally (...) relevant ones. Is there a value alignment problem in other domains? In this paper, I explore whether the AI value alignment problem extends beyond morally relevant values to include aesthetic values. I demonstrate that the value alignment problem as typically framed includes aesthetic values, and, using examples from computer vision, put forward that AI may be misaligned with human values in the artistic realm. Whilst misalignment may be cause for concern when considering AI creativity, I argue that aesthetic value is a case in which we do not want AI to be fully aligned with human values. In doing so, I offer support to Peterson’s moderate value alignment thesis. (shrink)

When discussing AI alignment, we usually refer to the problem of teaching or training advanced autonomous AI systems to make decisions that are aligned with human values or preferences. Proponents of this approach believe it can be employed as means to stay in control over sophisticated intelligent systems, thus avoiding certain existential risks. We identify three general obstacles on the path to implementation of value alignment: a technological/technical obstacle, a normative obstacle, and a calibration problem. Presupposing, for the (...) purposes of this discussion, that the technical and normative problems are solved, we focus on the problem of how to calibrate a system, for a specific value, to be on a specific location within a spectrum stretching between righteous and normal or average human behavior. Calibration, or more specifically mis-calibration, also raises the issue of trustworthiness. If we cannot trust AI systems to perform tasks the way we intended, we would not use them on our roads and at our homes. In an era where we strive to construct autonomous machines endowed with common sense, reasoning abilities and a connection to the world, so they would be able to act in alignment with human values, such mis-calibrations can make the difference between trustworthy and untrustworthy systems. (shrink)

The challenge of AI alignment is not just a technological issue but fundamentally an epistemic one. AI safety research predominantly relies on empirical validation, often detecting failures only after they manifest. However, certain risks—such as deceptive alignment and goal misspecification—may not be empirically testable until it is too late, necessitating a shift toward leading-indicator logical reasoning. This paper explores how mainstream AI research systematically filters out deep epistemic insight, hindering progress in AI safety. We assess the rarity of (...) such insights, conduct an experiment testing large language models for epistemic blind spots, and propose structural reforms, including contrarian epistemic screening, decentralized collective intelligence mechanisms, and epistemic challenge platforms. Our findings suggest that while AGI may emerge through incremental engineering, ensuring its safe alignment likely requires an epistemic paradigm shift. (shrink)

As artificial intelligence (AI) continues to advance, one key challenge is ensuring that AI aligns with certain values. However, in the current diverse and democratic society, reaching a normative consensus is complex. This paper delves into the methodological aspect of how AI ethicists can effectively determine which values AI should uphold. After reviewing the most influential methodologies, we detail an intuitionist research agenda that offers guidelines for aligning AI applications with a limited set of reliable moral intuitions, each underlying a (...) refined cooperative view of AI. We discuss appropriate epistemic tools for collecting, filtering, and justifying moral intuitions with the aim of reducing cognitive and social biases. The proposed methodology facilitates a large collective participation in AI alignment, while ensuring the reliability of the considered moral judgments. (shrink)

Value alignment is essential for ensuring that AI systems act in ways that are consistent with human values. Existing approaches, such as reinforcement learning with human feedback and constitutional AI, however, exhibit power asymmetries and lack transparency. These “authoritarian” approaches fail to adequately accommodate a broad array of human opinions, raising concerns about whose values are being prioritized. In response, we introduce the Dynamic Value Alignment approach, theoretically grounded in the principles of parallel constraint satisfaction, which models moral (...) reasoning as a dynamic process that balances multiple value principles. Our approach also enhances users’ moral and epistemic agency by granting users greater control over the values that influence AI behavior. As a more user-centric, transparent, and participatory framework for AI ethics, our approach not only addresses the democratic deficits inherent in current practices but also ensures that AI systems are flexibly aligned with a diverse array of human values. (shrink)

This paper offers three contributions to the burgeoning movements of AI for Social Good (AI4SG) and AI and the United Nations Sustainable Development Goals (SDGs). First, we introduce the SDG-Intense Evaluation framework (SDGIE) that aims to situate variegated automated/AI models in a larger ecosystem of computational approaches to advance the SDGs. To foster knowledge collaboration for solving complex social and environmental problems encompassed by the SDGs, the SDGIE framework details a benchmark structure of data-algorithm-output to effectively standardize AI approaches to (...) the SDGs. Second, as a specific instantiation of the SDGIE framework, the SDG Impact Intensity Model (SDGIIM) is theoretically and operationally established. SDGIIM embeds expert decision-making and SDG keyword banks in textual data processing to determine overall SDG “impact intensity.” Ideally, SDGIIM can be applied to textual data sets from any sector or discipline: academia, business, government, non-profit, civil society, etc. Third, the SDGIIM instantiation is applied to the specific domain of academic journal rating systems as a case study. Traditionally, academic journals have been evaluated on loosely conceived and empirically shaky notions of ‘quality.’ Aligned with the trend of AI4SG and broader calls to action, ‘impact’ is rapidly becoming the primary normative consideration for assessing academic journals. We hypothesize and demonstrate that SDGIIM is capable of producing evaluations aligned with experts’ expectations of SDG impact intensity; the consistent analysis and rating of textual data sets that embody the SDGs with varying degrees of meaning and, ultimately, promote positive impact on the actual material conditions of the world. (shrink)

Artificial Intelligence (AI) must be directed at humane ends. The development of AI has produced great uncertainties of ensuring AI alignment with human values (AI value alignment) through AI operations from design to use. For the purposes of addressing this problem, we adopt the phenomenological theories of material values and technological mediation to be that beginning step. In this paper, we first discuss the AI value alignment from the relevant AI studies. Second, we briefly present what are (...) material values and technological mediation and reflect on the AI value alignment through the lenses of these theories. We conclude that a set of finite human values can be defined and adapted to the stable life tasks that AI systems will be called upon to accomplish. The AI value alignment can also be fostered between designers and users through technological mediation. Upon that foundation, we propose a set of common principles to understand the AI value alignment through phenomenological theories. This paper contributes the unique knowledge of phenomenological theories to the discourse on AI alignment with human values. (shrink)

Inchul Yum (2024) argues that the widespread adoption of language agent architectures would likely increase the risk posed by AI by simplifying the process of aligning artificial systems with human values and thereby making it easier for malicious actors to use them to cause a variety of harms. Yum takes this to be an example of a broader phenomenon: progress on the alignment problem is likely to be net safety-negative because it makes artificial systems easier for malicious actors to (...) control. I offer some reasons for skepticism about this surprising and pessimistic conclusion. (shrink)

This paper looks at philosophical questions that arise in the context of AI alignment. It defends three propositions. First, normative and technical aspects of the AI alignment problem are interrelated, creating space for productive engagement between people working in both domains. Second, it is important to be clear about the goal of alignment. There are significant differences between AI that aligns with instructions, intentions, revealed preferences, ideal preferences, interests and values. A principle-based approach to AI alignment, (...) which combines these elements in a systematic way, has considerable advantages in this context. Third, the central challenge for theorists is not to identify ‘true’ moral principles for AI; rather, it is to identify fair principles for alignment that receive reflective endorsement despite widespread variation in people’s moral beliefs. The final part of the paper explores three ways in which fair principles for AI alignment could potentially be identified. (shrink)

Providing meaningful and actionable explanations for end-users is a situated problem requiring the intersection of multiple disciplines to address social, operational, and technical challenges. However, the explainable artificial intelligence community has not commonly adopted or created tangible design tools that allow interdisciplinary work to develop reliable AI-powered solutions. This paper proposes a formative architecture that defines the explanation space from a user-inspired perspective. The architecture comprises five intertwined components to outline explanation requirements for a task: (1) the end-users’ mental models, (...) (2) the end-users’ cognitive process, (3) the user interface, (4) the Human-Explainer Agent, and (5) the agent process. We first define each component of the architecture. Then, we present the Abstracted Explanation Space, a modeling tool that aggregates the architecture’s components to support designers in systematically aligning explanations with end-users’ work practices, needs, and goals. It guides the specifications of what needs to be explained (content: end-users’ mental model), why this explanation is necessary (context: end-users’ cognitive process), to delimit how to explain it (format: Human-Explainer Agent and user interface), and when the explanations should be given. We then exemplify the tool’s use in an ongoing case study in the aircraft maintenance domain. Finally, we discuss possible contributions of the tool, known limitations or areas for improvement, and future work to be done. (shrink)

This paper uses famous problems from philosophy of science and philosophical psychology—underdetermination of theory by evidence, Nelson Goodman’s new riddle of induction, theory-ladenness of observation, and “Kripkenstein’s” rule-following paradox—to show that it is empirically impossible to reliably interpret which functions a large language model (LLM) AI has learned, and thus, that reliably aligning LLM behavior with human values is provably impossible. Sections 2 and 3 show that because of how complex LLMs are, researchers must interpret their learned functions largely in (...) terms of empirical observations of their outputs and network behavior. Sections 4–7 then show that for every “aligned” function that might appear to be confirmed by empirical observation, there is always an infinitely larger number of “misaligned”, arbitrarily time-limited functions equally consistent with the same data. Section 8 shows that, from an empirical perspective, we can thus never reliably infer that an LLM or subcomponent of one has learned any particular function at all before any of an uncountably large number of unpredictable future conditions obtain. Finally, Section 9 concludes that the probability of LLM “misalignment” is—at every point in time, given any arbitrarily large body of empirical evidence—always vastly greater than the probability of “alignment.”. (shrink)

The Salivaomics Knowledge Base (SKB) is designed to serve as a computational infrastructure that can permit global exploration and utilization of data and information relevant to salivaomics. SKB is created by aligning (1) the saliva biomarker discovery and validation resources at UCLA with (2) the ontology resources developed by the OBO (Open Biomedical Ontologies) Foundry, including a new Saliva Ontology (SALO). We define the Saliva Ontology (SALO; http://www.skb.ucla.edu/SALO/) as a consensus-based controlled vocabulary of terms and relations dedicated to the salivaomics (...) domain and to saliva-related diagnostics following the principles of the OBO (Open Biomedical Ontologies) Foundry. The Saliva Ontology is an ongoing exploratory initiative. The ontology will be used to facilitate salivaomics data retrieval and integration across multiple fields of research together with data analysis and data mining. The ontology will be tested through its ability to serve the annotation ('tagging') of a representative corpus of salivaomics research literature that is to be incorporated into the SKB. Background Saliva (oral fluid) is an emerging biofluid for non-invasive diagnostics used in the detection of human diseases. The need to advance saliva research is strongly emphasized by the National Institute of Dental and Craniofacial Research (NIDCR), and is included in the NIDCR's 2004- 2009 expert panel long-term research agenda [1]. The ability to monitor health status, disease onset, progression, recurrence and treatment outcome through noninvasive means is highly important to advancing health care management. Saliva is a perfect medium to be explored for personalized individual medicine including diagnostics, offering a non-invasive, easy to obtain means for detecting and monitoring diseases. Saliva testing potentially allows the patient to collect their own saliva samples at home, yielding convenience for the patient and savings in health costs, and facilitating multiple sampling. Specimen collection is less objectionable to patients and easier in children and elderly individuals. Due to these advantages. (shrink)

We outline a four-step process for ML/AI developers to align development choices with multiple values, by adapting a widely-utilized framework from bioethics: (1) identify the values that matter, (2) specify identified values, (3) find solution spaces that allow for maximal alignment with identified values, and 4) make hard choices if there are unresolvable trade-offs between the identified values. Key to this approach is identifying resolvable trade-offs between values (Step 3). We survey ML/AI methods that could be used to this (...) end, identifying approaches at each stage of the development process. All steps should be guided by community engagement. The framework outlines what it means to build a value-aligned ML/AI system, providing development teams with practical guidance to maximize the chances their work has desirable impacts. (shrink)

In lieu of an abstract, here is a brief excerpt of the content:Toleration and Justice in the Laozi:Engaging with Tao Jiang's Origins of Moral-Political Philosophy in Early ChinaAi Yuan (bio)IntroductionThis review article engages with Tao Jiang's ground-breaking monograph on the Origins of Moral-Political Philosophy in Early China with particular focus on the articulation of toleration and justice in the Laozi (otherwise called the Daodejing).1 Jiang discusses a naturalistic turn and the re-alignment of values in the Laozi, resulting in a (...) naturalization of justice (impartiality) by rejecting artificial humaneness and rigid hierarchical moral-political structures. In this Laozian just world, there is no room for human intervention and rigid top-down enforcement of values, thus leaving justice to naturalized Heaven or Dao as the ultimate source of the cosmos.Based on Jiang's interpretative context, I show that there is a twofold justification for the "paradoxical" and "elusive"2 value of political toleration.3 A negative expression of toleration focuses on non-interference with choices, and non-enforcement of values toward those whom one reasonably disagrees with and regards as morally wrong. Such an attitude of toleration results in practical advantages such as avoiding entrenchment in bloodshed, abandonment, and deprivation.A positive expression of toleration operating in the Laozian ideal world requires equal protection of people as capable knowers, including those to whom we object. Such protection includes their ways of expression and an unbiased recognition of them as equally capable knowers so that they are heard without bias and prejudice. Such a kind of toleration is revealed as a constituent component of justice since it enables equal contribution toward the shaping of society without discrimination. In other words, toleration is seen as naturalized justice in a Laozian world within which no one is "being wronged with the capacity as a knower"—a kind of epistemic justice articulated by Miranda Fricker (Fricker 2007).4 [End Page 466]This article discusses how Jiang's political reconstruction of Laozian philosophy contains the seeds for a discussion of toleration, a value associated with distributive justice and against epistemic injustice. First, it introduces Jiang's arguments on a naturalization of justice and impartiality as Heavenly attributes in the Laozi. Second, it articulates the value of tolerance in the Laozi. Finally, I compare Laozian toleration with Confucian toleration using youwei- wuwei metaethical criticism.Moral-Political Philosophy in the LaoziJiang's interpretative framework about the origins of distributive justice in the Laozi starts with a discussion of the cosmogonic-mystical worldviews that signal "a new understanding of the nature of the cosmos, as well as a broad reorientation in the Heaven-human relationship in the mid-Warring States period" (p. 192). Differing from the interests in the origins of human culture or civilization in the Ru and Mo traditions, Jiang agrees with Franklin Perkins and identifies a "cosmogonic turn" in the late fourth century b.c. that reveals a demotion of anthropomorphic Heaven. By paring Heaven with earth and accordingly decentering human beings in the intellectual discourse, the Laozi directs equal values to the ten thousand things, with humans as just one among them (Perkins 2016, quoted in Jiang 2021, p. 196).5 Laozi's philosophy thus stands out by rejecting Heaven as caring for human affairs while taking Dao as the ultimate origin of the cosmos, as claimed in chapter 5 of the received text: "Heaven and earth are not humane; they treat ten-thousand things in the world as straw dogs; the sage is not humane; he treats people as straw dogs" (p. 199).With Jiang's reading of a naturalistic heaven, we see a re-alignment between cosmos and the human since sage-rulers follow the Heaven-Dao. The Laozian Dao is "self-so-ing" (ziran 自然), "accommodating" (rong 容), and "impartial" (gong 公), and thus marks a sharp contrast to Confucian familial bias resulting in "actively intervening in human affairs on behalf of the (certain) humans by appointing them to carry out its mission" (p. 200). With the sage emulating the ultimate Dao, the Laozi replaces the Confucian anthropocentric Heaven, which focuses on partiality (qin 親) and humaneness (ren 仁).This Laozian understanding of the cosmos also grounds its philosophical departure from Mohism despite both having a political... (shrink)

Some researchers have recognized that privileged communities dominate the discourse on AI Ethics, and other voices need to be heard. As such, we identify the current ethics milieu as arising from WEIRD (Western, Educated, Industrialized, Rich, Democratic) contexts, and aim to expand the discussion to non-WEIRD global communities, who are also stakeholders in global sociotechnical systems. We argue that accounting for honor, along with its values and related concepts, would better approximate a global ethical perspective. This complex concept already underlies (...) some of the WEIRD discourse on AI ethics, but certain cultural forms of honor also bring overlooked issues and perspectives to light. We first describe honor according to recent empirical and philosophical scholarship. We then review “consensus” principles for AI ethics framed from an honor-based perspective, grounding comparisons and contrasts via example settings such as content moderation, job hiring, and genomics databases. A better appreciation of the marginalized concept of honor could, we hope, lead to more productive AI value alignment discussions, and to AI systems that better reflect the needs and values of users around the globe. (shrink)

As the capabilities of artificial intelligence systems improve, it becomes important to constrain their actions to ensure their behaviour remains beneficial to humanity. A variety of ethical, legal and safety-based frameworks have been proposed as a basis for designing these constraints. Despite their variations, these frameworks share the common characteristic that decision-making must consider multiple potentially conflicting factors. We demonstrate that these alignment frameworks can be represented as utility functions, but that the widely used Maximum Expected Utility paradigm provides (...) insufficient support for such multiobjective decision-making. We show that a Multiobjective Maximum Expected Utility paradigm based on the combination of vector utilities and non-linear action–selection can overcome many of the issues which limit MEU’s effectiveness in implementing aligned AI. We examine existing approaches to multiobjective AI, and identify how these can contribute to the development of human-aligned intelligent agents. (shrink)

How can one build AI systems such that they pursue the goals their designers want them to pursue? This is the alignment problem. Numerous authors have raised concerns that, as research advances and systems become more powerful over time, misalignment might lead to catastrophic outcomes, perhaps even to the extinction or permanent disempowerment of humanity. In this paper, I analyze the severity of this risk based on current instances of misalignment. More specifically, I argue that contemporary large language models (...) and game-playing agents are sometimes misaligned. These cases suggest that misalignment tends to have a variety of features: misalignment can be hard to detect, predict and remedy, it does not depend on a specific architecture or training paradigm, it tends to diminish a system’s usefulness and it is the default outcome of creating AI via machine learning. Subsequently, based on these features, I show that the risk of AI alignment magnifies with respect to more capable systems. Not only might more capable systems cause more harm when misaligned, aligning them should be expected to be more difficult than aligning current AI. (shrink)

This paper considers challenges resulting from the use of advanced artificial judicial intelligence (AAJI). We argue that these challenges should be considered through the lens of value alignment. Instead of discussing why specific goals and values, such as fairness and nondiscrimination, ought to be implemented, we consider the question of how AAJI can be aligned with goals and values more generally, in order to be reliably integrated into legal and judicial systems. This value alignment framing draws on AI (...) safety and alignment literature to introduce two otherwise neglected considerations for AAJI safety: specification and assurance. We outline diverse research directions and suggest the adoption of assurance and specification mechanisms as the use of AI in the judiciary progresses. While we focus on specification and assurance to illustrate the value of the AI safety and alignment literature, we encourage researchers in law and philosophy to consider what other lessons may be drawn. (shrink)

Healthcare systems are grappling with critical challenges, including chronic diseases in aging populations, unprecedented health care staffing shortages and turnover, scarce resources, unprecedented demands and wait times, escalating healthcare expenditure, and declining health outcomes. As a result, policymakers and healthcare executives are investing in artificial intelligence (AI) solutions to increase operational efficiency, lower health care costs, and improve patient care. However, current level of investment in developing healthcare AI among members of the global digital health partnership does not seem to (...) yield a high return yet. This is mainly due to underinvestment in the supporting infrastructure necessary to enable the successful implementation of AI. If a healthcare-specific AI winter is to be avoided, it is paramount that this disparity in the level of investment in the development of AI itself and in the development of the necessary supporting system components is evened out. (shrink)

Abstract: One approach to alignment with human values in AI and robotics is to engineer artiTicial systems isomorphic with human beings. The idea is that robots so designed may autonomously align with human values through similar developmental processes, to realize project ideal conditions through iterative interaction with social and object environments just as humans do, such as are expressed in narratives and life stories. One persistent problem with human value orientation is that different human beings champion different values as (...) ideal, meaning that the values to which an AI should be aligned are ambiguous. Prior work considered human development of purpose and source of meaning in life in terms of project ideal conditions, in effect establishing lifelong value orientations according to which intermediate situations are evaluated. The present work compares views on motivating values from St. Augustine and from popular cognitive science. These accounts are described as divergent error theories which present to their proponents as mutually exclusive yet accurate accounts of personal experience due to differential development of variable innate potentials. SpeciTically, the hypothesis proposed is that differential development of spindle neural projections establishes enduring connections between prior established relatively immediate routine processes entrained during childhood and prioritized in popular cognitive science, and later developing higher-level social and self-processes entrained during adolescence and emphasized in Augustine's account, with these projections hard-wiring lifelong motivating value orientations more or less inaccessible to modiTication through material interaction. Robot experiments informed by this study may evaluate variable value orientation by design, with for example autonomous robots developing motivating associations with temporally distal project ideal conditions through love for humanity, as described by Augustine, and others pursuing adaptive Tit to passing norms consistent with popular contemporary accounts. (shrink)

Human beings are internally inconsistent in various ways. One way to develop this thought involves using the language of value alignment: the values we hold are not always aligned with our behavior, and are not always aligned with each other. Because of this self-misalignment, there is room for potential projects of human enhancement that involve achieving a greater degree of value alignment than we presently have. Relatedly, discussions of AI ethics sometimes focus on what is known as the (...) value alignment problem, the challenge of how to build AI that acts in accordance with our human values. We argue that there is an especially close connection between solving the value alignment problem in AI ethics and using AI to pursue certain forms of human enhancement. But in addition, we also argue that there are important limits to what kinds of human enhancement can be pursued in this way, because some forms of human enhancement—namely moral revolutions—involve a kind of value misalignment rather than alignment. (shrink)

The paper demonstrates that social alignment is distinct from value alignment as it is currently understood in the AI safety literature, and argues that social alignment is an important research agenda. Work provides an important example for the argument, since work is a cooperative endeavor, and it is part of the larger manifold of social cooperation. These cooperative aspects of work are individually and socially valuable, and so they must be given a central place when evaluating the (...) impact of AI upon work. Workplace technologies are not simply instruments for achieving productive goals, but ways of mediating interpersonal relations. They are aspects of a cooperative interface i.e. the infrastructure by which we engage cooperative behavior with others. The concept of the cooperative interface suggests two conjectures to foreground in the social alignment agenda, motivated by the experience of algorithmic trading and social robotics: that AI impacts cooperation through its effects on social networks, and through its effects on social norms. (shrink)

The concept of alignment has undergone a remarkable rise in recent years to take center stage in the ethics of artificial intelligence. There are now numerous philosophical studies of the values that should be used in this ethical framework as well as a technical literature operationalizing these values in machine learning models. This article takes a step back to address a more basic set of critical questions: Where has the ethical imperative of alignment come from? What is the (...) ethical logic of alignment—how, exactly, does it propose to regulate machines’ and peoples’ conduct? And what are the social and political implications of this ethics? After discussing the logical and normative implications of the term itself—in what sense alignment can have an ethical meaning—we undertake a four-part “anatomy” of alignment in contemporary large language models (LLMs): first, a relatively technical sense in sequence modeling; second, a more normative sense relating to how outputs of pre-trained models are ethically evaluated; a third sense where external values are introduced using fine-tuning techniques to manage undesired model behaviors; and fourth sense, where alignment is given extreme ethical stakes in philosophical discussions of existential risks. We find that the ethics of alignment is fundamentally concerned with the problem of control, with unintended model behaviors that arise from divergences between training objectives and the normative expectations that govern the contexts in which they are used. Alignment serves to bridge the gap between what we call an “is” normativity, of statistical patterns identified by models and an “ought” normativity where values are technically introduced in models to steer them away from undesired behaviors. By problematizing control, the ethics of alignment weakens capacities to both make more substantive ethical judgments and also political decisions about how to live with AI. (shrink)

This paper categorizes current approaches to AI ethics into four perspectives and briefly summarizes them: (1) Case studies and technical trend surveys, (2) AI governance, (3) Technologies for AI alignment, (4) Philosophy. In the second half, we focus on the fourth perspective, the philosophical approach, within the context of applied ethics. In particular, the explicability of AI may be an area in which scientists, engineers, and AI developers are expected to engage more actively relative to other ethical issues in (...) AI. (shrink)

This paper explores the potential of integrating ancient educational principles from diverse eastern cultures into modern AI ethics curricula. It draws on the rich educational traditions of ancient China, India, Arabia, Persia, Japan, Tibet, Mongolia, and Korea, highlighting their emphasis on philosophy, ethics, holistic development, and critical thinking. By examining these historical educational systems, the paper establishes a correlation with modern AI ethics principles, advocating for the inclusion of these ancient teachings in current AI development and education. The proposed integration (...) aims to provide a comprehensive education that not only encompasses foundational knowledge but also advanced learning, thereby equipping future AI professionals with the necessary tools to develop AI systems that are ethically responsible, culturally aware, and aligned with human values such as fairness, safety, transparency, and collaboration. This approach not only addresses the AI alignment problem but also fosters cultural harmony and global understanding, which are crucial in an increasingly interconnected world. The paper posits that the wisdom of ancient educational systems, when harmonized with modern AI ethics, can guide the development of AI technologies that are beneficial for humanity, ensuring these advancements are not just technologically sound but also ethically and culturally informed. (shrink)

The “Value Alignment Problem” is the challenge of how to align the values of artificial intelligence with human values, whatever they may be, such that AI does not pose a risk to the existence of humans. Existing approaches appear to conceive of the problem as "how do we ensure that AI solves the problem in the right way", in order to avoid the possibility of AI turning humans into paperclips in order to “make more paperclips” or eradicating the human (...) race to “solve climate change”. This paper proposes an approach to Alignment rooted in the Enactive theory of mind that reconceptualises it as "how do we make relevant to AI what it relevant to humans". This conceptualisation is supported with a discussion of 4E cognition and goes on to suggest that the Alignment problem and the Frame problem are the same problem. The paper concludes with a discussion of the tradeoffs of the different approaches to value alignment. (shrink)

The value alignment problem for artificial intelligence (AI) asks how we can ensure that the “values”—i.e., objective functions—of artificial systems are aligned with the values of humanity. In this paper, I argue that linguistic communication is a necessary condition for robust value alignment. I discuss the consequences that the truth of this claim would have for research programmes that attempt to ensure value alignment for AI systems—or, more loftily, those programmes that seek to design robustly beneficial or (...) ethical artificial agents. (shrink)

AI research is being challenged with ensuring that autonomous agents learn to behave ethically, namely in alignment with moral values. Here, we propose a novel way of tackling the value alignment problem as a two-step process. The first step consists on formalising moral values and value aligned behaviour based on philosophical foundations. Our formalisation is compatible with the framework of (Multi-Objective) Reinforcement Learning, to ease the handling of an agent’s individual and ethical objectives. The second step consists in (...) designing an environment wherein an agent learns to behave ethically while pursuing its individual objective. We leverage on our theoretical results to introduce an algorithm that automates our two-step approach. In the cases where value-aligned behaviour is possible, our algorithm produces a learning environment for the agent wherein it will learn a value-aligned behaviour. (shrink)

The AGI alignment problem has a bimodal distribution of outcomes with most outcomes clustering around the poles of total success and existential, catastrophic failure. Consequently, attempts to solve AGI alignment should, all else equal, prefer false negatives (ignoring research programs that would have been successful) to false positives (pursuing research programs that will unexpectedly fail). Thus, we propose adopting a policy of responding to points of philosophical and practical uncertainty associated with the alignment problem by limiting and (...) choosing necessary assumptions to reduce the risk of false positives. Herein we explore in detail two relevant points of uncertainty that AGI alignment research hinges on---meta-ethical uncertainty and uncertainty about mental phenomena---and show how to reduce false positives in response to them. (shrink)

As artificial intelligence systems are becoming increasingly autonomous and will soon be able to make decisions on their own about what to do, AI researchers have started to talk about the need to align AI with human values. The AI ‘value alignment problem’ faces two kinds of challenges—a technical and a normative one—which are interrelated. The technical challenge deals with the question of how to encode human values in artificial intelligence. The normative challenge is associated with two questions: “Which (...) values or whose values should artificial intelligence align with?” My concern is that AI developers underestimate the difficulty of answering the normative question. They hope that we can easily identify the purposes we really desire and that they can focus on the design of those objectives. But how are we to decide which objectives or values to induce in AI, given that there is a plurality of values and moral principles and that our everyday life is full of moral disagreements? In my paper I will show that although it is not realistic to reach an agreement on what we, humans, really want as people value different things and seek different ends, it may be possible to agree on what we do not want to happen, considering the possibility that intelligence, equal to our own, or even exceeding it, can be created. I will argue for pluralism which is compatible with objectivism. In spite of the fact that there is no uniquely best solution to every moral problem, it is still possible to identify which answers are wrong. And this is where we should begin the value alignment of AI. (shrink)

Large language models (LLMs) produce sequences learned as statistical patterns from large corpora. Their emergent status as representatives of the advances in artificial intelligence (AI) have led to an increased attention to the possibilities of regulating the automated production of linguistic utterances and interactions with human users in a process that computer scientists refer to as ‘alignment’—a series of technological and political mechanisms to impose a normative model of morality on algorithms and networks behind the model. Alignment, which (...) can be viewed as the superimposition of normative structure onto a statistical model, however, reveals a conflicted and complex history of the conceptualisation of an interrelationship between language, mind and technology. This relationship is shaped by and, in turn, influences theories of language, linguistic practice and subjectivity, which are especially relevant to the current sophistication in artificially produced text. In this paper, we propose a critical evaluation of the concept of alignment, arguing that the theories and practice behind LLMs reveal a more complex social and technological dynamic of output coordination. We examine this dynamic as a two-way interaction between users and models by analysing how ChatGPT4 redacts perceived ‘anomalous’ language in fragments of Joyce’s Ulysses. We then situate this alignment problem historically, revisiting earlier postwar linguistic debates which counterposed two views of meaning: as discrete structures, and as continuous probability distributions. We discuss the largely occluded work of the Moscow Linguistic School, which sought to reconcile this opposition. Our attention to the Moscow School and later related arguments by Searle and Kristeva casts the problem of alignment in a new light: as one involving attention to the social regulation of linguistic practice, including rectification of anomalies that, like the Joycean text, exist in defiance of expressive conventions. The “problem of alignment” that we address here is, therefore, twofold: on one hand, it points to its narrow and normative definition in current technological development and critical research and, on the other hand, to the reality of complex and contradictory relations between subjectivity, technology and language that alignment problems reveal. (shrink)

AI research is being challenged with ensuring that autonomous agents learn to behave ethically, namely in alignment with moral values. Here, we propose a novel way of tackling the value alignment problem as a two-step process. The first step consists on formalising moral values and value aligned behaviour based on philosophical foundations. Our formalisation is compatible with the framework of (Multi-Objective) Reinforcement Learning, to ease the handling of an agent’s individual and ethical objectives. The second step consists in (...) designing an environment wherein an agent learns to behave ethically while pursuing its individual objective. We leverage on our theoretical results to introduce an algorithm that automates our two-step approach. In the cases where value-aligned behaviour is possible, our algorithm produces a learning environment for the agent wherein it will learn a value-aligned behaviour. (shrink)

Since the release of ChatGPT, there has been a lot of debate about whether AI systems pose an existential risk to humanity. This paper develops a general framework for thinking about the existential risk of AI systems. We analyze a two-premise argument that AI systems pose a threat to humanity. Premise one: AI systems will become extremely powerful. Premise two: if AI systems become extremely powerful, they will destroy humanity. We use these two premises to construct a taxonomy of ‘survival (...) stories’, in which humanity survives into the far future. In each survival story, one of the two premises fails. Either scientific barriers prevent AI systems from becoming extremely powerful; or humanity bans research into AI systems, thereby preventing them from becoming extremely powerful; or extremely powerful AI systems do not destroy humanity, because their goals prevent them from doing so; or extremely powerful AI systems do not destroy humanity, because we can reliably detect and disable systems that have the goal of doing so. We argue that different survival stories face different challenges. We also argue that different survival stories motivate different responses to the threats from AI. Finally, we use our taxonomy to produce rough estimates of ‘P(doom)’, the probability that humanity will be destroyed by AI. (shrink)

Debates about the development of artificial superintelligence and its potential threats to humanity tend to assume that such a system would be historically unprecedented, and that its behavior must be predicted from first principles. I argue that this is not true: we can analyze multiagent intelligent systems (the best candidates for practical superintelligence) by comparing them to states, which also unite heterogeneous intelligences to achieve superhuman goals. States provide a model for several problems discussed in the literature on superintelligence, such (...) as principal-agent problems and Instrumental Convergence. Philosophical arguments about governance, therefore, provide possible solutions to these problems, or point out problems in previously suggested solutions. In particular, the liberal concept of checks and balances, and Hannah Arendt’s concept of legitimacy, describe how state behavior is constrained by the preferences of constituents that could also apply to artificial systems. However, they also point out ways in which present-day computational developments could destabilize the international order by reducing the number of decision-makers involved in state actions. Thus, interstate competition not only serves as a model for the behavior of dangerous computational intelligences but also as the impetus for their development. (shrink)

Upon previous research, the matrilineal system seems to oppose Islamic teaching. However, the matrilineal system practiced by the Minangkabau society in West Sumatra, Indonesia has its uniqueness. Thus, this study aims to examine the correlation between the Qur’an and gender roles within the context of Minangkabau customs, specifically focusing on the matrilineal aspect. The present study employs qualitative methods for conducting library research through critical analysis. This study discovered that the matrilineal system practiced by the Minangkabau society aligns with Qur’anic (...) teachings that respect the position of women in Islam. In the matrilineal system of Minangkabau, women hold significant strength and play an influential role in cultural and traditional practices. The Minangkabau tribe’s matrilineal system encompasses three key elements: marriage, inheritance and offspring. The marriage system is exogamous. Additionally, the practice of marriage is localised, with the husband typically residing in the wife’s household. Their inheritance is divided into pusako tinggi [from ancestors from generation to generation] and pusako randah (from the efforts of married couple). Women get pusako tinggi as they serve as protectors and a symbol of their community. In comparison, men take the responsibility of overseeing its preservation to prevent any form of misuse. Pusako randah is distributed as Islamic teaching. Lineage determination is primarily based on paternal lineage, aligning with Islamic teaching. Upon comprehensive analysis, it becomes evident that there is no conflict between the matrilineal system and Islamic teachings, particularly concerning principles of gender equality. Both women and men have significant roles, as the Qur’an provides extensive explanations regarding gender equality.Contribution: This research holds significant findings in establishing a great relation between Islamic teachings based on the Qur’an and the matrilineal principles practiced by the Minangkabau tribe. (shrink)