The human brain has some capabilities that the brains of other animals lack. It is to these distinctive capabilities that our species owes its dominant position. Other animals have stronger muscles or sharper claws, but we have cleverer brains. If machine brains one day come to surpass human brains in general intelligence, then this new superintelligence could become very powerful. As the fate of the gorillas now depends more on us humans than on the gorillas themselves, so the fate of (...) our species then would come to depend on the actions of the machine superintelligence. But we have one advantage: we get to make the first move. Will it be possible to construct a seed AI or otherwise to engineer initial conditions so as to make an intelligence explosion survivable? How could one achieve a controlled detonation? To get closer to an answer to this question, we must make our way through a fascinating landscape of topics and considerations. Read the book and learn about oracles, genies, singletons; about boxing methods, tripwires, and mind crime; about humanity's cosmic endowment and differential technological development; indirect normativity, instrumental convergence, whole brain emulation and technology couplings; Malthusian economics and dystopian evolution; artificial intelligence, and biological cognitive enhancement, and collective intelligence. (shrink)

I argue that at least one of the following propositions is true: the human species is very likely to become extinct before reaching a ’posthuman’ stage; any posthuman civilization is extremely unlikely to run a significant number of simulations of its evolutionary history ; we are almost certainly living in a computer simulation. It follows that the belief that there is a significant chance that we shall one day become posthumans who run ancestor-simulations is false, unless we are currently living (...) in a simulation. I discuss some consequences of this result. (shrink)

The human desire to acquire new capacities is as ancient as our species itself. We have always sought to expand the boundaries of our existence, be it socially, geographically, or mentally. There is a tendency in at least some individuals always to search for a way around every obstacle and limitation to human life and happiness.

Because of accelerating technological progress, humankind may be rapidly approaching a critical phase in its career. In addition to well-known threats such as nuclear holocaust, the propects of radically transforming technologies like nanotech systems and machine intelligence present us with unprecedented opportunities and risks. Our future, and whether we will have a future at all, may well be determined by how we deal with these challenges. In the case of radically transforming technologies, a better understanding of the transition dynamics from (...) a human to a "posthuman" society is needed. Of particular importance is to know where the pitfalls are: the ways in which things could go terminally wrong. While we have had long exposure to various personal, local, and endurable global hazards, this paper analyzes a recently emerging category: that of existential risks. These are threats that could case our extinction or destroy the potential of Earth - originating intelligent life. Some of these threats are relatively well known while others, including some of the gravest, have gone almost unrecognized. Existential risks have a cluster of features that make ordinary risk management ineffective. A final section of this paper discusses several ethical and policy implications. A clearer understanding of the threat picture will enable us to formulate better strategies. (shrink)

Non-representational approaches to cognition have struggled to provide accounts of long-term planning that forgo the use of representations. An explanation comes easier for cognitivist accounts, which hold that we concoct and use contentful mental representations as guides to coordinate a series of actions towards an end state. One non-representational approach, ecological-enactivism, has recently seen several proposals that account for “high-level” or “representation-hungry” capacities, including long-term planning and action coordination. In this paper, we demonstrate the explanatory gap in these accounts that (...) stems from avoiding the incorporation of long-term intentions, as they play an important role both in action coordination and perception on the ecological account. Using recent enactive accounts of language, we argue for a non-representational conception of intentions, their formation, and their role in coordinating pre-reflective action. We provide an account for the coordination of our present actions towards a distant goal, a skill we call distal engagement. Rather than positing intentions as an actual cognitive entity in need of explanation, we argue that we take them up in this way as a practice due to linguistically scaffolded attitudes towards language use. (shrink)

Remarkable progress in the mathematics and computer science of probability has led to a revolution in the scope of probabilistic models. In particular, ‘sophisticated’ probabilistic methods apply to structured relational systems such as graphs and grammars, of immediate relevance to the cognitive sciences. This Special Issue outlines progress in this rapidly developing field, which provides a potentially unifying perspective across a wide range of domains and levels of explanation. Here, we introduce the historical and conceptual foundations of the approach, explore (...) how the approach relates to studies of explicit probabilistic reasoning, and give a brief overview of the field as it stands today. (shrink)

An influential line of thinking in behavioral science, to which the two authors have long subscribed, is that many of society's most pressing problems can be addressed cheaply and effectively at the level of the individual, without modifying the system in which the individual operates. We now believe this was a mistake, along with, we suspect, many colleagues in both the academic and policy communities. Results from such interventions have been disappointingly modest. But more importantly, they have guided many (though (...) by no means all) behavioral scientists to frame policy problems in individual, not systemic, terms: To adopt what we call the “i-frame,” rather than the “s-frame.” The difference may be more consequential than i-frame advocates have realized, by deflecting attention and support away from s-frame policies. Indeed, highlighting the i-frame is a long-established objective of corporate opponents of concerted systemic action such as regulation and taxation. We illustrate our argument briefly for six policy problems, and in depth with the examples of climate change, obesity, retirement savings, and pollution from plastic waste. We argue that the most important way in which behavioral scientists can contribute to public policy is by employing their skills to develop and implement value-creating system-level change. (shrink)

Scientific and technological progress might change people’s capabilities or incentives in ways that would destabilize civilization. For example, advances in DIY biohacking tools might make it easy for anybody with basic training in biology to kill millions; novel military technologies could trigger arms races in which whoever strikes first has a decisive advantage; or some economically advantageous process may be invented that produces disastrous negative global externalities that are hard to regulate. This paper introduces the concept of a vulnerable world: (...) roughly, one in which there is some level of technological development at which civilization almost certainly gets devastated by default, i.e. unless it has exited the “semi-anarchic default condition”. Several counterfactual historical and speculative future vulnerabilities are analyzed and arranged into a typology. A general ability to stabilize a vulnerable world would require greatly amplified capacities for preventive policing and global governance. The vulnerable world hypothesis thus offers a new perspective from which to evaluate the risk-benefit balance of developments towards ubiquitous surveillance or a unipolar world order. (shrink)

Human enhancement has emerged in recent years as a blossoming topic in applied ethics. With continuing advances in science and technology, people are beginning to realize that some of the basic parameters of the human condition might be changed in the future. One important way in which the human condition could be changed is through the enhancement of basic human capacities. If this becomes feasible within the lifespan of many people alive today, then it is important now to consider the (...) normative questions raised by such prospects. The answers to these questions might not only help us be better prepared when technology catches up with imagination, but they may be relevant to many decisions we make today, such as decisions about how much funding to give to various kinds of research. Enhancement is typically contraposed to therapy. In broad terms, therapy aims to fix something that has gone wrong, by curing specific diseases or injuries, while enhancement interventions aim to improve the state of an organism beyond its normal healthy state. However, the distinction between therapy and enhancement is problematic, for several reasons. First, we may note that the therapy-enhancement dichotomy does not map onto any corresponding dichotomy between standard-contemporary-medicine and medicineas-it-could-be-practised-in-the-future. Standard contemporary medicine includes many practices that do not aim to cure diseases or injuries. It includes, for example, preventive medicine, palliative care, obstetrics, sports medicine, plastic surgery, contraceptive devices, fertility treatments, cosmetic dental procedures, and much else. At the same time, many enhancement interventions occur outside of the medical framework. Office workers enhance their performance by drinking coffee. Make-up and grooming are used to enhance appearance. Exercise, meditation, fish oil, and St John’s Wort are used to enhance mood. Second, it is unclear how to classify interventions that reduce the probability of disease and death.. (shrink)

I consider the metaphysical consequences of the view that propositional attitudes have essential normative properties. I argue that realism should take a weak rather than a strong form. I argue that expressivism cannot get off the ground. And I argue that eliminativism is self-refuting.

We present a new notion of game equivalence that captures basic powers of interacting players. We provide a representation theorem, a complete logic, and a new game algebra for basic powers. In doing so, we establish connections with imperfect information games and epistemic logic. We also identify some new open problems concerning logic and games.

In some dark alley. . . Mugger: Hey, give me your wallet. Pascal: Why on Earth would I want to do that? Mugger: Otherwise I’ll shoot you. Pascal: But you don’t have a gun. Mugger: Oops! I knew I had forgotten something. Pascal: No wallet for you then. Have a nice evening. Mugger: Wait! Pascal: Sigh. Mugger: I’ve got a business proposition for you. . . . How about you give me your wallet now? In return, I promise to come (...) to your house tomorrow and give you double the value of what’s in the wallet. Not bad, eh? A 200% return on investment in 24 hours. (shrink)

It has long been thought that observing distinctive traces of quantum gravity in a laboratory setting is effectively impossible, since gravity is so much weaker than all the other familiar forces in particle physics. But the quantum gravity phenomenology community today seeks to do the (effectively) impossible, using a challenging novel class of `tabletop' Gravitationally Induced Entanglement (GIE) experiments, surveyed here. The hypothesized outcomes of the GIE experiments are claimed by some (but disputed by others) to provide a `witness' of (...) the underlying quantum nature of gravity in the non-relativistic limit, using superpositions of Planck-mass bodies. We inspect what sort of achievement it would possibly be to perform GIE experiments, as proposed, ultimately arguing that the positive claim of witness is equivocal. Despite various sweeping arguments to the contrary in the vicinity of quantum information theory or given low-energy quantum gravity, whether or not one can claim to witness the quantum nature of the gravitational field in these experiments decisively depends on which out of two legitimate modelling paradigms one finds oneself in. However, by situating GIE experiments in a tradition of existing experiments aimed at making gravity interestingly quantum in the laboratory, we argue that, independently of witnessing or paradigms, there are powerful reasons to perform the experiments, and that their successful undertaking would indeed be a major advance in physics. (shrink)

The essentialist approach to word meaning has been used to undermine the fundamental assumptions of the cognitive psychology of concepts. Essentialism assumes that a word refers to a natural kind category in virtue of category members possessing essential properties. In support of this thesis, Kripke and Putnam deploy various intuitions concerning word use under circumstances in which discoveries about natural kinds are made. Although some studies employing counterfactual discoveries and related transformations appear to vindicate essentialism, we argue that the intuitions (...) have not been investigated exhaustively. In particular, we argue that discoveries concerning the essential properties of whole categories (rather than simply of particular category members) are critical to the essentialist intuitions. The studies reported here examine such discovery contexts, and demonstrate that words and concepts are not used in accordance with essentialism. The results are, however, consistent with "representational change" views of concepts, which are broadly Fregean in their motivation. We conclude that since essentialism is not vindicated by ordinary word use, it fails to undermine the cognitive psychology of concepts. (shrink)

Stable logics are modal logics characterized by a class of frames closed under relation preserving images. These logics admit all filtrations. Since many basic modal systems such as K4 and S4 are not stable, we introduce the more general concept of an M-stable logic, where M is an arbitrary normal modal logic that admits some filtration. Of course, M can be chosen to be K4 or S4. We give several characterizations of M-stable logics. We prove that there are continuum many (...) S4-stable logics and continuum many K4-stable logics between K4 and S4. We axiomatize K4-stable and S4-stable logics by means of stable formulas and discuss the connection between S4-stable logics and stable superintuitionistic logics. We conclude the article with many examples (and nonexamples) of stable, K4-stable, and S4-stable logics and provide their axiomatization in terms of stable rules and formulas. (shrink)

Recent research suggests that language evolution is a process of cultural change, in which linguistic structures are shaped through repeated cycles of learning and use by domain-general mechanisms. This paper draws out the implications of this viewpoint for understanding the problem of language acquisition, which is cast in a new, and much more tractable, form. In essence, the child faces a problem of induction, where the objective is to coordinate with others (C-induction), rather than to model the structure of the (...) natural world (N-induction). We argue that, of the two, C-induction is dramatically easier. More broadly, we argue that understanding the acquisition of any cultural form, whether linguistic or otherwise, during development, requires considering the corresponding question of how that cultural form arose through processes of cultural evolution. This perspective helps resolve the “logical” problem of language acquisition and has far-reaching implications for evolutionary psychology. (shrink)

We address the problem of predicting how people will spontaneously divide into groups a set of novel items. This is a process akin to perceptual organization. We therefore employ the simplicity principle from perceptual organization to propose a simplicity model of unconstrained spontaneous grouping. The simplicity model predicts that people would prefer the categories for a set of novel items that provide the simplest encoding of these items. Classification predictions are derived from the model without information either about the number (...) of categories sought or information about the distributional properties of the objects to be classified. These features of the simplicity model distinguish it from other models in unsupervised categorization (where, for example, the number of categories sought is determined via a free parameter), and we discuss how these computational differences are related to differences in modeling objectives. The predictions of the simplicity model are validated in four experiments. We also discuss the significance of simplicity in cognitive modeling more generally. (shrink)

Evolutionary development is sometimes thought of as exhibiting an inexorable trend towards higher, more complex, and normatively worthwhile forms of life. This paper explores some dystopian scenarios where freewheeling evolutionary developments, while continuing to produce complex and intelligent forms of organization, lead to the gradual elimination of all forms of being that we care about. We then consider how such catastrophic outcomes could be avoided and argue that under certain conditions the only possible remedy would be a globally coordinated policy (...) to control human evolution by modifying the fitness function of future intelligent life forms. (shrink)

Theodicy, the enterprise of searching for greater goods that might plausibly justify God’s permission of evil, is often criticized on the grounds that the project has systematically failed to unearth any such goods. But theodicists also face a deeper challenge, one that places under question the very attempt to look for any morally sufficient reasons God might have for creating a world littered with evil. This ‘anti-theodical’ view argues that theists (and non-theists) ought to reject, primarily for moral reasons, the (...) project of ‘justifying the ways of God to men’. Unfortunately, this view has not received the serious attention it deserves, particularly in analytic philosophy of religion. Taking my cues from such anti-theodicists as Kenneth Surin, D.Z. Phillips and Dostoyevsky’s Ivan Karamazov, I defend several reasons for holding that the way of thinking about God and evil enshrined in theodical discourse can only add to the world’s evils, not remove or illuminate them. (shrink)

Cognitive enhancements in the context of converging technologies. [Annals of the New York Academy of Sciences, Vol. 1093, pp. 201-207] [with Anders Sandberg] [pdf].

The study of active matter systems demonstrates how interactions might co-constitute agential dynamics. Active matter systems are comprised of self-propelled independent entities which, en masse, take part in complex and interesting collective group behaviors at a far-from-equilibrium state (Menon, 2010 ; Takatori & Brady, 2015 ). These systems are modelled using very simple rules (Vicsek at al. 1995), which reveal the interactive nature of the collective behaviors seen from humble to highly complex entities. Here I show how the study of (...) active matter systems supports two related proposals regarding interaction and agency. First, I argue that the study of interactive dynamics in these systems evidences the utility of treating interaction as an ontological category (Longino, 2021 ) and challenges methodological individualism as the received explanatory primitive in the study of agency. Second, the methods used to research active matter systems demonstrate how a minimal approach to agency can scale up in studying interactive agential dynamics in more complex systems. The examples of coordination dynamics (Kelso, 2001 ) and participatory sense-making (De Jaegher & Di Paolo, 2007 ) are provided to show how understanding agency requires us to look beyond the individuals to the interactive agential dynamics that can guide, scaffold, or constrain their activity. (shrink)

The standard topological representation of a Boolean algebra via the clopen sets of a Stone space requires a nonconstructive choice principle, equivalent to the Boolean Prime Ideal Theorem. In this article, we describe a choice-free topological representation of Boolean algebras. This representation uses a subclass of the spectral spaces that Stone used in his representation of distributive lattices via compact open sets. It also takes advantage of Tarski’s observation that the regular open sets of any topological space form a Boolean (...) algebra. We prove without choice principles that any Boolean algebra arises from a special spectral space X via the compact regular open sets of X; these sets may also be described as those that are both compact open in X and regular open in the upset topology of the specialization order of X, allowing one to apply to an arbitrary Boolean algebra simple reasoning about regular opens of a separative poset. Our representation is therefore a mix of Stone and Tarski, with the two connected by Vietoris: the relevant spectral spaces also arise as the hyperspace of nonempty closed sets of a Stone space endowed with the upper Vietoris topology. This connection makes clear the relation between our point-set topological approach to choice-free Stone duality, which may be called the hyperspace approach, and a point-free approach to choice-free Stone duality using Stone locales. Unlike Stone’s representation of Boolean algebras via Stone spaces, our choice-free topological representation of Boolean algebras does not show that every Boolean algebra can be represented as a field of sets; but like Stone’s representation, it provides the benefit of a topological perspective on Boolean algebras, only now without choice. In addition to representation, we establish a choice-free dual equivalence between the category of Boolean algebras with Boolean homomorphisms and a subcategory of the category of spectral spaces with spectral maps. We show how this duality can be used to prove some basic facts about Boolean algebras. (shrink)

Beauty is an important part of our lives. Ugliness too. It is no surprise then that philosophers since antiquity have been interested in our experiences of and judgments about beauty and ugliness. They have tried to understand the nature of these experiences and judgments, and they have also wanted to know whether these experiences and judgments were legitimate. Both these projects took a sharpened form in the twentieth century, when this part of our lives came under a sustained attack in (...) both European and American intellectual circles. Much of the discourse about beauty since the Eighteenth century had deployed a notion of the ‘aesthetic’, and so that notion in particular came in for criticism. This disdain for the aesthetic may have roots in a broader cultural Puritanism, which fears the connection between the aesthetic and pleasure. Even to suggest, in the recent climate, that an artwork might be good because it is pleasurable, as opposed to cognitively, morally or politically beneficial, is to court derision. The twentieth century has not been kind to the notions of beauty or the aesthetic. Nevertheless, some thinkers — philosophers, as well as others in the study of particular arts — have persisted in thinking about beauty and the aesthetic in a traditional way. In the first part of this essay, I look at the particular rich account of judgments of beauty given to us by Immanuel Kant. The notion of a ‘judgment of taste’ is central to Kant's account and also to virtually everyone working in traditional aesthetics; so I begin by examining Kant's characterisation of the judgment of taste. In the second part, I look at the issues that twentieth century thinkers have raised. I end by drawing on Kant's accout of the judgement of taste to consider whether the notion of the aesthetic is viable. (shrink)

Current cosmological theories say that the world is so big that all possible observations are in fact made. But then, how can such theories be tested? What could count as negative evidence? To answer that, we need to consider observation selection effects.

This paper develops and defends three related forms of relationism about spacetime against attacks by contemporary substantivalists. It clarifies Newton's globes argument to show that it does not bear on relations that fail to determine geodesic motions, since the inertial effects on which Newton relies are not simply correlated with affine structure, but must be understood in dynamical terms. It develops remarks by Sklar and van Fraassen into relational versions of Newtonian mechanics, and argues that Earman does not show them (...) to trivialize the debate. (shrink)

In some situations a number of agents each have the ability to undertake an initiative that would have significant effects on the others. Suppose that each of these agents is purely motivated by an altruistic concern for the common good. We show that if each agent acts on her own personal judgment as to whether the initiative should be undertaken, then the initiative will be undertaken more often than is optimal. We suggest that this phenomenon, which we call the unilateralist’s (...) curse, arises in many contexts, including some that are important for public policy. To lift the curse, we propose a principle of conformity, which would discourage unilateralist action. We consider three different models for how this principle could be implemented, and respond to an objection that could be raised against it. (shrink)

Anthony Brueckner, in a recent article, proffers ‘a new way of thinking about Bostrom's Simulation Argument’ . His comments, however, misconstrue the argument; and some words of explanation are in order.The Simulation Argument purports to show, given some plausible assumptions, that at least one of three propositions is true . Roughly stated, these propositions are: almost all civilizations at our current level of development go extinct before reaching technological maturity; there is a strong convergence among technologically mature civilizations such that (...) almost all of them lose interest in creating ancestor-simulations; almost all people with our sorts of experiences live in computer simulations. I also argue that conditional on you should assign a very high credence to the proposition that you live in a computer simulation. However, pace Brueckner, I do not argue that we should believe that we are in simulation. 1 In fact, I believe that we are probably not simulated. The Simulation Argument purports to show only that, as well as , at least one of – is true; but it does not tell us which one.Brueckner also writes: " It is worth noting that one reason why Bostrom thinks that the number of Sims [computer-generated minds with experiences similar to those typical of normal, embodied humans living in a Sim-free early 21 st century world] will vastly outstrip the number of humans is that Sims ‘will run their … ". (shrink)

Evidence based-policy (EBP) is a popular research paradigm in the applied social sciences and within government agencies. Informally, EBP represents an explicit commitment to applying scientific methods to public affairs, in contrast to ideologically-driven or merely intuitive “common-sense” approaches to public policy. More specifically, the EBP paradigm places great weight on the results of experimental research designs, especially randomised controlled trials (RCTs), and systematic literature reviews that place evidential weight on experimental results. One hope is that such research designs and (...) approaches to analysing the scientific literature are sufficiently robust that they can settle what really ‘works’ in public policy. Can EBP succeed in displacing reliance on domain-specific expertise? On our account, this is seldom, if ever, the case. The key reason for this is that underlying this approach is generally an appeal to argument by induction, which always requires further assumptions to underwrite its validity, and if not induction, some other argument form that also requires assumptions that are very often not validated for the case at hand. (shrink)

In this paper we define the notion of frame based formulas. We show that the well-known examples of formulas arising from a finite frame, such as the Jankov-de Jongh formulas, subframe formulas and cofinal subframe formulas, are all particular cases of the frame based formulas. We give a criterion for an intermediate logic to be axiomatizable by frame based formulas and use this criterion to obtain a simple proof that every locally tabular intermediate logic is axiomatizable by Jankov-de Jongh formulas. (...) We also show that not every intermediate logic is axiomatizable by frame based formulas. (shrink)

The hope that art could be personally or socially transformational is an important part of art history and contemporary art practice. In the twentieth century, it shaped a movement away from traditional media in an effort to make social life a medium. Artists imagined and created participatory situations designed to facilitate potentially transformative expression in those who engaged with the works. This chapter develops the concept of “transformative expression,” and illustrates how it informs a diverse range of such works. Understanding (...) these artworks in this way raises two interesting questions, one about the nature of aesthetic value and the other about the nature of action. Answers to these questions lie in understanding the social and aesthetic character of our capacity to distance ourselves from our commitments and act in the expressive, playful, spontaneous, or imaginative ways that participatory art invites. (shrink)

I argue that people do not and cannot have religious experiences that are perceptual experiences with theological content and that provide some justification for the belief in God. I discuss William Alston's resourceful defence of this idea. My strategy is to say that religious perception would either have to be by means of one of the ordinary five senses or else by means of some special sixth religious sense. In either case insoluble epistemological problems arise. The problem is with perceiving (...) God as God, which we need to do if reasons to believe in God are to be generated. To do so, we would have to perceive the instantiation of His essential properties – being all-knowing, all-powerful, and all-good. But perceiving the instantiation of these properties of God, even by some special sixth religious sense, is impossible. Hence, God cannot be perceived either by the ordinary five senses or by a sixth religious sense. Religious perceptual experiences are a myth. (Published Online February 17 2004). (shrink)

I shall discuss several related issues about beauty. These are: (1) The place of beauty among other aesthetic properties. (2) The general principle of aesthetic supervenience. (3) The problem of aesthetic relevance. (4) The distinction between free and dependent beauty. (5) The primacy of our appreciation of free beauty over our appreciation of dependent beauty. (6) Personal beauty as a species of beauty. (7) The metaphysics of beauty.

§1.1 What m otivated the perpetrators of the holocaust? Christopher Browning and Daniel Goldhagen differ in their analysis of Reserve Police Battalion 101 (Browning 1992, Goldhagen 1996). The battalion consisted of around 500 ‘ordinary’ Germ ans who, during the period 1942-44, killed around 40,000 Jews and who deported as m any to the death cam ps. Browning and Goldhagen differ over the m otivation wit h which the m en killed. I want to com m ent on a central aspect (...) of this debate. (shrink)

This article proposes that a general theory of assessment of historical testimony should do justice to the long tradition of adjudication in accordance with maxims of reliability and competence. I argue that an explanatory genealogical theory (along lines first adumbrated by Charles Seignobos) satisfies this condition, and that it has further notable virtues: respect for the strengths of rival theories, regard for the links between adjudication of testimony and other basic procedures of historical inquiry, and the promise of profitable lines (...) of investigation. (shrink)

When water was discovered on Mars, people got very excited. Where there is water, there may be life. Scientists are planning new missions to study the planet up close. NASA’s next Mars rover is scheduled to arrive in 2010. In the decade following, a Mars Sample Return mission might be launched, which would use robotic systems to collect samples of Martian rocks, soils, and atmosphere, and return them to Earth. We could then analyze the sample to see if it contains (...) any traces of life, whether extinct or still active. Such a discovery would be of tremendous scientific significance. What could be more fascinating than discovering life that had evolved entirely independently of life here on Earth? Many people would also find it heartening to learn that we are not entirely alone in this vast cold cosmos. (shrink)