We are living in an algorithmic age where mathematics and computer science are coming together in powerful new ways to influence, shape and guide our behaviour and the governance of our societies. As these algorithmic governance structures proliferate, it is vital that we ensure their effectiveness and legitimacy. That is, we need to ensure that they are an effective means for achieving a legitimate policy goal that are also procedurally fair, open and unbiased. But how can (...) we ensure that algorithmic governance structures are both? This article shares the results of a collective intelligence workshop that addressed exactly this question. The workshop brought together a multidisciplinary group of scholars to consider barriers to legitimate and effective algorithmic governance and the research methods needed to address the nature and impact of specific barriers. An interactive management workshop technique was used to harness the collective intelligence of this multidisciplinary group. This method enabled participants to produce a framework and research agenda for those who are concerned about algorithmic governance. We outline this research agenda below, providing a detailed map of key research themes, questions and methods that our workshop felt ought to be pursued. This builds upon existing work on research agendas for critical algorithm studies in a unique way through the method of collective intelligence. (shrink)

Algorithms, or rather algorithmic actions, are seen as problematic because they are inscrutable, automatic, and subsumed in the flow of daily practices. Yet, they are also seen to be playing an important role in organizing opportunities, enacting certain categories, and doing what David Lyon calls “social sorting.” Thus, there is a general concern that this increasingly prevalent mode of ordering and organizing should be governed more explicitly. Some have argued for more transparency and openness, others have argued for more (...) democratic or value-centered design of such actors. In this article, we argue that governing practices—of, and through algorithmic actors—are best understood in terms of what Foucault calls governmentality. Governmentality allows us to consider the performative nature of these governing practices. They allow us to show how practice becomes problematized, how calculative practices are enacted as technologies of governance, how such calculative practices produce domains of knowledge and expertise, and finally, how such domains of knowledge become internalized in order to enact self-governing subjects. In other words, it allows us to show the mutually constitutive nature of problems, domains of knowledge, and subjectivities enacted through governing practices. In order to demonstrate this, we present attempts to govern academic writing with a specific focus on the algorithmic action of Turnitin. (shrink)

The objective of this study is to examine the effects of generative artificial intelligence (AI) tools, with a specific focus on ChatGPT, on the analytical proficiencies of policy analysts operating in Indonesia. Considering the increasing intricacies of contemporary governance and the emergence of "wicked problems," this study investigates the potential of AI to facilitate the development of inventive, data-centric public policies. Involving postgraduate students in a quasi-experimental design, this study investigated the efficacy of ChatGPT in assisting in the development (...) of policy solutions pertaining to air pollution in Jakarta. Specialized prompt engineering techniques are utilized in this study to enhance the utility of ChatGPT for policy analysis. The results suggest that, although generative AI tools such as ChatGPT provide systematic frameworks for tackling complex policy issues and may enhance analytical skills, they still require additional refinement to increase their empirical support and practical applicability in the Indonesian policy context. In accordance with incrementalism theory, the study concludes that incorporating AI into policy analysis can facilitate the gradual formulation of evidence-based policies, offering a pragmatic approach to the resolution of complex problems in the public sector. (shrink)

Many actors mobilize the cognitive, legal and technical tool-box of data protection when they discuss and address controversial issues such as digital mass surveillance. Yet, critical approaches to the digital only barely explore the politics of data protection in relation to data-driven governance. Building on governmentality studies and Actor-Network-Theory, this article analyses the potential and limits of using data protection to critique the ‘digital age’. Using the conceptual tool of dispositifs, it sketches an analytics of data protection and the (...) emergence of its configuration as ‘data protection by design and by default’. This exploration reminds us that governing through data implies, first and foremost, governing digital data. (shrink)

One of the most common contemporary approaches for developing an ethics of artificial intelligence (AI) involves elaborating guiding principles. This essay explores the limitations of this approach, using the history of bioethics as a comparative case. The examples of bioethics and recent AI ethics suggest that principles are difficult to implement in everyday practice, fail to direct individual action, and can frequently result in a pure proceduralism. The essay encourages an additional attention to virtue, which forms the dispositions of actors, (...) directs them to valued ends, and encourages right action in particular situation to complement principles. Practices and professions like medicine or warfighting that already have rich traditions of reflection on virtue ethics will be especially fruitful fields for developing an AI ethics of virtue. (shrink)

This essay introduces this special issue on virtue ethics in relation to military AI. It describes the current situation of military AI ethics as following that of AI ethics in general, caught between consequentialism and deontology. Virtue ethics serves as an alternative that can address some of the weaknesses of these dominant forms of ethics. The essay describes how the articles in the issue exemplify the value of virtue-related approaches for these questions, before ending with thoughts for further research.

Algorithm is the core category and basic methods of the digital age, and advanced technologies such as big data, artificial intelligence, and blockchain all need to rely on various algorithm designs or take the algorithm as the underlying principle. However, due to the characteristics of algorithm design, application, and technology itself, there are also hidden worries such as algorithm black-box, algorithm discrimination, and difficulty in accountability in the operation process to varying degrees. This paper summarizes these problems into three aspects: (...) unexplainable, self-reinforcing and autonomous. Facing the opportunities and risks generated by the application of the algorithm in national governance, while actively promoting the development of algorithm technology to continuously promote the modernization process of national governance, it is also necessary to increase the governance of the algorithm. The practice has proved that enhancing the interpretability of algorithm, optimizing algorithm design, and adopting legal regulatory algorithm are the basic approaches to effective regulatory algorithm in the era of intelligent governance. (shrink)

Everyday surveillance work is increasingly performed by non-human algorithms. These entities can be conceptualised as machinic flâneurs that engage in distanciated flânerie: subjecting urban flows to a dispassionate, calculative and expansive gaze. This paper provides some theoretical reflections on the nascent forms of algorithmic practice materialising in two Australian cities, and some of their implications for urban relations and social justice. It looks at the idealisation – and operational black boxing – of automated watching programs, before considering their impacts (...) on notions such as ‘the right to the city’ and ‘the right to the face’. It will argue that the turn to facial recognition software for the purposes of automating urban governance reconstitutes the meanings and phenomenology of the face. In particular, the fleshly and communicative physicality of the face is reduced to a measurable object that can be identified by a virtualised referent and then consequently tracked. Moreover, the asymmetrical and faceless nature of these machinic programs of recognition unsettles conventional notions of civil inattention and bodily sovereignty, and the prioritisation given to pattern recognition renders them amenable to ideas/ideals from phrenology and physiognomy. In this way, algorithmic governance may generate not only forms of facial vulnerability and estrangement, but also facial artifice, where individuals come to develop tacit and artful ways of de-facing and re-facing in order to subvert the processes of recognition which leverage these modes of biopower. Thus, the datafication of urban governance gives rise to a dynamic biopolitics of the face. (shrink)

A growing literature is taking an institutionalist and governance perspective on how algorithms shape society based on unprecedented capacities for managing social complexity. Algorithmic governance altogether emerges as a novel and distinctive kind of societal steering. It appears to transcend established categories and modes of governance—and thus seems to call for new ways of thinking about how social relations can be regulated and ordered. However, as this paper argues, despite its novel way of realizing outcomes of (...) collective steering and coordination, it can nevertheless be grasped with an old and fundamental figure in political philosophy: that of Thomas Hobbes’ Leviathan. Comparing algorithmic governance with this figure serves to highlight their similarities as socio-political arrangements, and specifically to clarify how algorithmic governance parallels the apolitical traits of the Leviathan—it eliminates the political as it requires compliance and forgoing contestation to best fulfill its role and to produce satisfying outcomes. (shrink)

Algorithms are used to calculate and govern varying aspects of public life for efficient use of the vast data available about citizens. Assuming that algorithms are neutral and efficient in data-based decision making, algorithms are used in areas such as criminal justice and welfare. This has ramifications on the ideal of democratic self-government as algorithmic decisions are made without democratic deliberation, scrutiny or justification. In the book _Democracy without Shortcuts_, Cristina Lafont argued against “shortcutting” democratic self-government. Lafont’s critique of (...) shortcuts turns to problematise taken-for-granted practices in democracies that bypass citizen inclusion and equality in authoring decisions governing public life. In this article, I extend Lafont’s argument to another shortcut: the algocratic shortcut. The democratic harms attributable to the algocratic shortcut include diminishing the role of voice in politics and reducing opportunities for civic engagement. In this article, I define the algocratic shortcut and discuss the democratic harms of this shortcut, its relation to other shortcuts to democracy and the limitations of using shortcuts to remedy algocratic harms. Finally, I reflect on remedy through “aspirational deliberation”. (shrink)

As government pressure on major technology companies builds, both firms and legislators are searching for technical solutions to difficult platform governance puzzles such as hate speech and misinformation. Automated hash-matching and predictive machine learning tools – what we define here as algorithmic moderation systems – are increasingly being deployed to conduct content moderation at scale by major platforms for user-generated content such as Facebook, YouTube and Twitter. This article provides an accessible technical primer on how algorithmic moderation (...) works; examines some of the existing automated tools used by major platforms to handle copyright infringement, terrorism and toxic speech; and identifies key political and ethical issues for these systems as the reliance on them grows. Recent events suggest that algorithmic moderation has become necessary to manage growing public expectations for increased platform responsibility, safety and security on the global stage; however, as we demonstrate, these systems remain opaque, unaccountable and poorly understood. Despite the potential promise of algorithms or ‘AI’, we show that even ‘well optimized’ moderation systems could exacerbate, rather than relieve, many existing problems with content policy as enacted by platforms for three main reasons: automated moderation threatens to further increase opacity, making a famously non-transparent set of practices even more difficult to understand or audit, further complicate outstanding issues of fairness and justice in large-scale sociotechnical systems and re-obscure the fundamentally political nature of speech decisions being executed at scale. (shrink)

Probabilistic predictions of future risk govern much of society: healthcare, genetics, social media, national security, and finance. Both policy-makers and private companies are increasingly working to design institutional structures that seek to manage risk by controlling the behavior of citizens and consumers, using new technologies of predictive control that comb through past data to predict and shape future action. These predictions not only control social institutions but also shape individual character and forms of practical reason. Risk-based decision theory shifts people's (...) relationships to the future through knowledge of possible dangers and foregone opportunities, thus inspiring anxious solicitude and deceptive hopes for total security. This book uses virtue ethics to analyze these problems and to suggest ways to enjoy these technologies' positive benefits while constraining their dangerous aspects through a better understanding of practical reasoning and a theological analysis of our responsibility for future risks. (shrink)

While the academic separation of policing and regulation is still largely operative, points of convergence are more significant than ever in the digital age, starting with concomitant debates about algorithms as a new figure of power. From the policing of illegal activities to the regulation of legal ones, the algorithmization of such critical social ordering practices has been the subject of growing attention. These burgeoning discussions are focused on one common element: big data surveillance. In accordance with such similarities and (...) paralleled developments in policing and regulation, the article aims to further bridge the gap between literatures to respond to the calls for studying big data surveillance across institutional domains and social fields. To do so, it is focused on one case study that articulates algorithmic policing and regulation domains, in-between security and economic fields. This is the fight against illicit finance, i.e. ‘the global action against the financial flows that fuel crime and terrorism’. To what extent does big data surveillance make a difference in the main global policy of crime-fighting and financial regulation? Drawing on a fieldwork in a large North American bank, the present article takes stock of the algorithmic overlap between policing and regulation. It argues that the final result is policing and regulation of neither too much nor too little, which gives rise to automated and everyday mass surveillance while remaining as far removed from regulatory and crime-fighting ambitions as it is from dystopian visions of big data and algorithmic drama. (shrink)

Algorithms have developed into somewhat of a modern myth. On the one hand, they have been depicted as powerful entities that rule, sort, govern, shape, or otherwise control our lives. On the other hand, their alleged obscurity and inscrutability make it difficult to understand what exactly is at stake. What sustains their image as powerful yet inscrutable entities? And how to think about the politics and governance of something that is so difficult to grasp? This editorial essay provides a (...) critical backdrop for the special issue, treating algorithms not only as computational artifacts but also as sensitizing devices that can help us rethink some entrenched assumptions about agency, transparency, and normativity. (shrink)

Since the Enlightenment, there have been advocates for the rationalizing efficiency of enlightened sovereigns, bureaucrats, and technocrats. Today these enthusiasms are joined by calls for replacing or augmenting government with algorithms and artificial intelligence, a process already substantially under way. Bureaucracies are in effect algorithms created by technocrats that systematize governance, and their automation simply removes bureaucrats and paper. The growth of algorithmic governance can already be seen in the automation of social services, regulatory oversight, policing, the (...) justice system, and the military. However, there have also always been justified democratic and economic criticisms of autocracy, bureaucracy, technocracy, and algorithmic governance. Bureaucrats, technocrats, and algorithms embody biases that tend to serve the interests of elites, and all require transparency and democratic accountability, oversight individual citizens are ill equipped to exercise. As state apparatuses are increasingly automated, mechanisms for collective action and democratic oversight also need to be automated. Algorithms and cyborg citizenry will enable a posthuman democracy. Democratically accountable algorithmic governance, enabled by artificial intelligence and human enhancement, can automate bottom-up citizen surveillance, inform debate, aggregate decision making, and ensure the efficient working of the gradually withering state. As paid work disappears and we transition to a postcapitalist economy with a universal basic income, market mechanisms can be replaced with democratic planning. Indeed, only algorithmic governance can secure our future against accelerating threats from technological innovation. (shrink)

The algorithmic accountability literature to date has primarily focused on procedural tools to govern automated decision-making systems. That prescriptive literature elides a fundamentally empirical question: whether and under what circumstances, if any, is the use of algorithmic systems to make public policy decisions perceived as legitimate? The present study begins to answer this question. Using factorial vignette survey methodology, we explore the relative importance of the type of decision, the procedural governance, the input data used, and outcome (...) errors on perceptions of the legitimacy of algorithmic public policy decisions as compared to similar human decisions. Among other findings, we find that the type of decision—low importance versus high importance—impacts the perceived legitimacy of automated decisions. We find that human governance of algorithmic systems increases perceptions of the legitimacy of algorithmic decision-making systems, even when those decisions are likely to result in significant errors. Notably, we also find the penalty to perceived legitimacy is greater when human decision-makers make mistakes than when algorithmic systems make the same errors. The positive impact on perceived legitimacy from governance—such as human-in-the-loop—is greatest for highly pivotal decisions such as parole, policing, and healthcare. After discussing the study’s limitations, we outline avenues for future research. (shrink)

Leading digital platform providers such as Google and Uber construct marketplaces in which algorithms set prices. The efficiency-maximising free market credentials of this approach are touted by the companies involved and by legislators, policy makers and marketers. They have also taken root in the public imagination. In this article we challenge this understanding of algorithmically constructed marketplaces. We do so by returning to Hayek’s (1945) classic defence of the price mechanism, and by arguing that algorithmically-mediated price mechanisms do not, and (...) probably cannot, accumulate and signal that same kinds of knowledge that Hayek felt were essential to effective market governance. Indeed, we argue that algorithmically-constructed marketplaces are closer to, though distinct from, the central planning model that Hayek critiqued. Regardless of how you feel about Hayek’s argument, this has important consequences for how we respond to the rise of algorithmic governance tools in both markets and elsewhere. (shrink)

In our information societies, tasks and decisions are increasingly outsourced to automated systems, machines, and artificial agents that mediate human relationships, by taking decisions and acting on the basis of algorithms. This raises a critical issue: how are algorithmic procedures and applications to be appraised and governed? This question needs to be investigated, if one wishes to avoid the traps of ICTs ending up in isolating humans behind their screens and digital delegates, or harnessing them in a passive role, (...) by curtailing their freedom and autonomy. (shrink)

AI technology is capturing the African imaginations as a gateway to progress and prosperity. There is a growing interest in AI by different actors across the continent including scientists, researchers, humanitarian and aid organizations, academic institutions, tech start-ups, and media organizations. Several African states are looking to adopt AI technology to capture economic growth and development opportunities. On the other hand, African researchers highlight the gap in regulatory frameworks and policies that govern the development of AI in the continent. They (...) argue that this could lead to AI technology exacerbating problems of inequalities and injustice in the continent. However, most of the literature on AI ethics is biased toward Euro-American perspectives and lack the understanding of how AI development is apprehended in the Global South, and particularly Africa. Drawing on the case study of the first African Master’s in Machine Intelligence program, this paper argues for looking beyond the question of ethics in AI and examining AI governance issues through the analytical lens of the raciality of computing and the political economy of technoscience to understand AI development in Africa. By doing so, this paper seeks a different theorization for AI ethics from the South that is based on lived experiences of those in the margins and avoids the framings of technological futures that simplistically pathologize or celebrate Africa. (shrink)

There is increasing criticism on the use of big data and algorithms in public governance. Studies revealed that algorithms may reinforce existing biases and defy scrutiny by public officials using them and citizens subject to algorithmic decisions and services. In response, scholars have called for more algorithmic transparency and regulation. These are useful, but ex post solutions in which the development of algorithms remains a rather autonomous process. This paper argues that co-design of algorithms with relevant stakeholders (...) from government and society is another means to achieve responsible and accountable algorithms that is largely overlooked in the literature. We present a case study of the development of an algorithmic tool to estimate the populations of refugee camps to manage the delivery of emergency supplies. This case study demonstrates how in different stages of development of the tool—data selection and pre-processing, training of the algorithm and post-processing and adoption—inclusion of knowledge from the field led to changes to the algorithm. Co-design supported responsibility of the algorithm in the selection of big data sources and in preventing reinforcement of biases. It contributed to accountability of the algorithm by making the estimations transparent and explicable to its users. They were able to use the tool for fitting purposes and used their discretion in the interpretation of the results. It is yet unclear whether this eventually led to better servicing of refugee camps. (shrink)

Should decision-making algorithms be held to higher standards of transparency than human beings? The way we answer this question directly impacts what we demand from explainable algorithms, how we govern them via regulatory proposals, and how explainable algorithms may help resolve the social problems associated with decision making supported by artificial intelligence. Some argue that algorithms and humans should be held to the same standards of transparency and that a double standard of transparency is hardly justified. We give two arguments (...) to the contrary and specify two kinds of situations for which higher standards of transparency are required from algorithmic decisions as compared to humans. Our arguments have direct implications on the demands from explainable algorithms in decision-making contexts such as automated transportation. (shrink)

This article examines the interplay between contemporary algorithmic security technology and the political theory of the state of exception. I argue that the exception, as both a political and a technological concept, provides a crucial way to understand the power operating through machine learning technologies used in the security apparatuses of the modern state. I highlight how algorithmic security technology, through its inherent technical properties, carries exceptions throughout its political and technological architecture. This leads me to engage with (...) Theodor Adorno’s negative dialectics to interrogate the question of ‘ground truth’ in machine learning. I conclude that most machine learning technology asserts identity between itself and bourgeois reality – and thus inherently reinforces and reproduces the relations of domination entailed in that image of the world. However, space still exists for machine learning to operate within spaces of political non-identity, or exceptions to the bourgeois totality, and aid in liberatory politics. (shrink)

Machine learning algorithms pervade contemporary society. They are integral to social institutions, inform processes of governance, and animate the mundane technologies of daily life. Consistently, the outcomes of machine learning reflect, reproduce, and amplify structural inequalities. The field of fair machine learning has emerged in response, developing mathematical techniques that increase fairness based on anti-classification, classification parity, and calibration standards. In practice, these computational correctives invariably fall short, operating from an algorithmic idealism that does not, and cannot, address (...) systemic, Intersectional stratifications. Taking present fair machine learning methods as our point of departure, we suggest instead the notion and practice of algorithmic reparation. Rooted in theories of Intersectionality, reparative algorithms name, unmask, and undo allocative and representational harms as they materialize in sociotechnical form. We propose algorithmic reparation as a foundation for building, evaluating, adjusting, and when necessary, omitting and eradicating machine learning systems. (shrink)

We are sceptical of concerns over the opacity of algorithmic decision tools. While transparency and explainability are certainly important desiderata in algorithmic governance, we worry that automated decision-making is being held to an unrealistically high standard, possibly owing to an unrealistically high estimate of the degree of transparency attainable from human decision-makers. In this paper, we review evidence demonstrating that much human decision-making is fraught with transparency problems, show in what respects AI fares little worse or better (...) and argue that at least some regulatory proposals for explainable AI could end up setting the bar higher than is necessary or indeed helpful. The demands of practical reason require the justification of action to be pitched at the level of practical reason. Decision tools that support or supplant practical reasoning should not be expected to aim higher than this. We cast this desideratum in terms of Daniel Dennett’s theory of the “intentional stance” and argue that since the justification of action for human purposes takes the form of intentional stance explanation, the justification of algorithmic decisions should take the same form. In practice, this means that the sorts of explanations for algorithmic decisions that are analogous to intentional stance explanations should be preferred over ones that aim at the architectural innards of a decision tool. (shrink)

Inspired by recent breakthroughs in predictive modeling, practitioners in both industry and government have turned to machine learning with hopes of operationalizing predictions to drive automated decisions. Unfortunately, many social desiderata concerning consequential decisions, such as justice or fairness, have no natural formulation within a purely predictive framework. In efforts to mitigate these problems, researchers have proposed a variety of metrics for quantifying deviations from various statistical parities that we might expect to observe in a fair world and offered a (...) variety of algorithms in attempts to satisfy subsets of these parities or to trade o the degree to which they are satised against utility. In this paper, we connect this approach to fair machine learning to the literature on ideal and non-ideal methodological approaches in political philosophy. The ideal approach requires positing the principles according to which a just world would operate. In the most straightforward application of ideal theory, one supports a proposed policy by arguing that it closes a discrepancy between the real and the perfectly just world. However, by failing to account for the mechanisms by which our non-ideal world arose, the responsibilities of various decision-makers, and the impacts of proposed policies, naive applications of ideal thinking can lead to misguided interventions. In this paper, we demonstrate a connection between the fair machine learning literature and the ideal approach in political philosophy, and argue that the increasingly apparent shortcomings of proposed fair machine learning algorithms reflect broader troubles faced by the ideal approach. We conclude with a critical discussion of the harms of misguided solutions, a reinterpretation of impossibility results, and directions for future research. (shrink)

Governments and citizens of nearly every nation have been compelled to respond to COVID-19. Many measures have been adopted, including contact tracing and risk assessment algorithms, whereby citizen whereabouts are monitored to trace contact with other infectious individuals in order to generate a risk status via algorithmic evaluation. Based on 38 in-depth interviews, we investigate how people make sense of Health Code, the Chinese contact tracing and risk assessment algorithmic sociotechnical assemblage. We probe how people accept or resist (...) Health Code by examining their ongoing, dynamic, and relational interactions with it. Participants display a rich variety of attitudes toward privacy and surveillance, ranging from fatalism to the possibility of privacy to trade-offs for surveillance in exchange for public health, which is mediated by the perceived effectiveness of Health Code and changing views on the intentions of institutions who deploy it. We show how perceived competency varies not just on how well the technology works, but on the social and cultural enforcement of various non-technical aspects like quarantine, citizen data inputs, and cell reception. Furthermore, we illustrate how perceptions of Health Code are nested in people’s broader interpretations of disease control at the national and global level, and unexpectedly strengthen the Chinese authority’s legitimacy. None of the Chinese public, Health Code, or people’s perceptions toward Health Code are predetermined, fixed, or categorically consistent, but are co-constitutive and dynamic over time. We conclude with a theorization of a relational perception and methodological reflections to study algorithmic sociotechnical assemblages beyond COVID-19. (shrink)

This article examines how algorithmic accountability is translated into action at the municipal level in the United States. Based on a review of task forces, ordinances, and policy toolkits from New York City and Seattle, I demonstrate the ways municipalities and local publics operationalize abstract notions of accountability. Municipal interventions often prioritize revealing computational tools (transparency) and their effects on people (impact assessments). While these two forms of accountability are crucial, they may neglect to examine institutions—and how they change—as (...) they incorporate automated decision systems. I thus propose a political-economic approach that recognizes algorithmic systems as part of municipal institutions and focuses on their role in intensifying data collection and commodification between public agencies and markets. I argue that algorithmic accountability, especially in public agencies, needs to focus on incompetence and asymmetries of power within a network of governments, tech companies, community groups, and technologies. With a mix of transparency, impact assessments, and political economic review, the paper proposes a more comprehensive assessment of automated decision systems through their development, procurement, use, impact, and decommissioning. (shrink)

Algorithms are probably the most sophisticated tools that men have had at their disposal since the beginnings of human history. They have transformed science, industry, society. They upset the concepts of work, property, government, private life, even humanity. Going easily from one extreme to the other, we rejoice that they make life easier for us, but fear that they will enslave us. To get beyond this vision of good vs evil, this book takes a new look at our time, the (...) Age of Algorithms. Creations of the human spirit, algorithms are what we made them. And they will be what we want them to be: it's up to us to choose the world we want to live in. (shrink)

Algorithmic vision, the computational process of making meaning from digital images or visual information, has changed the relationship between the image and the human subject. In this paper, I explicate on the role of algorithmic vision as a technique of algorithmic governance, the organization of a population by algorithmic means. With its roots in the United States post-war cybernetic sciences, the ontological status of the computational image undergoes a shift, giving way to the hegemonic use (...) of automated facial recognition technologies towards predatory policing and profiling practices. By way of example, I argue that algorithmic vision reconfigures the philosophical links between vision, image, and truth, paradigmatically changing the way a human subject is represented through imagistic data. With algorithmic vision, the relationship between subject and representation challenges the humanistic discourse around images, calling for a critical displacement of the human subject from the center of an analysis of how computational images make meaning. I will explore the relationship between the operative image, the image that acts but is not seen by human eyes, and what Louise Amoore calls an “emergent subject,” a subject that is made visible through algorithmic techniques. Algorithmic vision reveals subjects to power in a mode that requires a new approach towards analyzing the entanglement and invisiblization of the human in automated decision-making systems. (shrink)

Although overarching if not foundational conceptualizations of digital governance in the field of critical data studies aptly account for and explain subjection, calculated resistance is left conceptually unattended despite case studies that document instances of resistance. I ask at the outset why conceptualizations of digital governance are so bleak, and I argue that all are underscored implicitly by a Deleuzian theory of desire that overlooks agency, defined here in Foucauldian terms. I subsequently conceptualize digital governance as encompassing (...) subjection as well as resistance, and I cast the two in relational perspective by making use of the concepts “affordance” and “assemblage” in conjunction with multiple subjectivities and Foucault's view of power as productive as well as his view of resistance as an “antagonism of strategies” in his late scholarship on resistance, ethics, and subjectivity. I offer examples of salient modes of what I call “productive” resistance, and from a Foucauldian perspective I explain how each mode targets and subverts technologies of repressive power to produce new elements of the digital environment and construct new truths. I conclude by recognizing the agency embodied in resistance as an end in itself, but I also consider that modes of productive resistance can have extrinsic value as they affect the fluid interaction among elements of the digital environment, potentially disrupting the presumed structure of dominance and dependence, and opening our conceptualization of algorithmic life to hopeful possibilities for change. (shrink)

Research on the ethics of algorithms has grown substantially over the past decade. Alongside the exponential development and application of machine learning algorithms, new ethical problems and solutions relating to their ubiquitous use in society have been proposed. This article builds on a review of the ethics of algorithms published in 2016, 2016). The goals are to contribute to the debate on the identification and analysis of the ethical implications of algorithms, to provide an updated analysis of epistemic and normative (...) concerns, and to offer actionable guidance for the governance of the design, development and deployment of algorithms. (shrink)

The world we inhabit is surrounded by ‘coded objects’ from credit cards to airplanes to telephones. Sadly the governance mechanisms of many of these technologies are only poorly understood, leading to the common premise that such technologies are ‘neutral’, thereby obscuring normative and power-related consequences of their design. In order to unpack supposedly neutral technologies, the following paper will try and foreground two of key questions around the technologies used on the global Internet: 1) how are content regulatory regimes (...) governed and 2) how are the algorithms embedded in software governed? The following paper will explore these two aspects in turn, before drawing conclusions on understanding the normative frameworks embedded in technological systems.Article first published online: 22 MARCH 2016. (shrink)

In a time of big data, thinking about how we are seen and how that affects our lives means changing our idea about who does the seeing. Data produced by machines is most often ‘seen’ by other machines; the eye is in question is algorithmic. Algorithmic seeing does not produce a computational panopticon but a mechanism of prediction. The authority of its predictions rests on a slippage of the scientific method in to the world of data. Data science (...) inherits some of the problems of science, especially the disembodied ‘view from above’, and adds new ones of its own. As its core methods like machine learning are based on seeing correlations not understanding causation, it reproduces the prejudices of its input. Rising in to the apparatuses of governance, it reinforces the problematic sides of ‘seeing like a state’ and links to the recursive production of paranoia. It forces us to ask the question ‘what counts as rational seeing?’. Answering this from a position of feminist empiricism reveals different possibilities latent in seeing with machines. Grounded in the idea of conviviality, machine learning may reveal forgotten non-market patterns and enable free and critical learning. It is proposed that a programme to challenge the production of irrational pre-emption is also a search for the possibility of algorithmic conviviality. (shrink)

Firms use algorithms to make important business decisions. To date, the algorithmic accountability literature has elided a fundamentally empirical question important to business ethics and management: Under what circumstances, if any, are algorithmic decision-making systems considered legitimate? The present study begins to answer this question. Using factorial vignette survey methodology, we explore the impact of decision importance, governance, outcomes, and data inputs on perceptions of the legitimacy of algorithmic decisions made by firms. We find that many (...) of the procedural governance mechanisms in practice today, such as notices and impact statements, do not lead to algorithmic decisions being perceived as more legitimate in general, and, consistent with legitimacy theory, that algorithmic decisions with good outcomes are perceived as more legitimate than bad outcomes. Yet, robust governance, such as offering an appeal process, can create a legitimacy dividend for decisions with bad outcomes. However, when arbitrary or morally dubious factors are used to make decisions, most legitimacy dividends are erased. In other words, companies cannot overcome the legitimacy penalty of using arbitrary or morally dubious factors, such as race or the day of the week, with a good outcome or an appeal process for individuals. These findings add new perspectives to both the literature on legitimacy and policy discussions on algorithmic decision-making in firms. (shrink)

This paper contributes to our understanding of farm data value chains with assistance from 54 semi-structured interviews and field notes from participant observations. Methodologically, it includes individuals, such as farmers, who hold well-known positionalities within digital agriculture spaces—platforms that include precision farming techniques, farm equipment built on machine learning architecture and algorithms, and robotics—while also including less visible elements and practices. The actors interviewed and materialities and performances observed thus came from spaces and places inhabited by, for example, farmers, crop (...) scientists, statisticians, programmers, and senior leadership in firms located in the U.S. and Canada. The stability of “the” artifacts followed for this project proved challenging, which led to me rethinking how to approach the subject conceptually. The paper is animated by a posthumanist commitment, drawing heavily from assemblage thinking and critical data scholarship coming out of Science and Technology Studies. The argument’s understanding of “chains” therefore lies on an alternative conceptual plane relative to most commodity chain scholarship. To speak of a data value chain is to foreground an orchestrating set of relations among humans, non-humans, products, spaces, places, and practices. The paper’s principle contribution involves interrogating lock-in tendencies at different “points” along the digital farm platform assemblage while pushing for a varied understanding of governance depending on the roles of the actors and actants involved. (shrink)

This paper contributes to inquiries into the genealogy of governmentality and the nature of secularization by arguing that pastoralism continues to operate in the algorithmic register. Drawing on Agamben’s notion of signature, I elucidate a pair of historically distant yet archaeologically proximate affinities: the first between the pastorate and algorithmic control, and the second between the absconded God of late medieval nominalism and the authority of algorithms in the cybernetic age. I support my hypothesis by attending to the (...) signaturial kinships between, on the one hand, temporality and authority in our contemporary conjuncture, and, on the other, obedience and submission in Christian thought from late antiquity and the late Middle Ages. I thereby illustrate the hidden genealogical continuities between theological-pastoral technologies of power and technocratic-algorithmic modalities of governance. I conclude by suggesting that medieval counter-conducts may be redeployed in our present circumstances for emancipatory ends. (shrink)

To date, research examining the socio-spatial effects of smart city technologies have charted how they are reconfiguring the production of space, spatiality and mobility, and how urban space is governed, but have paid little attention to how the temporality of cities is being reshaped by systems and infrastructure that capture, process and act on real-time data. In this article, we map out the ways in which city-scale Internet of Things infrastructures, and their associated networks of sensors, meters, transponders, actuators and (...) algorithms, are used to measure, monitor and regulate the polymorphic temporal rhythms of urban life. Drawing on Lefebvre, and subsequent research, we employ rhythmanalysis in conjunction with Miyazaki’s notion of ‘algorhythm’ and nascent work on algorithmic governance, to develop a concept of ‘algorhythmic governance’. We then use this framing to make sense of two empirical case studies: a traffic management system and sound monitoring and modelling. Our analysis reveals: how smart city technologies computationally perform rhythmanalysis and undertake rhythm-making that intervenes in space-time processes; distinct forms of algorhythmic governance, varying on the basis of adaptiveness, immediacy of action, and whether humans are in-, on-, or, off-the-loop; and a number of factors that shape how algorhythmic governance works in practice. (shrink)

This article analyses some of the governmental issues at stake in contemporary institutional politics in its confrontation with the challenges of digitalisation. Through notions such as algorithmic governmentality (Rouvroy and Berns), platformisation (Bratton, Stiegler), extractivism, and the affect theory (Massumi), and following a symptomatologic method, we will try to establish and discuss some key points that could be useful in order to update certain concepts regarding micro- and biopolitics (Deleuze and Guattari, Foucault), the public sphere, and the management of (...) social and individual bodies. Finally, we will try to understand how the manipulation of affects via the staging of possible catastrophic futures (Parisi) is used as a powerful neurosensory weapon, with performative effects for governing and modelling behaviours. If contemporary capitalism is capable of abstracting and extracting affects, of accumulating them and of creating value through this, which policies should we apply in this situation of uncertainty and of the manipulation of affectivity? Is there any place left for public institutions, or is the market the sole political actor left? (shrink)

Algorithmic systems are increasingly used by state agencies to inform decisions about humans. They produce scores on risks of recidivism in criminal justice, indicate the probability for a job seeker to find a job in the labor market, or calculate whether an applicant should get access to a certain university program. In this contribution, we take an interdisciplinary perspective, provide a bird’s eye view of the different key decisions that are to be taken when state actors decide to use (...) an algorithmic system, and illustrate these decisions with empirical examples from case studies. Building on these insights, we discuss the main pitfalls and promises of the use of algorithmic system by the state and focus on four levels: The most basic question whether an algorithmic system should be used at all, the regulation and governance of the system, issues of algorithm design, and, finally, questions related to the implementation of the system on the ground and the human–machine-interaction that comes with it. Based on our assessment of the advantages and challenges that arise at each of these levels, we propose a set of crucial questions to be asked when such intricate matters are addressed. (shrink)

Historically, the dynamic between philosophy of artificial intelligence and its practical application has been essential for the development of both, and thus the encounter between theory of AI and architectural/urban theory should be a site of considerable productivity. However, in many ways, it is not. This is due to two primary factors, one arising from each side of this encounter. First, legacies of overly-anthropomorphic models of AI permeate design discourses, where issues of how well AI can be constrained to social (...) issues of philosophy of mind exclude more foundational and cross-cultural questions about artificial intelligence as material process in the physical world that could inform new philosophical insights. Concurrently, the mobilization of investment in “Smart Cities” discourses dominates the space of application of AI at urban scale in ways that prematurely fixes solutions in the skeuomorphic image of architectural and urban conventions. To break this impasse, we held a series of think-tanks, workshops, and design charettes that brought together leading figures in philosophy of artificial intelligence, urban design, and commercial AI platforms. The goal was to re-think from first principles how alternative philosophical models of AI as a distributed, discontinuous, landscape-scale technology in a direct encounter with the applications in contexts of ecological sensing, automation’s impact on urban form, and issues of algorithmic governance. The design brief for this research establishes a generative framework for conceiving how embedded machine sensing and intelligence is already changing urban form and could change it in the future. The framework focuses on issues of urban zoning and architectural programming, data modeling and governance, platform cognition and design, and how these inform shifts in dynamics of public and private institutions. (shrink)

Recent rapid advances in Artificial Intelligence (AI) and Machine Learning have raised many questions about the regulatory and governance mechanisms for autonomous machines. Many commentators, scholars, and policy-makers now call for ensuring that algorithms governing our lives are transparent, fair, and accountable. Here, I propose a conceptual framework for the regulation of AI and algorithmic systems. I argue that we need tools to program, debug and maintain an algorithmic social contract, a pact between various human stakeholders, mediated (...) by machines. To achieve this, we can adapt the concept of human-in-the-loop (HITL) from the fields of modeling and simulation, and interactive machine learning. In particular, I propose an agenda I call society-in-the-loop (SITL), which combines the HITL control paradigm with mechanisms for negotiating the values of various stakeholders affected by AI systems, and monitoring compliance with the agreement. In short, ‘SITL = HITL + Social Contract.’. (shrink)

There is an abundance of enthusiasm and optimism about how governments at all levels can make use of big data, algorithms and artificial intelligence. There is also growing concern about the risks that come with these new systems. This article makes the case for greater government transparency and accountability about uses of big data through a Government of Canada qualitative research case study. Adapting a method from critical cartographers, I employ counter-mapping to map government big data practices and internal discussions (...) of risk and challenge. I do so by drawing on interviews and freedom of information requests. The analysis reveals that there are more concerns and risks than often publicly discussed and that there are significant areas of silence that need greater attention. The article underlines the need for our democratic systems to respond to our new datafied contexts by ensuring that our institutions make changes to better protect citizen rights, uphold democratic principles and ensure means for citizen intervention. (shrink)

We consider two ways one might use algorithmic randomness to characterize a probabilistic law. The first is a generative chance* law. Such laws involve a nonstandard notion of chance. The second is a probabilistic* constraining law. Such laws impose relative frequency and randomness constraints that every physically possible world must satisfy. While each notion has virtues, we argue that the latter has advantages over the former. It supports a unified governing account of non-Humean laws and provides independently motivated solutions (...) to issues in the Humean best-system account. On both notions, we have a much tighter connection between probabilistic laws and their corresponding sets of possible worlds. Certain histories permitted by traditional probabilistic laws are ruled out as physically impossible. As a result, such laws avoid one variety of empirical underdetermination, but the approach reveals other varieties of underdetermination that are typically overlooked. (shrink)