We discuss potential biorisks from large language models (LLMs). AI assistants based on LLMs such as ChatGPT have been shown to significantly reduce barriers to entry for actors wishing to synthesize dangerous, potentially novel pathogens and chemical weapons. The harms from deploying such bioagents could be further magnified by AI-assisted misinformation. We endorse several policy responses to these dangers, including prerelease evaluations of biomedical AIs by subject-matter experts, enhanced surveillance and lab screening procedures, restrictions on AI training data, and (...) access controls on biomedical AI. We conclude with a suggestion about future research directions in bioethics. (shrink)

Large Language Models (LLMs) have been transformative. They are pre-trained foundational models that are self-supervised and can be adapted with fine tuning to a wide range of natural language tasks, each of which previously would have required a separate network model. This is one step closer to the extraordinary versatility of human language. GPT-3 and more recently LaMDA can carry on dialogs with humans on many topics after minimal priming with a few examples. However, there has (...) been a wide range of reactions and debate on whether these LLMs understand what they are saying or exhibit signs of intelligence. This high variance is exhibited in three interviews with LLMs reaching wildly different conclusions. A new possibility was uncovered that could explain this divergence. What appears to be intelligence in LLMs may in fact be a mirror that reflects the intelligence of the interviewer, a remarkable twist that could be considered a Reverse Turing Test. If so, then by studying interviews we may be learning more about the intelligence and beliefs of the interviewer than the intelligence of the LLMs. As LLMs become more capable they may transform the way we interact with machines and how they interact with each other. Increasingly, LLMs are being coupled with sensorimotor devices. LLMs can talk the talk, but can they walk the walk? A road map for achieving artificial general autonomy is outlined with seven major improvements inspired by brain systems. LLMs could be used to uncover new insights into brain function by downloading brain data during natural behaviors. (shrink)

To what degree can language be acquired from linguistic input alone? This question has vexed scholars for millennia and is still a major focus of debate in the cognitive science of language. The complexity of human language has hampered progress because studies of language–especially those involving computational modeling–have only been able to deal with small fragments of our linguistic skills. We suggest that the most recent generation of Large Language Models (LLMs) might finally provide the (...) computational tools to determine empirically how much of the human language ability can be acquired from linguistic experience. LLMs are sophisticated deep learning architectures trained on vast amounts of natural language data, enabling them to perform an impressive range of linguistic tasks. We argue that, despite their clear semantic and pragmatic limitations, LLMs have already demonstrated that human-like grammatical language can be acquired without the need for a built-in grammar. Thus, while there is still much to learn about how humans acquire and use language, LLMs provide full-fledged computational models for cognitive scientists to empirically evaluate just how far statistical learning might take us in explaining the full complexity of human language. (shrink)

Do large language models like Chat-GPT or Claude meaningfully use the words they produce? Or are they merely clever prediction machines, simulating language use by producing statistically plausible text? There have already been some initial attempts to answer this question by showing that these models meet the criteria for entering meaningful states according to metasemantic theories of mental content. In this paper, I will argue for a different approach—that we should instead consider whether language models meet the (...) criteria given by our best metasemantic theories of linguistic content. In that vein, I will illustrate how this can be done by applying two such theories to the case of language models: Gareth Evans’ ( 1982 ) account of naming practices and Ruth Millikan’s ( 1984, 2004, 2005 ) teleosemantics. In doing so, I will argue that it is a mistake to think that the failure of LLMs to meet plausible conditions for mental intentionality thereby renders their outputs meaningless, and that a distinguishing feature of linguistic intentionality—dependency on a pre-existing linguistic system—allows for the plausible result that LLM outputs are meaningful. (shrink)

Are LLMs cultural technologies like photocopiers or printing presses, which transmit information but cannot create new content? A challenge for this idea, which we call bibliotechnism, is that LLMs generate novel text. We begin with a defense of bibliotechnism, showing how even novel text may inherit its meaning from original human-generated text. We then argue that bibliotechnism faces an independent challenge from examples in which LLMs generate novel reference, using new names to refer to new entities. Such examples could be (...) explained if LLMs were not cultural technologies but had beliefs, desires, and intentions. According to interpretationism in the philosophy of mind, a system has such attitudes if and only if its behavior is well explained by the hypothesis that it does. Interpretationists may hold that LLMs have attitudes, and thus have a simple solution to the novel reference problem. We emphasize, however, that interpretationism is compatible with very simple creatures having attitudes and differs sharply from views that presuppose these attitudes require onsciousness, sentience, or intelligence (topics about which we make no claims). (shrink)

Current work in language models (LMs) helps us speed up or even skip thinking by accelerating and automating cognitive work. But can LMs help us with critical thinking -- thinking in deeper, more reflective ways which challenge assumptions, clarify ideas, and engineer new concepts? We treat philosophy as a case study in critical thinking, and interview 21 professional philosophers about how they engage in critical thinking and on their experiences with LMs. We find that philosophers do not find LMs (...) to be useful because they lack a sense of selfhood (memory, beliefs, consistency) and initiative (curiosity, proactivity). We propose the selfhood-initiative model for critical thinking tools to characterize this gap. Using the model, we formulate three roles LMs could play as critical thinking tools: the Interlocutor, the Monitor, and the Respondent. We hope that our work inspires LM researchers to further develop LMs as critical thinking tools and philosophers and other 'critical thinkers' to imagine intellectually substantive uses of LMs. (shrink)

Large language models (LLMs) such as ChatGPT or Google’s Bard have shown significant performance on a variety of text-based tasks, such as summarization, translation, and even the generation of new...

Can large language models produce expert‐quality philosophical texts? To investigate this, we fine‐tuned GPT‐3 with the works of philosopher Daniel Dennett. To evaluate the model, we asked the real Dennett 10 philosophical questions and then posed the same questions to the language model, collecting four responses for each question without cherry‐picking. Experts on Dennett's work succeeded at distinguishing the Dennett‐generated and machine‐generated answers above chance but substantially short of our expectations. Philosophy blog readers performed similarly to (...) the experts, while ordinary research participants were near chance distinguishing GPT‐3's responses from those of an “actual human philosopher”. (shrink)

Large language models (LLMs) have now entered the realm of medical ethics. In a recent study, Balaset alexamined the performance of GPT-4, a commercially available LLM, assessing its performance in generating responses to diverse medical ethics cases. Their findings reveal that GPT-4 demonstrates an ability to identify and articulate complex medical ethical issues, although its proficiency in encoding the depth of real-world ethical dilemmas remains an avenue for improvement. Investigating the integration of LLMs into medical ethics decision-making appears to (...) be an interesting avenue of research. However, despite the promising trajectory of LLM technology in medicine, it is crucial to exercise caution and refrain from attributing their expertise to medical ethics. Our thesis follows an examination of the nature of expertise and the epistemic limitations that affect LLM technology. As a result, we propose two more fitting applications of LLMs in medical ethics: first, as tools for mining electronic health records or scientific literature, thereby supplementing evidence for resolving medical ethics cases, and second, as educational platforms to foster ethical reflection and critical thinking skills among students and residents. The integration of LLMs in medical ethics, while promising, requires careful consideration of their epistemic limitations. Consequently, a well-considered definition of their role in ethically sensitive decision-making is crucial. (shrink)

Large Language Models (LLMs) could be a useful tool for lawyers. However, empirical research on their effectiveness in conducting legal tasks is scant. We study securities cases involving cryptocurrencies as one of numerous contexts where AI could support the legal process, studying GPT-3.5’s legal reasoning and ChatGPT’s legal drafting capabilities. We examine whether a) GPT-3.5 can accurately determine which laws are potentially being violated from a fact pattern, and b) whether there is a difference in juror decision-making based on (...) complaints written by a lawyer compared to ChatGPT. We feed fact patterns from real-life cases to GPT-3.5 and evaluate its ability to determine correct potential violations from the scenario and exclude spurious violations. Second, we had mock jurors assess complaints written by ChatGPT and lawyers. GPT-3.5’s legal reasoning skills proved weak, though we expect improvement in future models, particularly given the violations it suggested tended to be correct (it merely missed additional, correct violations). ChatGPT performed better at legal drafting, and jurors’ decisions were not statistically significantly associated with the author of the document upon which they based their decisions. Because GPT-3.5 cannot satisfactorily conduct legal reasoning tasks, it would be unlikely to be able to help lawyers in a meaningful way at this stage. However, ChatGPT’s drafting skills (though, perhaps, still inferior to lawyers) could assist lawyers in providing legal services. Our research is the first to systematically study an LLM’s legal drafting and reasoning capabilities in litigation, as well as in securities law and cryptocurrency-related misconduct. (shrink)

Newly powerful large language models have burst onto the scene, with applications across a wide range of functions. We can now expect to encounter their outputs at rapidly increasing volumes and frequencies. Some commentators claim that large language models are bullshitting, generating convincing output without regard for the truth. If correct, that would make large language models distinctively dangerous discourse participants. Bullshitters not only undermine the norm of truthfulness (by saying false things) but the normative status of (...) truth itself (by treating it as entirely irrelevant). So, do large language models really bullshit? I argue that they can, in the sense of issuing propositional content in response to fact-seeking prompts, without having first assessed that content for truth or falsity. However, I further argue that they need not bullshit, given appropriate guardrails. So, just as with human speakers, the propensity for a large language model to bullshit depends on its own particular make-up. (shrink)

In a recent letter, Dillion et. al (2023) make various suggestions regarding the idea of artificially intelligent systems, such as large language models, replacing human subjects in empirical moral psychology. We argue that human subjects are in various ways indispensable.

This essay explores the possible contribution of Large Language Models (LLMs) to human cognition. It investigates whether human cognition can be enhanced by advanced AI systems such as LLMs. Can LLMs make people as learners smarter, or, on the contrary, make them reason/think less? The author discusses the concepts of human and artificial intelligence and examines LLMs as advanced AI systems, which use deep learning techniques and can be considered as excelling in neural network architectures, data volume, generalisation and (...) scalability. The author suggests that while LLMs can assist in facilitating numerous cognitive tasks, more research and philosophical inquiry is needed to understand whether such kind of AI assistance would make people cultivate human intelligence more, and not less. Presumably, Large Language Models (LLMs) can contribute to human intelligence and cognition just under strict (addressed existing limitations, questioning prompting, time-sensitivity, etc.) conditions. However, it is important that these theoretical considerations could be verified by experimental research. (shrink)

This article sets in with the question whether current or foreseeable transformer-based large language models (LLMs), such as the ones powering OpenAI’s ChatGPT, could be language users in a way comparable to humans. It answers the question negatively, presenting the following argument. Apart from niche uses, to use language means to act. But LLMs are unable to act because they lack intentions. This, in turn, is because they are the wrong kind of being: agents with intentions need (...) to be autonomous organisms while LLMs are heteronomous mechanisms. To conclude, the article argues, based on structural aspects of transformer-based LLMs, that these LLMs have taken a first step away from mechanistic artificiality to autonomous self-constitution, which means that these models are (slowly) moving into a direction that someday might result in non-human, but equally non-artificial agents, thus subverting the time-honored Kantian distinction between organism and mechanism. (shrink)

Today, large language models are very powerful, informational and analytical tools that significantly accelerate most of the existing methods and methodologies for processing informational processes. Scientific information is of particular importance in this capacity, which gradually involves the power of large language models. This interaction of science and qualitative new opportunities for working with information lead us to new, unique scientific discoveries, their great quantitative diversity. There is an acceleration of scientific research, a reduction in the time spent (...) on its implementation – the freed up time can be spent both on solving new scientific problems and on scientific creativity, which, although it may not necessarily lead to a specific solution to a particular scientific problem, but is able to demonstrate the beauty of science in various disciplinary areas. As a result, the interaction of large language models and scientific information is at the same time a research for solutions to scientific problems, scientific problems, and scientific creativity. Solving scientific problems requires the ability to efficiently process big data, which cannot be done without an effective method – one of the significant methods was the Transformer architecture, introduced in 2017 and comprehensively integrated into the GPT‑3 model, which, as of September 2020, was the largest and most advanced language model in the world. Therefore, GPT‑3 can be called the basis of most scientific developments carried out in the context of using large language models. The interaction of science and large language models has become a factor in the emergence of a large number of questions, among which are: «Is the result of data analysis new knowledge?», «What are the prospects for scientific creativity in the era of big computing?». Currently, these issues are extremely important, because they allow us to develop the foundations for effective human‑computer interaction. Therefore, this study analyzes the issues presented. (shrink)

[This is an edited version of a keynote talk at the conference on Neural Information Processing Systems (NeurIPS) on November 28, 2022, with some minor additions and subtractions.] -/- There has recently been widespread discussion of whether large language models might be sentient or conscious. Should we take this idea seriously? I will break down the strongest reasons for and against. Given mainstream assumptions in the science of consciousness, there are significant obstacles to consciousness in current models: for example, (...) their lack of recurrent processing, a global workspace, and unified agency. At the same time, it is quite possible that these obstacles will be overcome in the next decade or so. I conclude that while it is somewhat unlikely that current large language models are conscious, we should take seriously the possibility that successors to large language models may be conscious in the not-too-distant future. (shrink)

This letter explores the intricate historical and contemporary links between large language models (LLMs) and cognitive science through the lens of information theory, statistical language models, and socioanthropological linguistic theories. The emergence of LLMs highlights the enduring significance of information‐based and statistical learning theories in understanding human communication. These theories, initially proposed in the mid‐20th century, offered a visionary framework for integrating computational science, social sciences, and humanities, which nonetheless was not fully fulfilled at that time. The subsequent (...) development of sociolinguistics and linguistic anthropology, especially since the 1970s, provided critical perspectives and empirical methods that both challenged and enriched this framework. This letter proposes that two pivotal concepts derived from this development, metapragmatic function and indexicality, offer a fruitful theoretical perspective for integrating the semantic, textual, and pragmatic, contextual dimensions of communication, an amalgamation that contemporary LLMs have yet to fully achieve. The author believes that contemporary cognitive science is at a crucial crossroads, where fostering interdisciplinary dialogues among computational linguistics, social linguistics and linguistic anthropology, and cognitive and social psychology is in particular imperative. Such collaboration is vital to bridge the computational, cognitive, and sociocultural aspects of human communication and human−AI interaction, especially in the era of large language and multimodal models and human‐centric Artificial Intelligence (AI). (shrink)

Artificial intelligence chatbots driven by large language models (LLMs) have the potential to increase public science literacy and support scientific research, as they can quickly summarize complex scientific information in accessible terms. However, when summarizing scientific texts, LLMs may omit details that limit the scope of research conclusions, leading to generalizations of results broader than warranted by the original study. We tested 10 prominent LLMs, including ChatGPT-4o, ChatGPT-4.5, DeepSeek, LLaMA 3.3 70B, and Claude 3.7 Sonnet, comparing 4900 LLM-generated summaries (...) to their original scientific texts. Even when explicitly prompted for accuracy, most LLMs produced broader generalizations of scientific results than those in the original texts, with DeepSeek, ChatGPT-4o, and LLaMA 3.3 70B overgeneralizing in 26–73% of cases. In a direct comparison of LLM-generated and human-authored science summaries, LLM summaries were nearly five times more likely to contain broad generalizations (OR = 4.85, 95% CI [3.06, 7.70], p < 0.001). Notably, newer models tended to perform worse in generalization accuracy than earlier ones. Our results indicate a strong bias in many widely used LLMs towards overgeneralizing scientific conclusions, posing a significant risk of large-scale misinterpretations of research findings. We highlight potential mitigation strategies, including lowering LLM temperature settings and benchmarking LLMs for generalization accuracy. (shrink)

Could the employment of large language models (LLMs) in place of human advisors improve the problem-solving ability of democratic assemblies? LLMs represent the most significant recent incarnation of artificial intelligence and could change the future of democratic governance. This paper assesses their potential to serve as expert advisors to democratic representatives. While LLMs promise enhanced expertise availability and accessibility, they also present specific challenges. These include hallucinations, misalignment and value imposition. After weighing LLMs’ benefits and drawbacks against human advisors, (...) I argue that time-tested democratic procedures, such as deliberation and aggregation by voting, provide safeguards that are effective against human and machine advisor shortcomings alike. Additional protective measures may include custom training for advisor LLMs or boosting representatives’ competencies in query formulation. Implementation of adversarial proceedings in which LLM advisors would debate each other and provide dissenting opinions is likely to yield further epistemic benefits. Overall, promising interventions that would mitigate the LLM risks appear feasible. Machine advisors could thus empower human decision-makers to make more autonomous, higher-quality decisions. On this basis, I defend the hypothesis that LLMs’ careful integration into policymaking could augment democracy’s ability to address today’s complex social problems. (shrink)

Abstract The explosion of interest in large language models (LLMs) has been accompanied by concerns over the extent to which generated outputs can be trusted, owing to the prevalence of bias, hallucinations, and so forth. Accordingly, there is a growing interest in the use of ontologies and knowledge graphs to make LLMs more trustworthy. This rests on the long history of ontologies and knowledge graphs in constructing human-comprehensible justification for model outputs as well as traceability concerning the impact (...) of evidence on other evidence. Understanding the nature of arguments and argumentation is critical to each, especially when LLM output conflicts with what is expected by users. The central contribution of this article is to extend the Arguments Ontology (ARGO) - an ontology specific to the domain of argumentation and evidence broadly construed - into the space of LLM fact-checking in the interest of promoting justification and traceability research through the use of ARGO-based ‘blueprints’. (shrink)

In the target article, Porsdam Mann and colleagues (2023) broadly survey the ethical opportunities and risks of using general and personalized large language models (LLMs) to generate academic pros...

Humans can attribute beliefs to others. However, it is unknown to what extent this ability results from an innate biological endowment or from experience accrued through child development, particularly exposure to language describing others' mental states. We test the viability of the language exposure hypothesis by assessing whether models exposed to large quantities of human language display sensitivity to the implied knowledge states of characters in written passages. In pre‐registered analyses, we present a linguistic version of the (...) False Belief Task to both human participants and a large language model, GPT‐3. Both are sensitive to others' beliefs, but while the language model significantly exceeds chance behavior, it does not perform as well as the humans nor does it explain the full extent of their behavior—despite being exposed to more language than a human would in a lifetime. This suggests that while statistical learning from language exposure may in part explain how humans develop the ability to reason about the mental states of others, other mechanisms are also responsible. (shrink)

If Large Language Models can make real scientific contributions, then they can genuinely use language, be systematically wrong, and be held responsible for their errors. AI models which can make scientific contributions thereby meet the criteria for scientific authorship.

This paper explores the intersection of AI hallucinations and the question of AI consciousness, examining whether the erroneous outputs generated by large language models (LLMs) could be mistaken for signs of emergent intelligence. AI hallucinations, which are false or unverifiable statements produced by LLMs, raise significant philosophical and ethical concerns. While these hallucinations may appear as data anomalies, they challenge our ability to discern whether LLMs are merely sophisticated simulators of intelligence or could develop genuine cognitive processes. By analyzing (...) the causes of AI hallucinations, their impact on the perception of AI cognition, and the potential implications for AI consciousness, this paper contributes to the ongoing discourse on the nature of artificial intelligence and its future evolution. (shrink)

How does our language relate to reality? This is a question that is especially pertinent in set theory, where we seem to talk of large infinite entities. Based on an analogy with the use of models in the natural sciences, we argue for a threefold correspondence between our language, models, and reality. We argue that so conceived, the existence of models can be underwritten by a weak notion of existence, where weak existence is to be understood as existing (...) in virtue of language. (shrink)

Word co‐occurrence patterns in language corpora contain a surprising amount of conceptual knowledge. Large language models (LLMs), trained to predict words in context, leverage these patterns to achieve impressive performance on diverse semantic tasks requiring world knowledge. An important but understudied question about LLMs’ semantic abilities is whether they acquire generalized knowledge of common events. Here, we test whether five pretrained LLMs (from 2018's BERT to 2023's MPT) assign a higher likelihood to plausible descriptions of agent−patient interactions than (...) to minimally different implausible versions of the same event. Using three curated sets of minimal sentence pairs (total n = 1215), we found that pretrained LLMs possess substantial event knowledge, outperforming other distributional language models. In particular, they almost always assign a higher likelihood to possible versus impossible events (The teacher bought the laptop vs. The laptop bought the teacher). However, LLMs show less consistent preferences for likely versus unlikely events (The nanny tutored the boy vs. The boy tutored the nanny). In follow‐up analyses, we show that (i) LLM scores are driven by both plausibility and surface‐level sentence features, (ii) LLM scores generalize well across syntactic variants (active vs. passive constructions) but less well across semantic variants (synonymous sentences), (iii) some LLM errors mirror human judgment ambiguity, and (iv) sentence plausibility serves as an organizing dimension in internal LLM representations. Overall, our results show that important aspects of event knowledge naturally emerge from distributional linguistic patterns, but also highlight a gap between representations of possible/impossible and likely/unlikely events. (shrink)

Large language models (LLMs) such as ChatGPT are opening up new areas of research and teaching potential across a variety of domains. The purpose of the present conceptual paper is to map this new terrain from the point of view of Critical Discourse Studies (CDS). We demonstrate that the usage of LLMs raises concerns that definitely fall within the remit of CDS; among them, power and inequality. After an initial explanation of LLMs, we focus on three key areas of (...) reflection. The first is a general stock-taking, where we look at CDS’ theoretical underpinnings and what they imply for working with AI-generated language data. The second issue is authorship, where we assess the traceability of linguistic metadata and the ethically sensitive situation with regard to ownership of texts. The third area is linguistic homogenisation, where we examine how LLM usage privileges the mainstream. Afterwards, we explore ways in which LLMs could be used in research, and we discuss the implications of exploring their use in the classroom through a CDS lens. We close the paper with some observations on likely future developments in AI and how CDS can contribute with its distinctive theoretical, methodological and critical apparatus. (shrink)

Literacy is in decline in many parts of the world, accompanied by drops in associated cognitive skills (including IQ) and an increasing susceptibility to fake news. It is possible that the recent explosive growth and widespread deployment of Large Language Models (LLMs) might exacerbate this trend, but there is also a chance that LLMs can help turn things around. We argue that cognitive science is ideally suited to help steer future literacy development in the right direction by challenging and (...) informing current educational practices and policy. Cognitive scientists have the right interdisciplinary skills to study, analyze, evaluate, and change LLMs to facilitate their critical use, to encourage turn-taking that promotes rather than hinders literacy, to support literacy acquisition in diverse and equitable ways, and to scaffold potential future changes in what it means to be literate. We urge cognitive scientists to take up this mantle—the future impact of LLMs on human literacy skills is too important to be left to the large, predominately U.S.-based tech companies. (shrink)

Informed consent in surgical settings requires not only the accurate communication of medical information but also the establishment of trust through empathic engagement. The use of large language models (LLMs) offers a novel opportunity to enhance the informed consent process by combining advanced information retrieval capabilities with simulated emotional responsiveness. However, the ethical implications of simulated empathy raise concerns about patient autonomy, trust and transparency. This paper examines the challenges of surgical informed consent, the potential benefits and limitations of (...) digital tools such as LLMs and the ethical implications of simulated empathy. We distinguish between active empathy, which carries the risk of creating a misleading illusion of emotional connection and passive empathy, which focuses on recognising and signalling patient distress cues, such as fear or uncertainty, rather than attempting to simulate genuine empathy. We argue that LLMs should be limited to the latter, recognising and signalling patient distress cues and alerting healthcare providers to patient anxiety. This approach preserves the authenticity of human empathy while leveraging the analytical strengths of LLMs to assist surgeons in addressing patient concerns. This paper highlights how LLMs can ethically enhance the informed consent process without undermining the relational integrity essential to patient-centred care. By maintaining transparency and respecting the irreplaceable role of human empathy, LLMs can serve as valuable tools to support, rather than replace, the relational trust essential to informed consent. (shrink)

Despite the exceptional performance of large language models (LLMs) on a wide range of tasks involving natural language processing and reasoning, there has been sharp disagreement as to whether their abilities extend to more creative human abilities. A core example is the interpretation of novel metaphors. Here we assessed the ability of GPT-4, a state-of-the-art large language model, to provide natural-language interpretations of a recent AI benchmark (Fig-QA dataset), novel literary metaphors drawn from Serbian poetry (...) and translated into English, and entire novel English poems. GPT-4 outperformed previous AI models on the Fig-QA dataset. For metaphors drawn from Serbian poetry, human judges – blind to the fact that an AI model was involved – rated metaphor interpretations generated by GPT-4 as superior to those provided by a group of college students. In interpreting reversed metaphors, GPT-4, as well as humans, exhibited signs of sensitivity to the Gricean cooperative principle. In addition, for several novel English poems GPT-4 produced interpretations that were rated as excellent or good by a human literary critic. These results indicate that LLMs such as GPT-4 have acquired an emergent ability to interpret literary metaphors, including those embedded in novel poems. (shrink)

Despite the exceptional performance of large language models (LLMs) on a wide range of tasks involving natural language processing and reasoning, there has been sharp disagreement as to whether their abilities extend to more creative human abilities. A core example is the interpretation of novel metaphors. Here we assessed the ability of GPT-4, a state-of-the-art large language model, to provide natural-language interpretations of a recent AI benchmark (Fig-QA dataset), novel literary metaphors drawn from Serbian poetry (...) and translated into English, and entire novel English poems. GPT-4 outperformed previous AI models on the Fig-QA dataset. For metaphors drawn from Serbian poetry, human judges – blind to the fact that an AI model was involved – rated metaphor interpretations generated by GPT-4 as superior to those provided by a group of college students. In interpreting reversed metaphors, GPT-4, as well as humans, exhibited signs of sensitivity to the Gricean cooperative principle. In addition, for several novel English poems GPT-4 produced interpretations that were rated as excellent or good by a human literary critic. These results indicate that LLMs such as GPT-4 have acquired an emergent ability to interpret literary metaphors, including those embedded in novel poems. (shrink)

We propose leveraging cognitive science research on emotions and communication to improve language models for emotion analysis. First, we present the main emotion theories in psychology and cognitive science. Then, we introduce the main methods of emotion annotation in natural language processing and their connections to psychological theories. We also present the two main types of analyses of emotional communication in cognitive pragmatics. Finally, based on the cognitive science research presented, we propose directions for improving language models (...) for emotion analysis. We suggest that these research efforts pave the way for constructing new annotation schemes, methods, and a possible benchmark for emotional understanding, considering different facets of human emotion and communication. (shrink)

By now a consensus has emerged that people, when subjected to high-stakes decisions through automated decision systems, have a moral right to have these decisions explained to them. However, furnishing such explanations can be costly. So the right to an explanation creates what we call the cost problem: providing subjects of automated decisions with appropriate explanations of the grounds of these decisions can be costly for the companies and organisations that use these automated decision systems. In this paper, we explore (...) whether large language models could prove significant in overcoming the cost problem. We provide an initial case for believing that they can but only with serious ethical costs. (shrink)

Recently developed large language models (LLMs) perform surprisingly well in many language-related tasks, ranging from text correction or authentic chat experiences to the production of entirely new texts or even essays. It is natural to get the impression that LLMs know the meaning of natural language expressions and can use them productively. Recent scholarship, however, has questioned the validity of this impression, arguing that LLMs are ultimately incapable of understanding and producing meaningful texts. This paper develops a (...) more optimistic view. Drawing on classic externalist accounts of reference, it argues that LLM-generated texts meet the conditions of successful reference. This holds at least for proper names and so-called paradigm terms. The key insight here is that the LLM may inherit reference from its training-data through a reference-sustaining training mechanism. (shrink)

Keywords are essential to the searchability and therefore discoverability of museum and archival collections in the modern world. Without them, the collection management systems (CMS) and online collections these cultural organisations rely on to record, organise, and make their collections accessible, do not operate efficiently. However, generating these keywords manually is time consuming for these already resource strapped organisations. Artificial intelligence (AI), particularly generative AI and Large Language Models (LLMs), could hold the key to generating, even automating, this key (...) data and as such be considered a co-creative add-on. This study contributes to the literature by introducing the use of Meta’s open-source LLM, Llama, to generate keywords from curator/archivist written descriptions of museum and archival collection items. Our findings suggest that these technologies add significant value compared to current manual methods for keyword generation. In particular, we find that through using carefully crafted prompts, successful keyword augmentations could be established making museum and archival collections much more accessible to wider and more diverse audiences. However, the results also showed that generative AI has biases (e.g., hallucinations, over generalisations, outdated language), though the frequency of occurrence was not as high as general perception may insist. Hence, we also discuss mitigation strategies to address these and how cultural institutions can recognise the risks and errors while getting the most from the systems. Finally, we discuss options to achieve structured results which allow easier ingestion of data back into CMS. Ultimately, LLMs hold significant potential to enhance accessibility to museum and archival collections, yet they are not without imperfection as we extensively discuss. (shrink)

Wittgenstein’s ideas are a common ground for developers of Natural Language Processing (NLP) systems and linguists working on Language Acquisition and Mastery (LAM) models (Mills 1993; Lowney, Levy, Meroney and Gayler 2020; Skelac and Jandrić 2020). In recent years, we have witnessed a fast development of NLP systems capable of performing tasks as never before. NLP and LAM have been implemented based on deep learning neural networks, which learn concepts representation from rough data, but are nonetheless very effective (...) in tasks such as question answering, textual entailment, and translation (Devlin et al. 2019; Kitaev, Cao and Klein 2019; Wang et al. 2019). In this paper, I will debate some Wittgensteinian concepts that impact the architectures of many NLP deep-learning systems. I will focus, in particular, on the attempt to build a specific kind of architecture to model a private language. The discussion, I think, helps extract philosophical assumptions leading the research and development of AI systems capable of language modelling. In this paper, I will address some of the main features of NLP systems2 used for word embedding and one proposal to manipulate through a neural network a form of private language (Lawney et al. 2020). In §2, I will reconstruct the complex path of the private language argument (PLA). In §3, I will discuss connectivist language models and introduce notions about NPL systems' architecture. An overview of this kind of model is helpful to introduce the work of Lowney, Levy, Meroney and Gayler (2020). They submit that their model can respond to the issues raised by Wittgenstein in the notorious Private Language Argument (PLA). This argument unexpectedly turned out to be relevant not only for the philosophy of language but also for NLP and LAM modellers. I will describe the language game concept in NLP, how it is embedded, and its role in inductive systems development. This central concept in Wittgenstein’s work is relevant to describe the role of context in understanding the meanings of words. In §4, I present the Wittgensteinian main concepts in play in the connectivist paradigm. I argue that the connectionist theoretical framework can better catch the dependency of word meaning on context. There seems to be a correlation between Wittgenstein's invitation to look, an invitation to dismiss the aim of theorizing about languages, and the absence of theory-ladenness in deep learning technologies involved in NLP. In §5, I criticize Lowney and colleagues’ claim, whose model does not successfully capture Wittgenstein’s Beetle in the Box case. Moreover, I argue that even if we can distinguish a strong and a weak definition of a private language, Wittgenstein’s argument also holds for deep-learning models, and his worries are still a good guide for NLP developers. (shrink)

This paper examines the potential educational uses of chat-based large language models (LLMs), moving past initial hype and skepticism. Although LLM outputs often evoke fascination and resemble human writing, they are unpredictable and must be used with discernment. Several metaphors—like calculators, cars, and drunk tutors—highlight distinct models for student interactions with LLMs, which we explore in the paper. We suggest that LLMs hold a potential in students’ learning by fostering proleptic reasoning through scaffolding, i.e., presenting a technological accompaniment in (...) anticipating and responding to potential objections to arguments. Here, the technical limitations of LLMs can be reframed as beneficial when fostering anticipatory reasoning. Whether their outputs are accurate or not, evaluating them stimulates learning. LLMs require students to critically engage, emphasizing analytical thinking over mere memorization. This interaction helps solidify knowledge. Additionally, we explore how engaging with LLMs can prepare students for constructive collective discussions and provide first steps in addressing epistemic injustices by highlighting potential research blind spots. Thus, while acknowledging the sociopolitical and ethical complexities of using LLMs in education, we suggest that when used in an informed way, they can promote critical thinking through anticipatory reasoning. (shrink)

In natural language understanding, a crucial goal is correctly interpreting open-textured phrases. In practice, disagreements over the meanings of open-textured phrases are often resolved through the generation and evaluation of interpretive arguments, arguments designed to support or attack a specific interpretation of an expression within a document. In this paper, we discuss some of our work towards the goal of automatically generating and evaluating interpretive arguments. We have curated a set of rules from the code of ethics of various (...) professional organizations and a set of associated scenarios that are ambiguous with respect to some open-textured phrase within the rule. We collected and evaluated arguments from both human annotators and state-of-the-art generative language models in order to determine the relative quality and persuasiveness of both sets of arguments. Finally, we performed a Turing test-inspired study in order to assess whether human annotators can tell the difference between human arguments and machine-generated arguments. The results show that machine-generated arguments, when prompted a certain way, can be consistently rated as more convincing than human-generated arguments, and to the untrained eye, the machine-generated arguments can convincingly sound human-like. (shrink)

Cohen (2023) takes a fair and measured approach to the question of what ChatGPT means for bioethics. The hype cycles around AI often obscure the fact that ethicists have developed robust frameworks...

Large language models (LLMs) have been proposed as candidate models of human semantics, and as such, they must be able to account for conceptual combination. This work explores the ability of two LLMs, namely, BERT-base and Llama-2-13b, to reveal the implicit meaning of existing and novel compound words. According to psycholinguistic theories, understanding the meaning of a compound (e.g., “snowman”) involves its automatic decomposition into constituent meanings (“snow,” “man”), which are then connected by an implicit semantic relation selected from (...) a set of possible competitors (FOR, MADE OF, BY, …) to obtain a plausible interpretation (“man MADE OF snow”). Here, we leverage the flexibility of LLMs to obtain contextualized representations for both target compounds (e.g., “snowman”) and their implicit interpretations (e.g., “man MADE OF snow”). We demonstrate that replacing a compound with a paraphrased version leads to changes to the embeddings that are inversely proportional to the paraphrase's plausibility, estimated by human raters. While this relation holds for both existing and novel compounds, results obtained for novel compounds are substantially weaker, and older distributional models outperform LLMs. Nonetheless, the present results show that LLMs can offer a valid approximation of the internal structure of compound words posited by cognitive theories, thus representing a promising tool to model word senses that are at once implicit and possible. (shrink)

This paper considers the kind of introspection that large language models (LLMs) might be able to have. It argues that LLMs, while currently limited in their introspective capabilities, are not inherently unable to have such capabilities: they already model the world, including mental concepts, and already have some introspection-like capabilities. With deliberate training, LLMs may develop introspective capabilities. The paper proposes a method for such training for introspection, situates possible LLM introspection in the 'possible forms of introspection' framework (...) proposed by Kammerer and Frankish, and considers the ethical ramifications of introspection and self-report in AI systems. (shrink)

Can large language models be considered intelligent? Arguments against this proposition often assume that genuine intelligence cannot exist without consciousness, understanding, or creative thinking. We discuss each of these roadblocks to machine intelligence and conclude that, in light of findings and conceptualizations in scientific research on these topics, none of them rule out the possibility of viewing current AI systems based on large language models as intelligent. We argue that consciousness is not relevant for AI, while creativity and (...) understanding should be considered functional traits that, in principle, can be implemented in a machine. Many arguments in circulation that claim current AI systems necessarily lack these traits either rely on human exceptionalism or mistaken understanding of human intelligence. Arguments that highlight alleged flaws in current systems—such as a lack of reliability, agency, or understanding—may be important, but they often obscure the many qualitative similarities between human cognition and current AI models. We further suggest that a critical examination of high-performance AI systems can serve as a mirror through which we can reflect on our own intelligence. Many arguments against the prospects of AI may prove disconcerting for naively optimistic assessments of capacities of the human mind. (shrink)

This paper explores the challenges posed by the rapid advancement of artificial intelligence specifically Large Language Models (LLMs). I show that traditional linguistic theories and corpus studies are being outpaced by LLMs’ computational sophistication and low perplexity levels. In order to address these challenges, I suggest a focus on language as a cognitive tool shaped by embodied-environmental imperatives in the context of Agentive Cognitive Construction Grammar. To that end, I introduce an Embodied Human Language Model (EHLM), (...) inspired by Active Inference research, as a promising alternative that integrates sensory input, embodied representations, and adaptive strategies for contextualized analysis and conceptual utility maximization. By incorporating Active Inference, which sees perception as inferring the world's state from sensory data, the findings reveal that the characterization of the English modal be able to, as a triadic construction encoding biological intelligent agency, introduces a more plausible theoretical basis for the positing of linguistic constructions. This emphasizes the crucial role of embodied human language models in the comprehension of how humans construct preferred futures through language. (shrink)

The development and ubiquitous availability of large language model based systems (LLMs) poses a plurality of potentials and risks for education in schools and universities. In this paper, we provide an analysis and discussion of the overreliance concern as one specific risk: that students might fail to acquire important capacities, or be inhibited in the acquisition of these capacities, because they overly rely on LLMs. We use the distinction between global and local goals of education to guide our (...) investigation. In our view, the harm of LLM overreliance is specifically that it severs the connection between local educational goals (e.g., solving a specific math problem) and global educational goals (e.g., acquiring general skills for mathematics). Based on this analysis, we sketch three possible responses to the overreliance concern: preserving the educational goals while distinguishing admissible from inadmissible uses of LLMs (a conservative approach), changing global and local educational goals (a revisionary perspective), and preserving global educational goals while changing local goals (an in-between view). Since different types of responses have different benefits and weaknesses, we think that it is likely that an optimal response ultimately combines elements from all three types. (shrink)

Discussions of artificial intelligence have been shaped by two brilliant thought-experiments: Turing’s Imitation Test for thinking systems and Searle’s Chinese Room Argument. In many ways, debates about large language models (LLMs) struggle to move beyond these original, opposing thought-experiments. So, in this paper, I ask whether we can move debate forward by exploring the features Sceptics about LLM abilities take to ground meaning. Section 1 sketches the options, while Sections 2 and 3 explore the common requirement for a robust (...) relation between linguistic signs and external objects. Section 3 argues that concerns about worldly connections can be met and thus that outputs of LLMs should be viewed as genuinely meaningful. Section 4 then turns to the argument that the kind of derived meaning explored in Section 3 is insufficient for LLMs to count as meaningful systems per se, looking at the claim that they must possess original intentionality before we admit them to the space of meaning. I argue that this demand is not a prerequisite for meaning per se, but rather for some kinds of agency or conscious understanding. I suggest that the demands for original intentionality are not currently met by LLMs and that we should be cautious about whether we want them to be. (shrink)

Named entity recognition (NER) is a very relevant task for text information retrieval in natural language processing (NLP) problems. Most recent state-of-the-art NER methods require humans to annotate and provide useful data for model training. However, using human power to identify, circumscribe and label entities manually can be very expensive in terms of time, money, and effort. This paper investigates the use of prompt-based language models (OpenAI’s GPT-3) and weak supervision in the legal domain. We apply both (...) strategies as alternative approaches to the traditional human-based annotation method, relying on computer power instead human effort for labeling, and subsequently compare model performance between computer and human-generated data. We also introduce combinations of all three mentioned methods (prompt-based, weak supervision, and human annotation), aiming to find ways to maintain high model efficiency and low annotation costs. We showed that, despite human labeling still maintaining better overall performance results, the alternative strategies and their combinations presented themselves as valid options, displaying positive results and similar model scores at lower costs. Final results demonstrate preservation of human-trained models scores averaging 74.0% for GPT-3, 95.6% for weak supervision, 90.7% for GPT + weak supervision combination, and 83.9% for GPT + 30% human-labeling combination. (shrink)

Many philosophers have argued that large language models (LLMs) subvert the traditional undergraduate philosophy paper. For the enthusiastic, LLMs merely subvert the traditional idea that students ought to write philosophy papers “entirely on their own.” For the more pessimistic, LLMs merely facilitate plagiarism. We believe that these controversies neglect a more basic crisis. We argue that, because one can, with minimal philosophical effort, use LLMs to produce outputs that at least “look like” good papers, many students will complete paper (...) assignments in a way that fails to develop their philosophical abilities. We argue that this problem exists even if students can produce better papers with AI and even if instructors can detect AI-generated content with decent reliability. But LLMs also create a pedagogical opportunity. We propose that instructors shift the emphasis of their assignments from philosophy papers to “LLM dialogues”: philosophical conversations between the student and an LLM. We describe our experience with using these types of assignments over the past several semesters. We argue that, far from undermining quality philosophical instruction, LLMs allow us to teach philosophy more effectively than was possible before. (shrink)

Large Language Models (LLMs) are generative AI systems capable of producing original texts based on inputs about topic and style provided in the form of prompts or questions. The introduction of the outputs of these systems into human discursive practices poses unprecedented moral and political questions. The article articulates an analysis of the moral status of these systems and their interactions with human interlocutors based on the Habermasian theory of communicative action. The analysis explores, among other things, Habermas's inquiries (...) into the analogy between human minds and computers, and into the status of atypical participants in the linguistic community such as genetically modified subjects and animals. Major conclusions are the LLMs seem to qualify as authors that originally participate in discursive practices but do display only a structurally derivative form of communicative competence and fail to meet the status of communicative agents. In this sense, while the contribution of AI writing systems in public discourse and deliberation can support the process of mutual understanding within the community of speakers, the human actors involved in the development, use, and diffusion of these systems share a collective responsibility for the disclosure of AI authorship and verification and adjudication of validity claims. (shrink)

Natural language processing based on large language models (LLMs) is a booming field of AI research. After neural networks have proven to outperform humans in games and practical domains based on pattern recognition, we might stand now at a road junction where artificial entities might eventually enter the realm of human communication. However, this comes with serious risks. Due to the inherent limitations regarding the reliability of neural networks, overreliance on LLMs can have disruptive consequences. Since it will (...) be increasingly difficult to distinguish between human-written and machine-generated text, one is confronted with new ethical challenges. This begins with the no longer undoubtedly verifiable human authorship and continues with various types of fraud, such as a new form of plagiarism. This also concerns the violation of privacy rights, the possibility of circulating counterfeits of humans, and, last but not least, it makes a massive spread of misinformation possible. (shrink)