Abstract

Wittgenstein’s ideas are a common ground for developers of Natural Language Processing (NLP) systems and linguists working on Language Acquisition and Mastery (LAM) models (Mills 1993; Lowney, Levy, Meroney and Gayler 2020; Skelac and Jandrić 2020). In recent years, we have witnessed a fast development of NLP systems capable of performing tasks as never before. NLP and LAM have been implemented based on deep learning neural networks, which learn concepts representation from rough data, but are nonetheless very effective in tasks such as question answering, textual entailment, and translation (Devlin et al. 2019; Kitaev, Cao and Klein 2019; Wang et al. 2019). In this paper, I will debate some Wittgensteinian concepts that impact the architectures of many NLP deep-learning systems. I will focus, in particular, on the attempt to build a specific kind of architecture to model a private language. The discussion, I think, helps extract philosophical assumptions leading the research and development of AI systems capable of language modelling. In this paper, I will address some of the main features of NLP systems2 used for word embedding and one proposal to manipulate through a neural network a form of private language (Lawney et al. 2020). In §2, I will reconstruct the complex path of the private language argument (PLA). In §3, I will discuss connectivist language models and introduce notions about NPL systems' architecture. An overview of this kind of model is helpful to introduce the work of Lowney, Levy, Meroney and Gayler (2020). They submit that their model can respond to the issues raised by Wittgenstein in the notorious Private Language Argument (PLA). This argument unexpectedly turned out to be relevant not only for the philosophy of language but also for NLP and LAM modellers. I will describe the language game concept in NLP, how it is embedded, and its role in inductive systems development. This central concept in Wittgenstein’s work is relevant to describe the role of context in understanding the meanings of words. In §4, I present the Wittgensteinian main concepts in play in the connectivist paradigm. I argue that the connectionist theoretical framework can better catch the dependency of word meaning on context. There seems to be a correlation between Wittgenstein's invitation to look, an invitation to dismiss the aim of theorizing about languages, and the absence of theory-ladenness in deep learning technologies involved in NLP. In §5, I criticize Lowney and colleagues’ claim, whose model does not successfully capture Wittgenstein’s Beetle in the Box case. Moreover, I argue that even if we can distinguish a strong and a weak definition of a private language, Wittgenstein’s argument also holds for deep-learning models, and his worries are still a good guide for NLP developers.