Abstract

This article addresses the issue of attributing phenomenal consciousness to Artificial Intelligence (AI), a mistake that can lead to ethically dangerous consequences and that is becoming widespread due to the advances of Large Language Models such as ChatGPT. We juxtapose advancements in AI with the notion of inner experience as it is present in humans. The study draws from various disciplines, including philosophy of mind, artificial intelligence, and theological texts such as "The Inner Castle" by Saint Theresa of Ávila and "Confessions" by Saint Augustine. Firstly, it reviews the current state of the relationship between phenomenal consciousness and AI, followed by a critique of the idea that advanced language models, like ChatGPT, can achieve an inner experience in the same sense we use the term to describe the human inner experience. A common objection is raised, suggesting that AI can become conscious by increasing its complexity. This is countered by presenting theoretical and empirical evidence on the independence of computational intelligence and phenomenal consciousness. The study concludes that, despite AI's notable cognitive abilities, it lacks the inner experience that characterizes human experience. Then, our second main contribution is an analogy of the dwellings of the Inner Castle to the range of different subjective experiences that are available to human beings, together with actions associated to them, which can be useful to understand where the machine can perform tasks that are similar to the human and where subjective experience is key. We are now in a pivotal moment where it is essential to understand the limitations of AI for deploying it ethically.