A case could be made that modern LLMs come very close to passing the Turing test. Unfortunately Turing’s test only makes clear whether a system is good enough at mimicking human language, not whether it actually commands the language. It is in fact more like an advanced parrot test.
Turing was a genius mathematisian and computer scientist but his understanding of human nature and language was not necessarily at the same level. The Turing test can be argued to be built on a quite superficial analysis of what it means to be human like. If you read you the original paper you will see that the test measures performance in the so called “imitation game.” To make a long story short the test actually measures the ability to deceive and passing it means the computer is equal to humans in their ability to deceive. Coming from a counter intelligence professional, such as what Turing was at the time, that makes sense, but is that really the quintessence of being human; you are able to deceive.
A more interesting test would look to another contemporary, though no less known for his humaneness, he did spent most of his life pondering what language is and how humans use it, enter Ludwig Wittgenstein. Rather than Turing’s decption based test, It would be more interesting to propose a “Wittgenstein test” for AI to show human abilities since language is a key component of human nature. That would test whether a system can really be said to be a competent linguistic entity or merely emulating superficial language characteristics. Holding up the performance of LLMs against Wittgenstein’s understanding of language would provide a more profound base for assessing the “human likeness” of a system than Turing’s. Such a test would be better at assessing whether AI could be said to have Artificial General Intelligence or Superhuman Intelligence.
In many ways LLMs seem to implement Wittgenstein’s philosophy of language. There are certain striking resemblances but also significant divergence, which might help explain certain “features/bugs” of LLMs such as it’s mendacity and hallucinatory character. It might also help us set expectations and conceptually align to what LLMs are and are not and more importantly what they can and can not be. So let us compare LLMs against Wittgenstein’s understanding of language.
One of Wittgenstein’s profound insights was that the meaning of a word lies in how it is used in language. The meaning of a word does not come from a representation of deeper convictions or models of the world, they arise from how they are used in practice.
In LLMs the equivalent of Wittgenstein’s concept of meaning in use is that they determine the meaning by looking at statistical patterns of co-occurence, or quite literally, their use in the training material. This statistical co-occurrence can be said to be the meaning of a word in an LLM.
Words occur in sequence and follow rules, something Wittgenstein called a language game. According to Wittgenstein a language game is a shared activity that defines the rules of how they are used. In an LLM this feature is represented by contextual prediction tokens. LLMs try to predict the next most probable token in a sequence, which is similar to how words are used in language games.
Another insight of Wittgenstein is that words do not have a single essence. Words are used in similar but overlapping senses depending on the language game in which they appear, they are related to what he termed family resemblances. In an LLM this would be characterized by vector embeddings. Words with similar usage have numerically similar multidimensional vectors.
In this sense LLMs can be seen to implement main insights of Wittgenstein’s philosophy of language, but there is one major difference. LLMs may play the game brilliantly but it lacks the inclusion in a form of life. “The speaking of language is part of an activity, or of a form of life” (Philosophical Investigations §23). It is not included in life, the interactions with others and the world. That is why it has no sense of pain, no sense of frustration or hunger.
Consequently, LLMs have no understanding of the truth. When there is no activity that couples it to the world there is no way for it to learn that something is true and something is not. When humans hallucinate that they can fly they quickly learn the truth (spoiler alert: they cannot), and they hit the ground. LLMs are unconstrained by such activities and that is why hallucinations and mendaciousness are inherent properties that will never go away. They can be patched up by tightening the language games it can play but such guardrails and prompt engineering will always be superficial and never address the underlying issue that they are not part of any form of life.
LLMs can superficially mimic playing the game but they are not. As long as it they are not coupled to the real world and it’s actions not embedded it will never be able to truly master language. So, until LLMs are somehow embedded in the world in a form of life they will not be able to pass the Wittgenstein test and will continue to hallucinate and lie.
Photo by Christian Lendl on Unsplash
English 
Danish