The Turing test is a proposal for a test of a machine's capability to perform human-like conversation. Described by Alan Turing in the 1950 paper "Computing machinery and intelligence", it proceeds as follows: a human judge engages in a natural language conversation with two other parties, one a human and the other a machine; if the judge cannot reliably tell which is which, then the machine is said to pass the test. It is assumed that both the human and the machine try to appear human. In order to keep the test setting simple and universal (to explicitly test the linguistic capability of some machine), the conversation is usually limited to a text-only channel.
The test was inspired by a party game known as the "Imitation Game", in which a man and a woman go into separate rooms, and guests try to tell them apart by writing a series of questions and reading the typewritten answers sent back. In this game, both the man and the woman aim to convince the guests that they are the woman.
Turing originally proposed the test in order to replace the emotionally charged and (for him) meaningless question "Can machines think?" with a more well-defined one.
One interesting part of his proposed test was that the answers in conversation would have to be delivered at controlled intervals and rates. He believed this necessary to prevent the observer drawing a conclusion based on the fact the computer potentially would answer so much faster than the human operator, especially on mathematical questions.
Objections and replies
Turing himself suggested several objections which could be made to the test. Below are some of the objections and replies from the article in which Turing first proposed the test.
Theological Objection: This states that thinking is a function of man's immortal soul and therefore a machine could not think. Turing replies by saying that he sees no reason why it would not be possible for God to grant a computer a soul if he so wished.
Mathematical Objections: This objection uses mathematical theorems, such as Gödel's incompleteness theorem, to show that there are limits to what questions a computer system based on logic can answer. Turing suggests that humans are too often wrong themselves and pleased at the fallibility of a machine.
- Argument From Consciousness: This argument, suggested by Professor Jefferson Lister states, "not until a machine can write a sonnet or compose a concerto because of thoughts and emotions felt, and not by the chance fall of symbols, could we agree that machine equals brain". Turing replies by saying that we have no way of knowing that any individual other than ourselves experiences emotions, and that therefore we should accept the test.
- Lady Lovelace Objection: One of the most famous objections, it states that computers are incapable of originality. Turing replies that computers could still surprise humans, in particular where the consequences of different facts are not immediately recognizable.
- Informality of Behaviour: This argument states that any system governed by laws will be predictable and therefore not truly intelligent. Turing replies by stating that this is confusing laws of behaviour with general rules of conduct.
Extra-sensory perception: Turing seems to suggest that there is evidence for extra-sensory perception. However he feels that conditions could be created in which this would not affect the test and so may be disregarded.
Discussion of relevance
It has been argued that the Turing test can not serve as a valid definition of machine intelligence or "machine thinking" for at least three reasons:
- A machine passing the Turing test may be able to simulate human conversational behavior, but this may be much weaker than true intelligence. The machine might just follow some cleverly devised rules. (A common rebuttal in the AI community has been to ask: how do we know humans don't just follow some cleverly devised rules?) Two famous examples of this line of argument against the Turing test are John Searle's Chinese room argument and Ned Block's Blockhead argument.
- A machine may very well be intelligent without being able to chat like a human.
- Many humans that we'd probably want to consider intelligent might fail this test (e.g., the young or the illiterate). On the other hand, the intelligence of fellow humans is almost always tested exclusively based on their utterances.
Another potential problem, related to the first objection above, is that even if the Turing test is a good operational definition of intelligence, it may not indicate that the machine has consciousness, or that it has intentionality. Perhaps intelligence and consciousness, for example, are such that neither one necessarily implies the other. In that case, the Turing test might fail to capture one of the key differences between intelligent machines and intelligent people. Of course, machines passing the test would most likely vehemently disagree.
In the words of science popularizer Larry Gonick, "I personally disagree with this criterion, on the grounds that a simulation is not the real thing."
Predictions and tests
Turing predicted that machines would eventually be able to pass the test. In fact, he estimated that by the year 2000, machines with 109 bits (about 119MB) of memory would be able to fool 30% of human judges during a 5-minute test. He also predicted that people would then no longer consider the phrase "thinking machine" contradictory. He further predicted that machine learning would be an important part of building powerful machines, a claim which is considered to be plausible by contemporary researchers in Artificial intelligence.
By extrapolating Moore's Law over several decades, futurist Ray Kurzweil predicted that Turing-capable computers would be manufactured around the year 2020, roughly speaking. See the Moore's Law article and the references therein for discussions of the plausibility of this argument.
As of 2005, no computer has passed the Turing test as such. Simple conversational programs such as ELIZA have fooled people into believing they are talking to another human being, such as in an informal experiment termed AOLiza. However, such "successes" are not the same as a Turing Test. Most obviously, the human party in the conversation has no reason to suspect they are talking to anything other than a human, whereas in a real Turing test the questioner is actively trying to determine the nature of the entity they are chatting with. Documented cases are usually in environments such as Internet Relay Chat where conversation is sometimes stilted and meaningless, and in which no understanding of a conversation is necessary, are common. Additionally, many internet relay chat participants use English as a second or third language, thus making it even more likely that they would assume that an unintelligent comment by the conversational program is simply something they have misunderstood, and are also probably unfamiliar with the technology of "chat bots" and don't recognize the very non-human errors they make. See ELIZA effect.
The Loebner prize is an annual competition to determine the best Turing test competitors. Although they award an annual prize for the computer system that, in the judges' opinions, demonstrates the "most human" conversational behaviour (with A.L.I.C.E. being a recent winner multiple times, and learning AI Jabberwacky in second), they have an additional prize for a system that in their opinion passes a Turing test. This second prize has not yet been awarded.
There is an ongoing $10,000 bet at the Long Bets Foundation between Mitch Kapor and Ray Kurzweil about the question whether a computer will pass a Turing Test by the year 2029. The bet specifies the Turing Test in some detail.
In Turing's paper, the term "Imitation Game" is used for his proposed test as well as the party game for men and women. The name "Turing test" may have been invented, and was certainly publicized, by Arthur C. Clarke in the science-fiction novel 2001: A Space Odyssey (1968), where it is applied to the computer HAL 9000.
- Alan Turing, "Computing machinery and intelligence". Mind, vol. LIX, no. 236, October 1950, pp. 433-460. Online versions: , 
Larry Gonick, The Cartoon Guide to the Computer (1983, originally The Cartoon Guide to Computer Science). ISBN 0062730975.
Ray Kurzweil, The Age of Intelligent Machines (1990). ISBN 0262111267.
Roger Penrose, The Emperor's New Mind (1990). ISBN 0140145346.
Last updated: 05-21-2005 23:51:57