In May 2024, two researchers at UC San Diego published a paper with a blunt title: “People cannot distinguish GPT-4 from a human in a Turing test”. It was the first robust evidence that any machine had passed an interactive version of the test that has stood as a benchmark for machine intelligence since 1950.
Ten months later, the same team went further. In a stricter version of the test, closer to the one Alan Turing actually described, GPT-4.5 was picked as the human more often than the real human it was being compared with (Jones and Bergen, 2025).
Here’s what the Turing test is, what the two studies found, what gave the machines away, and what it means for anyone whose customers talk to them in writing. There are three figures, and one of them is a game.
What is the Turing test?
A test of whether a machine can hold a conversation well enough to pass as a person. Alan Turing, best known for his codebreaking work at Bletchley Park during the Second World War, proposed it in his 1950 paper “Computing Machinery and Intelligence”.
He called it the imitation game. An interrogator exchanges typed messages with two unseen players, one human and one machine, and has to say which is which. Turing’s point was that “can machines think?” is too vague to answer, while “can a machine play this game well?” is not.
He also made a prediction: that in about fifty years, computers would play the game so well that “an average interrogator will not have more than 70 per cent chance of making the right identification after five minutes of questioning”. The five minutes matter. Both of the recent studies used exactly that length of conversation.
Where does the test turn up in fiction?
Everywhere. I grew up on a diet of science fiction and have seen the Turing test referenced in many works. Ridley Scott’s Blade Runner (1982) has the Voight-Kampff test, which measures emotional responses to tell humans from replicants. Alex Garland’s Ex Machina (2014) is built around a modern Turing test. HBO’s Westworld and Spike Jonze’s Her (2013) both play with the line between human and machine. And Isaac Asimov’s The Bicentennial Man (1976, filmed in 1999) and William Gibson’s Neuromancer (1984) explore machine intelligence and self-awareness.
What did the 2024 study do?
Cameron Jones and Benjamin Bergen, cognitive scientists at UC San Diego, recruited 500 people online and split them into groups. Some acted as human witnesses. The rest were interrogators, who each had one five-minute text conversation and then said whether they thought they had been talking to a human or an AI.
The witnesses were real people, GPT-4, GPT-3.5, and ELIZA, a simple chatbot from the 1960s included as a baseline. The two GPT models were given a prompt that told them to play a young person who was concise, didn’t take the game too seriously, used slang and made the occasional spelling mistake.
The results:
- GPT-4 was judged to be human 54% of the time.
- GPT-3.5, 50%.
- ELIZA, 22%.
- Real humans, only 67%.
Statistically, interrogators were no better than chance at spotting GPT-4. Notice the last line, too: a third of the time, people decided a real person was a machine.
How often each witness was judged human
What gave the machines away?
Very little that looks like intelligence. The researchers classified what interrogators asked and the reasons they gave.
Most of them made small talk. When they explained their verdicts, the most common reasons were about style: spelling, grammar, capital letters and tone. Socio-emotional cues came next. Knowledge and reasoning hardly featured. The authors concluded that stylistic and socio-emotional factors play a bigger part in passing the Turing test than traditional ideas of intelligence.
What interrogators did, and what worked
The approaches that worked best were the ones people used least: asking about human experiences, or setting a logic or maths question.
Could you tell?
Try it. Each round has a question and two replies. One is from a human and one is from a machine. Pick the human.
Spot the human
Interrogator
What happened next?
In March 2025, Jones and Bergen ran the harder, three-player version. Each interrogator chatted with a human and a machine at the same time, for five minutes, then picked the one they thought was human. They ran it twice, with 126 UC San Diego students and then 158 people recruited online, over eight rounds each.
With a persona prompt, GPT-4.5 was picked as the human 73% of the time: significantly more often than the real person it was up against. Meta’s LLaMa-3.1, with the same prompt, was picked 56% of the time. Without the persona, GPT-4.5 dropped to 36%, and the baselines, GPT-4o and ELIZA, scored 21% and 23%. The authors called it the first empirical evidence that any artificial system passes a standard three-party Turing test.
Two details stand out. The persona did most of the work: the same model with a plain instruction to seem human was far easier to spot. And the old 1960s chatbot, ELIZA, still fooled people almost a quarter of the time.
Does this mean AI is intelligent?
Not on its own. The test measures whether a machine can pass for a person in a short, text-only chat, and the studies show that comes down mostly to style. Five minutes is not long. A prompt telling the model who to be makes a large difference. And the authors themselves frame their results as raising questions about what kind of intelligence large language models show, rather than settling them.
What the studies do settle is a practical point. In short written conversations, many people can no longer tell whether they are talking to a person.
What does it mean for business?
The 2024 paper put it plainly: deception by current AI systems may go undetected. That has a few consequences.
- Disclosure is becoming law. Under Article 50 of the EU AI Act, which applies from 2 August 2026, providers must make sure people are told when they are interacting with an AI system, unless it is obvious. A chatbot that passes for a person is exactly the case the rule is written for.
- Trust becomes a selling point. If customers can’t tell, the companies that tell them anyway will stand out.
- Writing alone no longer proves a human is there. Sales emails, support chats and even job applications can all be written by machines that pass for people. Processes that rely on “a person must have written this” need another check.
- For the language industry, the same shift is happening in speech and video. Machines that sound human in another language raise the same questions about consent, disclosure and who is accountable when they get it wrong. I’ve written about that in YouTube’s live auto-dubbing and in Nobody asked the language industry.
Final thoughts
Turing predicted that by around the year 2000 an average interrogator would have no better than a 70% chance of spotting the machine after five minutes. He was out by about a quarter of a century, and in the 2025 study the interrogators did far worse than that.
When I imagined a future where we had passed the Turing test, I thought something amazing would happen. Yet here we are, seemingly in the eye of the storm, awaiting the full impact of these advances. History will judge how we navigate this moment.
Sources: Jones, C.R. and Bergen, B.K. (2024) “People cannot distinguish GPT-4 from a human in a Turing test”, arXiv:2405.08007; Jones, C.R. and Bergen, B.K. (2025) “Large Language Models Pass the Turing Test”, arXiv:2503.23674; Turing, A.M. (1950) “Computing Machinery and Intelligence”, Mind, 59, pp. 433–460; Wikipedia, ELIZA; EU AI Act Explorer, Article 50: Transparency obligations.