It’s not making Turing test obsolete. It was obvious from day 1 that Turing test is not an intelligence test. You could simply create a sufficiently big dictionary of “if human says X respond with Y” and it would fool any person that its talking with a human with 0 intelligence behind it. Turing test was always about checking how good a program is at chatting. If you want to test something else you have to come up with other test. If you want to test chat bots you will still use Turing test.
Sounds to me like that sufficiently large dictionary would be intelligent. Like, a dictionary that can produce the correct response to every thing said sounds like a system that can produce the correct response to any thing said. Like, that system could advise you on your career or invent machines or whatever.
No, a dictionary is not intelligent. A dictionary simply matches one text to another. A HashMap is not intelligent. But it can fool a human that it is.
Yes, but you could argue that human brain is a large pattern matcher with a dictionary. What separates human intelligence from machine intelligence?
The question is not if something is a patter matcher or not. The question is how this matching is done. There are ways we consider intelligent and ways that are not. Human brain is generally considered intelligent, some algorithms using heuristics or machine learning would be considered artificial intelligence, a hash map matching string A to string B is not in any way intelligent. But all this methods can produce the same results so it’s impossible to determine if something is intelligent or not without looking inside the black box.
Yet language and abstraction are the core of intelligence. You cannot have intelligence without 2 way communication, and if anything, your brain contains exactly that dictionary you describe. Ask any verbal autistic person, and 90% of their conversations are scripted to a fault. However, there’s another component to intelligence that the Turing Test just scrapes against. I’m not philosophical enough to identify it, but it seems like the turing test is looking for lightning by listening for rumbling that might mean thunder.
If you want to get philosophical the truth it we don’t know what intelligence is and there’s no way to identify it in a black box. We may say that something behaves intelligently or not but we will never be able say if it’s really intelligent. Turing test check if a program is able to chat intelligently. We can come up with a test for solving math intelligently or driving car intelligently but we will never have a test for what most people understand as intelligence.
So would a book could be considered intelligent if it was large enough to contain the answer to any possible question? Or maybe the search tool that simply matches your input to the output the book provides, would that be intelligence?
To me, something can’t be considered intelligent if it lacks the ability to learn.
I disagree with the “limitations” they ascribe to the Turing test - if anything, they’re implementation issues. For example:
For instance, any of the games played during the test are imitation games designed to test whether or not a machine can imitate a human. The evaluators make decisions solely based on the language or tone of messages they receive.
There’s absolutely no reason why the evaluators shouldn’t take the content of the messages into account, and use it to judge the reasoning ability of whoever they’re chatting with.
Ironically GPT4 fails the turing test for having so wide knowledge about almost everything that you just know it’s not a human you’re talking to.
The idea that “a computer would deserve to be called intelligent if it could deceive a human into believing that it was human” was already obsolete 50 years ago with ELIZA. Clever though it was, examining the source code made it clear that it did not deserve to be called intelligent any more than does today’s average toaster.
And then more recently, the ever-evolving chatbots have made it increasingly difficult to administer a meaningful Turing test over the past 30 years as well. It requires care and expertise. It can’t be automated, and it can’t be done by the average person who hasn’t been specifically trained in it. They’re much better at fooling people who’ve never talked to one before, but I think someone with lots of practice identifying the bots of 2013 would still have not much trouble catching out those of today.
The problem with AI is that it does not understand anything. You can have a completely reasonable sounding conversation that is just full of stupidity and the AI does not know it because it does not no anything.
Another AI issue is it works until it does not and that failure can be rather severe and unexpected. Again because the AI knows nothing.
Seems like we need some test to address this. They are basically the same problem. Or maybe it is some training so that the AI can know what it does not know.
Define “understand” as you’re using it here? What exactly does the AI not do, that humans do, that comprises “understanding”?
Understanding the general sanity of some of their responses. Synthesizing new ideas. Having a larger context. AI tends to be idiot savants on one hand and really mediocre on the other.
You could argue that this is just a reflection of lack of training and scale but I wonder.
You will change my mind when I have had a machine interaction where the machine does not seem like an idiot.
Edit: AI people call the worst of these hallucinations but they are just nonsensical stuff that proves AI knows nothing and are just dumb correlation engines.
The point of logic is to carry you when your emotions try to stop you from thinking.
Yes AI is scary. No, that doesn’t mean we get to through out our definition of AI in order to avoid recognizing its presence.
I’m reminded of the apocryphal Ghandi quote “first they ignore you, then they laugh at you, then they fight you, then you win.” It seems like the general zeitgeist is in between the laugh/fight stages for AI right now.
🤖 I’m a bot that provides automatic summaries for articles:
Click here to see the summary
To try to answer this question, a team of researchers has proposed a novel framework that works like a psychological study for software.
This is why the Turing Test may no longer be relevant, and there is a need for new evaluation methods that could effectively assess the intelligence of machines, according to the researchers.
During the Turing Test, evaluators play different games involving text-based communications with real humans and AI programs (machines or chatbots).
The same applies to AI as well, according to a study from Stanford University which suggests that machines that could self-reflect are more practical for human use.
“AI agents that can leverage prior experience and adapt well by efficiently exploring new or changing environments will lead to much more adaptive, flexible technologies, from household robotics to personalized learning tools,” Nick Haber, an assistant professor from Stanford University who was not involved in the current study, said.
It doesn’t tell us anything about what a system can do or understand, anything about whether it has established complex inner monologues or can engage in planning over abstract time horizons, which is key to human intelligence,” Mustafa Suleyman, an AI expert and founder of DeepAI, told Bloomberg.
Saved 73% of original text.
