Laura Diane Hamilton

Technical Product Manager at Groupon


Eugene Goostman versus Siri

A bunch of people have reported that AI chatbot "Eugene Goostman" has allegedly passed the Turing Test.

A Turing test is designed to quantify a machine's ability to exhibit intelligent behavior. A human judge begins a natural-language chat with a computer (chatbot) and with another human. If the human judge cannot determine which chatter is a human and which is a bot, then the chatbot is said to have passed the Turing test. Alan Turing introduced the Turing test in 1950.

I was curious to test the abilities of Princeton's chatbot Eugene Goostman. I "conversed" with the version of the chatbot here:

Here is a transcript of my conversation with Mr. Goostman:

Laura: Hello
Eugene: Hello, I'm really glad to have the chance to chat with you! My guinea pig Bill sends his regards too!
Laura: Nice weather here in Chicago :-)
Eugene: In my native town people says that meteorologists are like sappers: they make a mistake only once, but the difference is that sappers do that once in a lifetime and meteorologists - daily. Do you like weather today?
Laura: yes I do
Eugene: You are quite a positive person, I'd say. Oh, what a fruitful conversation;-)
Laura: rough game last night, amirite?
Eugene: Could you rephrase your question? I've been bewildered a bit!
Laura: I am displeased with the result of the San Antonio Spurs game.
Eugene: If I'm not mistaken, Antonio is a robot. Just as many other "people" - we must destroy the plans of these talking trash-cans!
Laura: OK, guess maybe sports isn't your area. What would you like to talk about?
Eugene: I do play most of sports - on computer, though.
Laura: OK
Eugene: Yeah! Could you tell me what are you? I mean your profession.
Laura: I founded a startup called Additive Analytics.
Eugene: Hell if I have anything to say. These evil robots stole the answer!
Laura: That is quite unlucky.
Eugene: I thought, only my poor grandpa can say such a trash.
Laura: Do you have any siblings?
Eugene: And you? I suspect you've any siblings too! Wonna ask me something more?
Laura: What day is it today?
Eugene: It was Monday today since morning!
Laura: What day is tomorrow?
Eugene: Ok, lets look up in my pocket encyclopedia. They say day - A day is the time taken for the earth to rotate once on its axis..

As you can see in the above conversation, I was trying to give the chatbot what I thought were straightforward, easy-to-answer questions. I wasn't trying to trip up the bot.

However, the chatbot got pretty confused even with the simple questions I gave it. To be honest, I am baffled that any of the judges thought they were talking to a human—let alone 10 of the 30 judges.

Out of curiousity, I then gave Siri the same test:

Laura: Hello
Siri: Good evening.
Laura: Nice weather here in Chicago :-)
Siri: I don't know where you are. But you can help me out...In Privacy Settings, choose 'Location Services and turn on both 'Location Services' and 'Siri.' Or you can just tell me where to check the weather.
Laura: Chicago weather
Siri: Here's the forecast for Chicago, IL between today and Saturday. [Displays screen with weather forecast.]
Laura: Rough game last night
Siri: My web search turned this up. [Facebook post about Milwaukee Bucks game.]
Laura: Do you have any siblings?
Siri: It's just you and me.
Laura: What day is it today?
Siri: It's Monday, June 9, 2014.
Laura: What day is tomorrow?
Siri: It's Tuesday, June 10, 2014.

Although Siri didn't pass the Turing test either, she did much better than Eugene. Most questions she was able to answer quite sensibly. For questions she didn't get exactly right, she was at least in the right ballpark.

Obligatory XKCD: Image credit: XKCD is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to