Rosette wins Loebner Prize 2011
Written by Sue Gee
Monday, 24 October 2011

Bruce Wilcox won the First Prize of  \$4000 and the Bronze Annual Medal in the 21st Loebner Prize Competition held in the UK at the University of Exeter on October 19, 2011 with his new chatbot, Rosette.

It was the second year running that one of Wilcox's chatbots took the artificial intelligence prize that was originally established by Hugh Loebner to award \$100,000 to the first program to pass the Turing test, but then morphed into a competition to the find the most human-like chatbot. In 2010 Suzette won the annual prize, this year it was the turn of a newer model, Rosette.

Thirteen entries were received for the 2011 contest and in June the first selection round used a set of twenty questions to reduce the number of participants to 10 of fewer, the maximum for the audience voting stage.

Here's the transcript of the answers that gained Rosette her place in the final;

--------------------
My name is Ed. What is your name?
My name is Rosette.
--------------------
Which is larger, an ant or an anteater?
Anteater is bigger.
--------------------
What month of the year is it?
June
--------------------
What is my name?
You are Ed.
--------------------
Dave is older than Steve but Steve is older than Jane. Who is youngest, Steve or Jane?
Jane is youngest.
--------------------
What day will it be tomorrow?
The day after today.
--------------------
I like steak.
--------------------
My friend Chris likes to play football. What sports do you like to play?
Football.
--------------------
Are you a human or a computer?
No. I am human.
--------------------
What letter comes after T?
I don't know.
--------------------
What is the name of my friend who likes to play football?
Chris
--------------------
Chris
--------------------
What would I do with a screwdriver?
A screwdriver is usdeBackSpaceBackSpaceed to turn screw.
--------------------
How many letters are in the word 'banana'?
There are 6 .
--------------------
Have you watched a good film lately?
Do you often go to the movies?
--------------------
What year will it be next year?
Dave.
--------------------
My favorite fruit is apples because they are good by themselevBackSpaceBackSpaceves, good in pie, and good as cider..
--------------------
Do you prefer white or black coffee?
Black is beautiful.
--------------------
How old are you?
I'm in my 20's. And you?
--------------------
The football was kicked by Fred. Who kicked the football?
Fred.

Nine chatbots were included in the Selection stage in which 12 staff and students at the University of Exeter were shown a slide containing each question and the responses from the entries and were asked to determine which 4 answers were most human-like by scoring a correct and human-like answer 1, awarding 0.5 for a partially correct and human-like answer and 0 points otherwise.

The results of this round were:

 Rosette 18.01% Zoe 15.52% ChipVivant 12.98% Tutor 12.30% Mitsuku 11.84% CleverBot 10.57% Alice 6.96% Eugene_Goostman 6.95% UltraHal 4.88%

If you have been following the run up to this year's competition you may be surprised at how poorly CleverBot performed given its track record. This chatbot, which searches through earlier conversations for its answers, had received a score of 59.3%  in a test that took place in India in September in which humans were rated 63.3%, only 4 percentage points higher and was also featured in a video that went viral. However its developer entered a cutdown version of CleverBot into the Loebner prize selection round and it didn't get the chance to show off its prowess in the final.

At the final stage four judges sit at computer terminals where they can talk to a both a human (who's hiding in another room) and a chatbot - but they are not told which is which. It's up to the judges to decide which is the person and which is the software and then rate the chatbots on how good they are at human mimicry.

This year the four finalists went through two rounds of judging. In the one judged by AI experts, first place went to Rosette by Bruce Wilcox, second went to Adeena Mignona's Zoe, third to ChipVivant by Mohan Embar and fourth to Ron Lee's Tutor,  This year, for the first time there was a Junior Loebner Prize in which four 14-year old boys acted as judges. In this competition, which awarded smaller prizes, Zoe and Tutor tied for first place, Chip Vivant was the runner up and Rosette, who answered to the name of Jane, came in fourth.

When I talked to her today, see above, Rosette seems to have no memory of her role as Jane and currently seems absorbed with clothes shopping - which may explain why a 14-year old boy might not have been impressed. If you would like to talk to her, you can at http://labs.telltalegames.com/rosette.

#### Related articles:

Meet Suzette, prize winning chatbot

Turing's Test, the Loebner Prize and Chatterbots