|Allen Institute Asks "Can You Make An AI Smarter Than An 8th Grader"|
|Written by MIke James|
|Wednesday, 14 October 2015|
Of course, the answer is "no, not yet". However, the difficulty of the task is lowered a bit by the restriction to multiple choice questions from a standardized 8th grade science exam.
In case you missed it, the Allen Institute for Artificial Intelligence (AI2) is the creation of Paul Allen of Microsoft co-founder fame. While Bill Gates gives money away to cure the world's disease and poverty, Paul Allen has devoted a chunk of his money to "AI for the Common Good".
The Institute has been working on a number of projects and Project Aristo is one of them. It is an "old school" symbolic information processor which gathers information and attempt to answer multiple choice science questions. What I mean by "old school" is that no neural networks were harmed during the building of the system. Instead a knowledge processing approach is used:
A typical session with Aristo is something like:
Growing thicker fur in the winter helps some animals to
A) Hide from danger
B) Attract a mate
C) Find food
D) Keep warm
to which Aristo answers:
D) Keep warm.
Artiso does reasonably well but AI2 wants it to do better so it has announced a Kaggle contest with $50,000 as the first prize for the system that gets most questions right - plus $20,000 for second place and $10,000 for third.
The competition - The Allen AI Science Challenge - provides some training data, a validation data set and a sample submission. The training set has 2,500 questions and the validation set has 8,132 questions. The final test will use a data set of 21,298 questions.
The really tough condition is:
"In order to be eligible for prizes, submitted models must run standalone without requiring internet connectivity (e.g., no use of runtime web searches or web APIs)."
Even IBM's Watson, the most successful question old school question answering system, couldn't get by with the occasional look at Wikipedia.
Notice that the contest doesn't imply that you have to make use of anything similar to Aristo's knowledge processing. Some competitors are clearly thinking about using neural networks, although in this case the small size of the training set will make it difficult to find enough regularities to make it workable.
It will be interesting to see how it all ends up.
The competition is open now and ends on 13th of Feburary 2016. Currenly there are 106 teams working on the problem.
or email your comment to: firstname.lastname@example.org
|Last Updated ( Thursday, 15 October 2015 )|