Allen Institute Asks "Can You Make An AI Smarter Than An 8th Grader"
Written by MIke James   
Wednesday, 14 October 2015

Of course, the answer is "no, not yet". However, the difficulty of the task is lowered a bit by the restriction to multiple choice questions from a standardized 8th grade science exam. 

 

Allen1

In case you missed it, the Allen Institute for Artificial Intelligence (AI2) is the creation of Paul Allen of Microsoft co-founder fame. While Bill Gates gives money away to cure the world's disease and poverty, Paul Allen has devoted a chunk of his money to "AI for the Common Good". 

The Institute has been working on a number of projects and Project Aristo is one of them. It is an "old school" symbolic information processor which gathers information and attempt to answer multiple choice science questions. What I mean by "old school" is that no neural networks were harmed during the building of the system. Instead a knowledge processing approach is used: 

  • Natural language parsing and processing of a variety of knowledge sources into a structured, computable knowledge base

  • Question understanding; transforming questions, diagrams, and answer options from exams into computable input

  • Integration with diagram processing systems to go beyond text to successfully understand exam questions

  • An array of solvers using various statistical and inference techniques to either select or generate the correct answer

A typical session with Aristo is something like:

Growing thicker fur in the winter helps some animals to

A) Hide from danger

B) Attract a mate

C) Find food

D) Keep warm

to which Aristo answers:

D) Keep warm. 

 

Artiso does reasonably well but AI2 wants it to do better so it has announced a Kaggle contest with $50,000 as the first prize for the system that gets most questions right - plus $20,000 for second place and $10,000 for third. 

The competition - The Allen AI Science Challenge - provides some training data, a validation data set and a sample submission. The training set has 2,500 questions and the validation set has 8,132 questions. The final test will use a data set of 21,298 questions. 

The really tough condition is:

"In order to be eligible for prizes, submitted models must run standalone without requiring internet connectivity (e.g., no use of runtime web searches or web APIs)."

Even IBM's Watson, the most successful question old school question answering system, couldn't get by with the occasional look at Wikipedia. 

Notice that the contest doesn't imply that you have to make use of anything similar to Aristo's knowledge processing. Some competitors are clearly thinking about using neural networks, although in this case the small size of the training set will make it difficult to find enough regularities to make it workable.

It will be interesting to see how it all ends up. 

The competition is open now and ends on 13th of Feburary 2016. Currenly there are 106 teams working on the problem. 

Allenicon

Banner


Call For Code 2024 Focuses On Generative AI
01/03/2024

This year's Call for Code challenge has been launched with a theme of the use of generative AI technology for solutions that aim to improve equitable access to resources and opportunities for historic [ ... ]



Master Large Language Model Ops
20/03/2024

New technology brings with it more career opportunities. You may never have imagined becoming an LLMOps consultant,  but there's now a Coursera Specialization which provides preparation for this  [ ... ]


More News

 

raspberry pi books

 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Thursday, 15 October 2015 )