An artificial intelligence (AI) system passed an eighth grade science test, answering 90 percent of the questions correctly. The result was achieved by Aristo, an AI built by the Allen Institute for Artificial Intelligence in the United States, demonstrating once again the rapid progress in the field of artificial intelligence. In the future, Aristo could be adapted to other contexts, for example to produce automatic response systems and autonomously carry out conversations with humans.
In 2015, a competition offered a prize of 80 thousand dollars for those who managed to build an AI capable of passing an eighth grade science test (“8th grade” in the United States). More than 700 computer experts participated in the initiative, but none of them had managed to reach the goal: the most promising AIs had stopped at 60 percent correct answers. Just four years later, Aristo has achieved surprising results, demonstrating the acceleration that the AI sector has had in recent times, with increasingly refined and efficient systems.
At the Allen Institute – a research organization founded by Microsoft co-founder Paul Allen – they began working on Aristo in 2013, wondering if it was possible to emancipate AI from the classic tests they are subjected to to measure their abilities, usually board games and video games. They thought that school science tests were a good alternative, because – in addition to requiring the learning of specific notions – they require the application of logic to solve problems.
The development of Aristo started with BERT, a system created by Google researchers (and then made available to everyone) to read and interpret the texts, in order to use the information collected for other purposes. By submitting thousands of Wikipedia entries and novels, for example, BERT learns to guess the words removed from a sentence, making it meaningful again. In a short time, Google's system has gathered an enormous amount of information on how language works, a capability that can be exploited by others – such as researchers at the Allen Institute – to build their artificial intelligences.
Aristo was given thousands of science test questions and answers, which allowed him to learn its characteristics and ultimately to answer other similar questions independently. In a short time, Aristo demonstrated that he was able to correctly answer both simpler and more notional questions, and others in which it was necessary to use logic, such as:
What change would most likely lead to a reduction in the number of squirrels in a given area?
a. A reduction in the number of predators.
b. A reduction in competition among squirrels.
c. An increase in available food.
d. An increase in the number of forest fires.
The correct answer is 'd', because fires could not only kill squirrels directly, but could also reduce the amount of food available in the area they live in, making their survival more difficult. Aristo answered correctly, demonstrating that he knows how to apply logic to find the solution to the problem.
The results obtained by Aristo, who is named in honor of Aristotle, were also promising with the administration of science tests in the last year of high school. The AI answered 80 percent of the questions correctly, thus making a few more errors than the third grade test, but still getting a satisfactory result.
Advances in Aristo and other language-based AI systems are remarkable, but researchers and experts urge you not to get too excited. For now, the system developed by the Allen Institute knows how to perform a specific task very well, but it certainly does not have the ability to think – even in a more creative way – of real students. The tests also contained only textual questions, excluding those accompanied by images and graphics, because the introduction of image recognition and interpretation systems would have complicated the development of the algorithms too much.
However, Aristo is an interesting starting point and could be the basis for enriching services such as search engines, document filing systems and registers. In the next few years, software that will be able to perform automatic data storage and organization tasks, for example, is expected to go on sale. The ability to interpret language can also be used to improve systems for conversing with machines, such as those used in home auto attendants or on our smartphones.
Hundreds of research institutes around the world work on systems like Aristo and, in a short time, they have obtained results that only ten years ago seemed unattainable. The greater computing power of computers and the refinement of data cataloging and learning systems have made it possible to make great progress, reducing programming times.
