Worm Balances A Pole On Its Tail
Written by Mike James   
Sunday, 10 December 2017

C. elegans is quickly becoming the mascot of AI. We have a complete circuit for its neurons and this has already resulted in some impressive achievements including building an artificial version of the worm's brain. Now we are moving into an era of using the circuits we have discovered creatively.

C.elegans is a tiny nematode worm, but at the moment it is very important. We have a complete map of its, admittedly simple, neural circuit. This has been used to implement the worm's brain in a number of ways and it has been surprising how well the artificial devices seem to behave like the worm. This is interesting, but the modeling of the individual neurons was less than realistic. More an artificial neural network wired up in the same way. Now a team of researchers, Mathias Lechner, Radu Grosu, and Ramin M. Hasani from TU Wien in Austria have created a more accurate model and re-purposed part of the neural network to do a different job.

The neurons were modeled using a differential equation for a Leaky-integrate and fire neuron - much more biologically plausible than the weighted sum type of neurons you find in an artificial neural network. The section of the network that was modelled was the "Tap withdrawal" TW circuit which causes the worm to move away when its tail is tapped.

tap

This was used to control an inverted pendulum - a standard AI task especially in reinforcement learning. An inverted pendulum is basically a hinged stick attached to a small cart. The cart can move to the left or the right and the task is to move it so as to balance the stick. This is similar enough to the tap withdrawal motion to allow the same neuronal circuit to do the job:

tap2

The circuit's structure was used but the parameters were adjusted by a search-based reinforcement learning algorithm. This seems to work:

"The final TW neural circuit achieves the maximum possible return of 1,000, which is equal to the performance of a PID-state-controller or an artificial neural net. However, as the position and velocity of the cart are either feed into the circuit or affect the reward directly, the controlled cart is subject to a small drift, that will make it reach one end of the movable space after the evaluate of the environment has ended"

You can see the circuit in action in the following short video:

 

You can also see the functioning of the neurons in the circuit as the cart moves.

This is interesting because it demonstrates that the structure of the neurons in the TW circuit are capable of a difficult control problem and all you have to do to make it work is adjust the parameters which control how sensitive the neurons are. Notice that the network didn't learn the task in the usual sense, but reinforcement learning was used to adjust it. Presumably the parameters could have been adjusted over time in an evolutionary manner.

tap3

It seems the structure in the network carries a lot of the functionality, which is another big difference between our current typical artificial neural networks and the TW.

More Information

Worm-level Control through Search-based Reinforcement Learning

Related Articles

A Worm's Mind In An Arduino Body

A Worm's Mind In A Lego Body

Neurokernel - A Fly's Brain

OpenWorm Building Life Cell By Cell        


To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on, Twitter, FacebookGoogle+ or Linkedin.

 

 

Banner


Neuromorphic Supercomputer Up and Running
04/11/2018

The University of Manchester's Faculty of Science and Engineering's SpiNNaker machine, made up of one million processors and designed to mimic the human brain was switched on for the first time on Nov [ ... ]



FSF Update Rules Commons Clause Non-Free
12/11/2018

The Free Software Foundation has added the Commons Clause to its list of non-free licenses among a number of recent updates to its licensing materials. Other changes clarify the GNU GPL position on tr [ ... ]


More News

 

Python

 



 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Sunday, 10 December 2017 )