Neural Network pioneer dies
Neural Network pioneer dies
Saturday, 19 March 2011

David Rumelhart, one of the men generally credited with the re-discovery of the back-propagation method of training neural networks, has died, aged 68.

To the AI community, David Rumelhart is best known for presenting, together with Jay McClelland, the algorithm that created the resurgence of interest in neural networks in the early 1980s.

Before this the AI community had abandoned research on neural networks because of the analysis presented by Minsky and Papert that there were lots of very simple things that an artificial neuron couldn't learn. This work was misunderstood at the time to mean that it wasn't worth working on artificial neurons of any kind. Of course the truth is that it merely indicated that a singly layer of neurons was limited in what it could learn and it said nothing about multiple layers. 


David E. Rumelhart, 1942-2011


At the time a few researchers realised that multiple layers of artificial neurons could learn more, but in the absence of an effective training algorithm it was viewed as ad-hoc and something of a dead end. Rumelhart and McClelland took a more optimistic view and experimented with small simulated neural networks.

A training method that could be used with multi-layer networks had been described as early as 1969, but it was mostly ignored by the AI community still in the grip of Minksy and Papert's dismissal of artificial neurons.  It wasn't until the publication of Rumelhart's 1987 paper with Hinton and Williams in Nature that the idea took hold that training general multilayer networks was not only possible but powerful.

Next Rumelhart and McCelland's two-volume book Parallel Distributed Processing (PDP) provided an overall framework in which multilayer neural networks and the backprop algorithm made perfect sense. It also promoted the idea of "distributed coding" - where you seek a coding that isn't efficient in the usual senses but on which makes learning easier.  

The paper, the book and the work of the PDP group changed the way that AI was done and gave rise to the machine learning approach based on various developments on the artificial neural network still in use to day. Indeed many of the "big data" analytic methods and even hardware like the Kinect wouldn't be possible without the neural network approach.

Although many AI researchers think of Rumelhart as a computer scientist, he was a psychologist and much of his work was directed towards understanding how humans worked - especially models of cognition, language and perception. Indeed the David E. Rumelhart Prize, a $100,000 award given annually by the Glushko-Samuelson Foundation, is targeted at contributions to “theoretical foundations of human cognition.”

Writing about his colleague and friend Jay  McClelland explained:

"He became fascinated by the idea that our minds work at a sub-symbolic level."The idea was that thoughts emerge from neural activity. They're the consequence of the interaction of neurons. He went behind the scenes to look for the actual basis for our thinking ability."

David Rumelhart died on the 13th of March the cause was complications of Pick's disease, an Alzheimer's-like disorder. 

Further reading

Marvin Minsky

Neural networks



Margaret Hamilton Apollo and Beyond

Margaret Hamilton was lead developer for the team that wrote the programs that landed men on the moon and has devoted a career to error detection and recovery. She coined the term "software engin [ ... ]

Jay Forrester and Whirlwind

Jay Forrester, who invented core memory while working on Project Whirlwind at MIT in the 1940s, died on November 16, 2016 aged 98. Discover the man who invented core memory and, indirectly, gave the C [ ... ]

Other Articles




Last Updated ( Saturday, 19 March 2011 )

RSS feed of news items only
I Programmer News
Copyright © 2017 All Rights Reserved.
Joomla! is Free Software released under the GNU/GPL License.