Claude Shannon - Information Theory And More
Written by Historian   
Wednesday, 13 March 2013
Article Index
Claude Shannon - Information Theory And More
Bell Labs
The bit, the nit, the dit and the Hartley

Claude Shannon is a little-known character in the history of computing. Even those people who do know Shannon's name think of him as nothing but a theoretician - the inventor of something obscure called information theory.

The truth is that Shannon was an important pioneer of computing theory and practice and was interested in a wide range of subjects.

In an effort to increase his profile you might even want to use the tabloid headline "The Father of the Bit", but you could equally well call him "the man who invented computer chess". However, both might be going just a little too far.



Claude Shannon (1916-2001)

The Father of the Bit

Claude Shannon is quite correctly described as a mathematician. He gained his PhD from MIT in the subject, but he made substantial contributions to the theory and practice of computing.

When Shannon was a student electronic computers didn't exist. There were a few mechanical analog computers that could be used to calculate trajectories and tide tables, but nothing that could be described as a digital computer.

Of course, Babbage had described the basic design of a stored program computer in the 1800s and there were mechanical calculating machines, but these were decimal devices. They performed arithmetic by using gear wheels marked up 0 to 9 and if you watched them at work you would have seen a mechanical copy of the way we humans do arithmetic. That is, there is a unit wheel, a tens wheel, hundreds wheel and so on, with each cog having ten possible states.

We may think now that a binary computer is the most obvious way of doing things, but it isn't as long as you are working with cogs that can have any number of positions. Of course, to make the transition to electronic computers it was vital to see that the decimal system had to be given up, and this is where Shannon comes in.




Understanding the differential analyzer

In 1930 Vannevar Bush built the most advanced mechanical analog computer of the day, possibly ever, at MIT. It was a technically a differential analyzer and it could solve differential equations but it was different in two respects.

The first is that it was general purpose in the sense that you could reconfigure it to solve any equation and it used some electronic components - valves and relays.



The differential analyser with Bush the closest figure.


Vannevar Bush is an important figure in the history of computing in his own right and he developed the idea into a fully electronic device. Shannon was a student of Bush and worked the differential analyzer to earn a few extra dollars. Relays were used configure the machine and these circuits had been designed in a fairly ad-hoc manner.

Bush suggested to Shannon that, being a mathematician, he might be able to work out what the principles were that lay behind such circuitry because it was obvious that there was some organizing principle. Shannon certainly saw what the organizing principle was and described if fully in a paper with the catchy title "Symbolic Analysis of Relay and Switching Circuits".




What Shannon had discovered was the relationship between a relay being on or off and the Boolean logic states true and false. He also realized the connection with binary arithmetic by identifying on and off with 1 and 0.

Shannon also demonstrated how relays could be used to build circuits that add, subtract, multiply and indeed do any arithmetic you wanted, all in binary.

Shannon's paper was very complete, almost as if he had attempted to write down the theory of the entire subject in one go. This desire for completeness seems to have been a characteristic of Shannon's personality as we shall see.

Of course, Shannon hadn't shown how to use relays to build a general purpose programmable computer, but only how to build some of the modules that are part of every computer - adders and logic gates. What was more important about Shannon's work was that it pointed the way to an electronic computer based on binary by showing that arithmetic circuits were much easier to design once you got rid of the decimal system.

Surprisingly, even though Shannon and many others demonstrated beyond any doubt that the binary system was best, nearly all of the first early electronic computers used the decimal system. You have to see this not as ignorance or stupidity, but a mark of just how deeply embedded the decimal system was in the minds of the engineers and scientists of the time.




Last Updated ( Sunday, 17 March 2013 )

RSS feed of all content
I Programmer - full contents
Copyright © 2015 All Rights Reserved.
Joomla! is Free Software released under the GNU/GPL License.