|Hollerith Census Machine - A Milestone In Big Data Big|
|Written by Historian|
|Tuesday, 16 August 2016|
August 16, 1890 was the date on which the US Census Bureau announced that the US population was 62,662,250. It was the first time that machinery was used to automate the process and the use of Hollerith Machines saved $5 million or more.
The US census began in August 1790. The first took nine months' of data collection and took about a year to count. Every ten years after that the US repeated the census, but the size of the population was growing at 35% per annum and by 1860 there were over 31 million people to count.
Over the years the census had also grown in complexity - so much so that in 1850 the congress passed a law to limit it to no more than 100 questions! It was beginning to look as it might take more than 10 years to process the data and what good would 10 year-old statistics have been. Modern government needed up-to-date information to plan services and predict taxation.
(February 29, 1848 - November 17, 1929)
It was rather by chance that Herman Hollerith, who had graduated from the Columbia School of Mines with a degree in mining engineering, became the person who would revolutionize the way the census data was coded and counted. He was dating Kate Sherman Billings, whose father was in charge of a large section of the census - vital statistics. Hollerith tried to impress Kate by trying to buy every one of the tickets for a lottery at the party they were both at. Unfortunately he missed just one ticket - the winning ticket! At the buffet he chose the chicken salad and this gave Kate the opportunity to invite him home where her mother made a great chicken salad!
At the Billings' home Hollerith got into a deep conversation with Kate's father about the census problem. Billings was sure that a mechanical method of counting could be built but he was a physician and not an engineer. Later Hollerith reported their conversation:
"He said to me that there ought to be a machine for doing the purely mechanical work of tabulating population statistics... He thought of using cards with the description of the individual shown by notches in the edge... After studying the problem I went back to Dr Billings and said that I could work out a solution and asked if he would go in with me. The Doctor said he was not interested any further than to see some solution worked out."
Hollerith was always generous in attributing the basic idea of using punch cards to Billings and even offered "Chicken Salad" as the two-word explanation of why he invented automatic data processing!
In his spare time Hollerith worked on a census machine and patented it in 1884. He took a part-working machine to the census office but before they put money into the project they wanted a working demonstration so Hollerith borrowed $2500 from his brother and became the first of the "garage" startups in the history of computing!
He had considered all sorts of ways of recording the data so that it could be processed including paper tape, an idea he abandoned because he realised that the tape format made it difficult to get at individual records. Hollerith recalled seeing a railroad ticket collector using a "punch photograph" a ticket that was punched to indicate what the passenger looked like - light, hair dark, large nose, etc - and he decided that punch cards were the thing to use.
At first his punch cards only had holes around the edge because his punch couldn't reach the centre but it didn't take long for him to solve this problem. He had re-invented the punch card of Babbage and the Jacquard loom but it is not clear if he knew of these earlier efforts. As his brother was in the silk business it is possible that he heard about card controlled looms from him.
An early Hollerith card
Hollerith looked around for ways of proving that his machines worked and so he offered to automate the Baltimore health records. Hollerith did much of the punching of the cards himself, a not inconsiderable feat using a manual punch at 1000 cards per day each with ten or more holes. Each card contained the data for one patient and once punched his tabulating and sorting machines made it possible to get answers to questions that previously seemed impossible.
His next project was the automation of health records kept by the War Department. They were also willing to rent his machines but in this case the problem was more difficult and needed more data stored on each card. Instead of simply increasing the size of the card Hollerith decided to allow combinations of holes to mean something. This was of course the start of the use of coding that would eventually lead to binary and ASCII codes.
Hollerith's eye was firmly on the coming census of 1890. He was offered a job with the census office but he turned it down. He wanted to sell his machines, not be part of the census and took part in a contest was arranged between Holleriths punch cards and two competing systems - one using coloured slips of paper, the other using poker chips. The test involved the data for 10,000 people and Hollerith's machines finished the job in just over 3 days. Most of the time was spent punching the cards, the actual processing took just 5 hours - the second fastest method took 4.5 days.
As a result the census office ordered 56 machines at a cost of $56,000 per annum and the 1890 census was completed within a year which has to be compared to the eight years needed for the 1880 census.
or email your comment to: firstname.lastname@example.org
|Last Updated ( Tuesday, 16 August 2016 )|