Search engines are black boxes. They need to be because any knowledge of how they work will be used against them to bias their results to somebody's financial, political or social reward. It's happened in the past - keyword stuffing and link farms are perhaps the best examples. But what does this leave us poachers with? Currently the gamekeeper really does own the game. It means the advice that goes by the name of Search Engine Optimisation (SEO) is really nothing more than a collection of folklore sayings and proverbs. We need a solid science to found the whole subject on, not just hearsay and the extracted wisdom of experience. It's time to organise.
When you say it like this it becomes obvious that what SEO lacks is a scientific basis and the means to acquire one is the same for any wannabe science - we need some experimental data. We need Experimental SEO (ESEO) - or Googlenomics if you need a catchier and more marketable name.
It really doesn't matter what you call it the need is great. With ESEO we can at last fix the laws of the subject and move on from folklore and snake oil to facts and certainties. Of course doing so changes the game and ESEO isn't a one off attack - it has to be a sustained chase of the quarry by whatever hounds seem appropriate at the time.
So what is ESEO?
How do you go about it?
The answer is obvious because it is just the application of scientific method to a new subject matter. All experiments work by isolating what they are trying to study in an environment where a hypothesis can be tested. Ok we all believe that keywords are no longer important and the keyword here is "believe". We can convert this belief into a fact by setting up an experiment. First prepare your website - fresh and clean with no exposure to a search engine. Next we have to let it mature a while to get rid of initial transients due to the domain name being young. We then place one page with keywords and one without keywords and wait to see what the result is in six months. Repeat many times and present the statistical evidence of effect of keywords complete with a measure of the strength of the effect.
Of course doing the experiment isn't going to be easy - no one ever said experimental science was easy. You need to control for spurious effects such as type of page, access, website structure, traffic, in coming links and so on. But with enough repetitions you can average out these effects and end up with a true result.
Slowly but surely ESEO could open the black box of the search engine and lay bare the truth about the way the web really works.
Can we afford not to do this?
Return to The Stone Tapes for more musings on the web, its failings and shortcomings