Twitter Can't Predict Elections Either
Written by Alex Armstrong   
Wednesday, 02 May 2012

While it seems that big data can solve more or less any problem, there are some things it just isn't up to. Using social network data to judge public opinion and specifically to predict the outcome of elections seems as it if is a rock solid data mining application, but is it?

Twitter data has been used to predict election results so often that we tend to think that the opinions expressed in the tweet are the key to predicting the future. Various papers have been published that claim predictions of all sorts of future results including the stock market and the occurrence of pandemics. But are these claims justified? Is there predictive knowledge in the massed tweet?

 

twit2

 

In an imaginatively titled paper “I Wanted to Predict Elections with Twitter and all I got was this Lousy Paper” researcher Daniel Gayo-Avello at the University of Oviedo puts the case that Twitter's predictive power is far from proven. He even goes so far as to say:

“No, you cannot predict elections with Twitter”

He then goes on to point out the flaws in trying to use Twitter in this way.  Perhaps the most damming is the observation:

"It’s not prediction at all! I have not found a single paper predicting a future result. All of them claim that a prediction could have been made; i.e. they are post-hoc analysis and, needless to say, negative results are rare to find."

Post-hoc prediction always carries with it the possibility of bias. He also points out that it isn't fair to compare the prediction against an even chance outcome - after all it isn't often that there is an apriori equal chance of all the parties winning. He goes on to list eight points where previous analyses are flawed and makes  recommendations for future work that might prove that Twitter can predict elections. As you can guess, the most important is for researchers to actually make predictions and firm up what would be a good prediction result. He also provides an annotated bibliography, complete with the fairly stark verdict that in no case was an election prediction actually made.

Notice that Gayo-Avello hasn't proved that you can't predict election results or anything else using Twitter data - just that no-one else has convincingly proved that it can.

 

twit2

 

Of course this has a commercial aspect. If companies are willing to spend big money to get hold of the big data that Twitter offers they need to be convinced that it has predictive power. If Twitter data can't predict elections can it predict market trends and public sentiment?

This is a matter that needs clearing up.

More Information

I Wanted to Predict Elections with Twitter and all I got was this Lousy Paper arXiv:1204.6441v1

 

Python

 



 

Comments




or email your comment to: comments@i-programmer.info

 

To be informed about new articles on I Programmer, subscribe to the RSS feed, follow us on Google+, Twitter, Linkedin or Facebook or sign up for our weekly newsletter.

 

Banner


Microsoft Updates PWABuilder
29/04/2019

Microsoft has updated PWABuilder, its open source tool for converting Web applications to Progressive Web Apps (PWAs). which act more like native desktop/mobile apps with the help of several technolog [ ... ]



Google Promotes ChromeOS at Google I/O
13/05/2019

Google seems determined to push heavily for ChromeOS
judging from the announced goodies at its annual developer event, Gogle I/O.But is it at the expense of its other well-loved operating system, Andro [ ... ]


More News

 

Last Updated ( Wednesday, 02 May 2012 )