A new study from Carnegie Mellon University shows that analyzing data from Twitter yields the same results as conducting a public opinion poll.
We’re willing to bet it probably costs less, takes less time and annoys fewer people, as well.
A CMU team from the computer science department looked at sentiments expressed in a billion Twitter messages between 2008 and 2009. The researchers then use simple text analysis methods to filter out updates about the economy and politics and determine if the overall sentiment of the update was positive or negative. The CMU team found that people’s attitudes on consumer confidence and presidential job approval were similar to the results generated by well-reputed, telephone-conducted public opinion polls, such as those conducted by Reuters, Gallup and pollster.com.
For at least some topics, CMU Assistant Professor Noah Smith thinks this kind of passive information gathering could work. “With seven million or more messages being tweeted each day, this data stream potentially allows us to take the temperature of the population very quickly,” Smith said. “The results are noisy, as are the results of polls. Opinion pollsters have learned to compensate for these distortions, while we’re still trying to identify and understand the noise in our data. Given that, I’m excited that we get any signal at all from social media that correlates with the polls.”
The CMU researchers did notice that Twitter sentiments had much more day-to-day variation compared to data gathered from traditional polling data. To compensate, the team averaged the Twitter results over a number of days; at that point, the results were generally quite similar to polling data.
For example, on Twitter as in life, consumer confidence slumped in 2008 and started to revive last spring. And Twitter updates showed the same general decrease in presidential job performance approval through 2009 as was seen in traditional polls. There were a few discrepancies — enough that the CMU folks don’t recommend using Twitter to poll for election results just yet. Still, researchers hope that better natural language processing (NLP) techniques will make Twitter and other social media a valuable source of public opinion information in the future.
The paper is available online (use the left column to navigate to the “papers” section, and scroll down to “From Tweets to Polls: Linking Text Sentiment to Public Opinion Time Series”), for those who’d like to dig deeper.
What do you think: Could Twitter and other social media eventually replace public opinion polls? Would you rather have a researcher mining your public stream of updates for information or calling your phone to get the same information?