Studying the impact of innovation on business and society

A Prediction: Twitter to Predict the Future

Trending topics reveal much more than the objects that captivate the hearts, minds, and keyboards of Twitter users around the world. Twitter’s trends is a cultural mirror that reflects the state of attention and intention. And as such, Tweets then offer an MRI that visualizes the minds of consumers and more importantly, serve as a crystal ball that reveals the future of products and services before and soon after they’re released.

For the most part, however, the vast amount of precious insight is widely untapped. Instead, businesses focus on volume and congregation, enticing brands to engage in the conversation rather than truly capturing and analyzing the activity that inherently inspires empathy and ultimately relevance.

I think that’s about to change…

Conversations as Predictive Markets

Research reveals much more than a state of events; it also unveils demand and intent, and when dissected through additional filters, data can predict what lies ahead.

Hollywood is no stranger to forecasts and predictions. One of the most accurate solutions to date is based on technology that converges the wisdom of the crowds to create predictive markets. The Hollywood Stock Exchange (HSX) for example, enables consumers to buy and sell virtual shares of movies and stars. As the world’s leading entertainment stock market, motion picture executives now have a real-time ticker to gauge interest, demand, and the prospective of projects.

Scientists at HP Labs in Palo Alto may have uncovered another real-time exchange that will empower studios as well as everyday businesses to surface opportunity and probability and that conversational stock market is better known as Twitter.

Sitaram Asur and Bernardo Huberman of HP Labs essentially proved that social data can accurately predict box office revenues. As explained in their own words…

In recent years, social media has become ubiquitous and important for social networking and content sharing. We demonstrate how social media content can be used to predict real-world outcomes. In particular, we use the chatter from Twitter.com to forecast box-office revenues for movies. We show that a simple model built from the rate at which tweets are created about particular topics can outperform market-based predictors. We further demonstrate how sentiments extracted from Twitter can be further utilized to improve the forecasting power of social media.

To provide a glimpse into their work, Asur and Huberman basically calculated the frequency of film titles as they appeared on Twitter, tracking 24 movies and 2.9 million tweets over the course of three months. Films ranged from Avatar to Twilight: New Moon.

There are two approaches that the duo factored into their analysis and in doing so, spotlight the need for businesses to consider the distinct effects of buzz, word of mouth, and experiences as they impact and influence behavior.

In the first analysis, Asur and Huberman examined first week performance based on computer modeling that factored two variables: the rate of tweets around the release date and the number of theaters playing the film.  The results of the first model were as astonishing as they were accurate, predicting the opening weekend box office with an accuracy of 97.3%. As a comparison, HSX.com, the current standard for opening box-office predictions, was just under 1% with 96.5% accuracy.

The second analysis separates buzz from experience and as such, required the creation of a ratio of positive to negative tweets. The data was then fused with another prediction algorithm, which resulted in 94% accuracy.

As in any research project, we must be mindful of the group in which we sample. If demographics are important to the results, then we should observe that the average age of Twitter users is 39.1 and the user base is comprised of 57% female and 43% male. But still, social data across a myriad of social networks is invaluable and mostly available for examination.

The Business of Prophesy

Predictive marketing and ultimately the reality of behavior are accessible to almost everyone ready to embrace the prospect of shifting from a market of uncertainty to one of prediction. One of the most fascinating aspects of the research stemming out of HP Labs is the ability to not only predict performance, but also learn from the reactions and actions of consumers.

For example, Apple could have applied a similar formula to officially forecast iPad sales in week 1 and week 2. In the weeks, and most notably, the several days leading to the official debut, iPad trended on several occasions.

Aside from foretelling performance, Apple, or any brand really, can monitor the same activity to unearth opportunities for new features, prioritize fixes, and also assess experiences to ideate future products, applications, and iterations. Brands now also have the ability, if properly aligned with the prowess and empowerment, to detect potential problems before they swell and become official social tsunamis. In the case of the iPad, for instance, conflicts, crashes, hardware issues, etc., are identifiable in real-time and if the infrastructure of the company is designed for rapid response, it can then initiate the development and distribution of fixes, communication, and solutions.

Let’s not ignore the power of IF – the Influence Factor either. These conversations, en masse, represent sentiment and intention and also influence the corresponding actions of affected social graphs. This sets the stage for a network effect that determines the state and fate of products and services, culminating in a centralized hub for future reference.

For any brand or organization, the mastery of listening and also informed engagement, contributes to perception and awareness and as such, influences those who are introduced to ancillary perspective. Conversations transpire with or without us, and our absence speaks louder than our words.

In the end however, nothing will help substandard, ordinary, or bland product and services. And the point of all of this is, if you continue to introduce objects into the marketplace that are not based on the readily available real-time information before you, then you deserve the fate you’re dealt. At some point, this becomes less about chance and more about destiny, with a little bit of luck thrown in for good measure. Twitter and other social networks are the crystal balls that present the intelligence as well as the influential voices, who can help us create and participate with purpose and significance.

The Future of Futures

The work of Sitaram Asur and Bernardo Huberman is profound as it fundamentally transforms how businesses forecast performance. But that’s just the beginning.

The research of Asur and Huberman manifests a catalyst that will transform businesses as well as the corresponding cultures and driving methodologies of product design, service, and engagement from the outside in and equally, from the bottom up.

The wisdom of the crowds, when harnessed creatively, will reveal everything businesses necessitate to inspire the creation of meaningful and authentic products…where consumers become stakeholders as their voices contribute to the shaping and evolution of the end result. And, those who weave the deliberate acts of listening, predictive analysis, and studying market reactions will learn and demonstrate that markets are indeed defined by conversations.

Connect with Brian Solis on Twitter, LinkedIn, Tumblr, Google Buzz, Facebook

Please consider reading my brand new book, Engage!


Get Putting the Public Back in Public Relations and The Conversation Prism
:


Image Credit: Shutterstock

99 COMMENTS ON THIS POST To “A Prediction: Twitter to Predict the Future”

  1. Greg says:

    Many thanks for sharing this with us.
    Really like the in-depth nature of the analysis that was conducted!

    In Dec 2008 I wrote “Social Media Monitoring as a guide to buying or producing goods for resale?” when I realised what a great tool social media monitoring could be for forecasting sales potential, but I obviously didn't dig as deep as the gentlemen from HP Labs.
    http://blog.mydigitalpartner.com/2008/12/social

    Really exciting times for marketing & communication professionals!

    All the best,

    Greg

  2. Brian, this is astonishing stuff. Thanks for sharing it. Btw, loved your appearance on WineLibrary. Good stuff. Do you see the tracking of Twitter topics as a feasible thing for a small business? Without much manpower and time available, what small steps would you suggest?

    • briansolis says:

      Michael, thanks…appreciate that. I was wondering if anyone saw the episode. I think small business owners can learn alot about multiple aspects of the business landscape, climate and their opportunities based on listening….definitely to surface intelligence that helps in the near and long term. Most of it is free and the information one gets as a result, is priceless.

  3. Dara Bell says:

    I think the conversation stock exchange element of Twitter goes to the societies valuing of a particular trending topics. The Trending tag Ipad, is useful for brands such as Apple as it is postive WOM chatter that 1) they did not pay for and 2) will facilate the launch of their product, real time PR in other terms through Twitter.

    So like you often assert it is the value that the society places on the Ipad, really a middle America device. Aimed at consumers of content not producers. The interesting thing for Apple is they do not have a large presence (to my knowledge) in social media compared to say Dell. The PR is self generating from Seth Godin style ” remarkable products” and spread through the web by people just well excited by it all.

  4. Mike Stenger says:

    Wow, those are some super impressive stats. An accuracy of 97.3%? That's insane. Pretty interesting that in comparison to Facebook giant, small Twitter has the power to predict things like this. Continued proof that word of mouth has the greatest power to shape and influence customer's decisions.

  5. Predicting? Or identifying in real time? The studies include tweets during first week of the film's release, which distinguishes it from the pre-release PR tweets and viral campaigns. The number of Snakes on a Plane tweets *before* the movie was released says nothing about the flop that it was in theaters. The real-time measurements of Asur and Huberman during week of release (with sentiment factored in – i.e. the 2nd chart) would be far more accurate. I like this study and I wonder when we'll begin to see it shift into other entertainment fields, i.e. predicting / influencing music and book charts.

    I like to say that if it's on Twitter, it must be true. Or it soon will be.

  6. danieljohnyoung says:

    This reminds me of a conversation I had at the weekend with a guy that imports CDs for an international music publisher. His marketing team are monitoring social media conversations to help them anticipate spikes in demand for music imports so that pre-empt the demand and begin manufacturing the CD locally to avoid the increased cost associated with imports. Its ten times more expensive to import than manufacture locally. I thought this was a great application of future-casting and I encouraged the guy who choses what to import to start his own blog so that he can start generating the demand as well as anticipating/responding to it.

  7. Megan says:

    New Moon is already highly anticipated on its own because first, it's a sequel–not just a sequel, a sequel to a box-office hit; and second, let's face it Robert Pattinson has a huge fan base who are willing to watch the movie over and over. I'm sure twitter helped spread the anticipation but come on, I doubt you needed twitter to predict it would be a box office hit. We all know that twitter is the new grapevine. It has become a source of insider scoop among celebs and filming of certain anticipated movies. And I'm sure it would be very beneficial to predicting movie outcomes because unlike other products/services, movies are created years before the actual release. They have time to build excitement and anticipation among movie-goers. How does twitter work on other products like say…cellphones or computers–where there's always a new model released every month?
    I also recall of a TV show that generated a lot of buzz over the internet but when it came out, it was a flop. People didn't like it and eventually it was cancelled.
    I'm not saying that twitter cannot do as you've claimed. I think it has potential to really do predict sales outcomes. I guess I never saw it that way. I've always thought of it as a means to communicate with your audience. And people can be fickle sometimes. So, I think I'll remain skeptic about this one.
    In the end, it all boils down to sales trends as it says here http://budurl.com/ybru I think it's still how you read the market, read the signs, and sure twitter is good way to receive feedback from your audience.

  8. joshmchugh says:

    Yes please! Okay @briansolis – one question: when does the Asur-Huberman API go live?

  9. Ike Pigott says:

    Brian – I think you're on to something, but Twitter is only part of that ecosystem of sharing.

    Google has the other parts.

  10. Akash Sharma says:

    Hi Brian,You have shared probably one of the best ways to use twitter for forecasting consumer behavior for a better brand focus in the near future.We are gradually learning that conversational engagement further on gets divided into many domains which are emphasized by several of these undiscovered tools.Thanks for sharing some good stuff again….

  11. David Perdew says:

    Hey Brian:

    First time I've paid a visit to your blog. Very impressive. Lots of information I can share with the my Twitter members. I'll “stay tuned for more”!

    Regards,
    David Perdew

    By the way, if you'd like to see the coolest Twitter applications on the web, visit My Twitter Toolbox at http://MyTwitterToolbox.com. With more than 250 Twitter tools for productivity and fun, you'll discover everything you need in one spot to be incredibly effective with Twitter.

    If you really want to know how to use Twitter and other social marketing tools, check out this 60-minute audio with four of the top online marketers in the business as they reveal their top strategies for social marketing.

  12. Helpd Hd says:

    You are stunningly beautiful!

    Regards, Helena

Leave a Reply

Your email address will not be published. Required fields are marked *

Join Our Mailing List

You have Successfully Subscribed!

Stay Connected