Studying the impact of innovation on business and society

The Human Algorithm: How Google Ranks Tweets in Real-Time Search

In 2009, Google struck a deal with Twitter, rumored at $15 million, to integrate tweets into keyword related Google searches. And last month, Google also integrated real-time search technology to surface blog posts and news content as they hit the Web – dramatically improving the previous five to 15 minutes its spiders would take to crawl the Web. I should also note that Collecta also offers the ability to search the real-time Web, but its results also include popular networks within the social Web. Between Google and Collecta, Twitter Search is starting to show its age.

The opportunities and benefits of accessing the real-time Web also represent its most notable deficiencies – the ability to truly focus the stream of cascading information into a river of relevance. Companies such as My6Sense are using a form of “digital intuition” to escalate tweets that match our patterns, behavior, and content we read.

We are now staring in the face of a more sophisticated era of real-time search that will further advance, localize and personalize over time. And, everything starts with the experimentation of sophisticated algorithms that filter and rank the content we’re hoping to discover.

For example, Google recently adapted its PageRank technology for surfacing related tweets. PageRank was originally developed to help find relevant Web pages through traditional search and was Google’s primary differentiation in a world of commodity search platforms. Essentially, Google’s PageRank assesses the importance of Web pages tied to keywords based on link structure. Authority is determined by the quantity and quality of inbound links to each page as well as the branches of outlying link relationships that tie other pages to those within the first degree of inbound connections. In other words, the more links to a page and the more linkers to each link, the greater the value of the original page.

The challenge with real-time search is tying tweets or other social content to notable producers and their networks of reputed followers and sub-follower architectures.

In an interview with Technology Review, Amit Singhal, a Google Fellow who led development of real-time search, said “You earn reputation, and then you give reputation. If lots of people follow you, and then you follow someone–then even though this [new person] does not have lots of followers, his tweet is deemed valuable because his followers are themselves followed widely.

As Singhal emphasized, “It is definitely, definitely more than a popularity contest.”

Google also examines the signal in the noise, to surface the most relevant tweets related to common as well as obscure subjects. And as Twitter itself advances the technology that packages tweets, such as geo-location data, we can expect to see a rapid evolution of real-time search.

Basically, a follower is the equivalent of one page linking to another on the Web. Google recognizes each as a form of recommendation. So as higher quality pages link to sources, the original page increases in value. In the Social Web, reputed users who follow other users inherently increase the stature of the individual to whom they connect.

Searching for a particular keyword now will produce qualified results for Web pages and also content published in Twitter and other social networks, ranked by the authority of the page and publisher of social objects as assessed by PageRank technology.

In the eyes of Google, the adaptation of PageRank for Social Media essentially creates a human algorithm or a PeopleRank of sorts that may eventually serve as a foundation for also assessing the authority of an individual in the social Web.

Other companies are also introducing new services to measure general authority for individuals online. Klout, for example, developed a sophisticated platform for measuring the influence of users in Twitter. Based on three sophisticated stages of semantic calculation (True Reach, Amplification Probability, Network Value) Klout can determine not only the level of influence of any user on Twitter but also the most influential voices tied to topics or keywords. Microsoft’s search engine, Bing, is also including tweets in its real-time search feed and could, for instance, integrate Klout’s influence engine to rank tweets and other social objects to qualify results.

But while the idea of ranking influence on the social web is interesting and necessary, it is far from perfected. Running searches in either engine today will only reinforce this sentiment. However, with that said, it is helping us by reducing the obstacles that typically prevent or prolong the process of finding pertinent information. It will only improve over time regardless of our personal views on establishing a hierarchy of people in social media.

As the human algorithm continues to evolve, it transforms the definition of and logic for relationships. We’re adapting how we connect to one another and also constructing new roads for sharing, filtering, and ranking relevant social objects. The ties that bind us now serve as the source of how we discover information and also how it finds us. And as such, the relationships we maintain on the Social Web determine the ranking of the content we produce and its place within the social hierarchy of search results.

Perhaps the next iterations of Search Engine Optimization (SEO) and Social Media Optimization (SMO) will focus on enhancing the link structures of human relationships to escalate the prominence of our stature and the social objects we create and share.

Connect with Brian Solis: Twitter, LinkedIn, Tumblr, Facebook

Pre-order the next book, Engage!



Click the image below to get the current book, poster, or iPhone app:



Image Credit: Shutterstock

142 COMMENTS ON THIS POST To “The Human Algorithm: How Google Ranks Tweets in Real-Time Search”

  1. Simon U Ford says:

    Brian,

    Social or friend rank algorithm runs a lot deeper than social links from a users profile page, to profile page. They have done for some time now. If you search the term “Google friend connect” in YouTube. You will see a video I published over a year ago (publisher was EventsListed). It still sits at number 2 ahead of 10+ Google owned videos about their own product.

    At the time my video ranked number 1 in Youtube. I had a second video called “Beating Google Friend Connect” showcasing just how I beat Google sitting in the 2nd spot as well. That's positions 1 and 2 ahead of more than 10 Google developer videos for months and months. This was shortly after Google launched their Google friend connect widget. Some of Google's videos had 200,000+ views and any one of Googles videos had many more comments than both my videos combined.

    My YouTube video page therefore held out 4th position in the Google.com search engine as well. This is behind 3 Google owned properties ahead of every A list website and blog on the open web. This is at a time when Google Friend Connect was the talk of the town.

    The reason I ranked so highly came down to a few factors.

    1. The number of comments my video's received in the first 24 hours compared to Googles videos.
    2. What was even more influential was the percentage of the total video (in minutes) that my commenter's (voters) actually sat through, compared to the percentage of the total video Google's commenter's watched.

    3. Especially the comparative statistics between commenter's who watched and commented on both videos. I specifically asked my audience to watch my videos from start to finish and then comment within 24 hours of publishing. Then I told them to watch Googles video from start to 30 seconds in, then stop it mid video and then comment on the Google video as well. This provided YouTubes algorithm enough comparative data between common commenter's which I believe to have been the difference (having plotted / compared all of the other variables).

    This is why the social (friend) rank system beats page rank hands down. Beating page rank only requires the right knowledge to game it. Beating the friend or social rank system requires both the knowledge and having a very responsive, large audience of followers who listen to you before you would even try.

    • Hmmm interesting… I ran a campaign w/300 vids and noticed the comments correlation. I also felt that the number of subscribers to the feed was a signal/factor in ranking and why IMO, not having vids on YouTube is a huge mistake if you are trying to optimize for Video slots in Google Universal SERPs. Google has access to all the data through YouTube so… these are very hard signals to spam.

    • It's interesting to know this fact and even I use twitter more and I know that google index twitter frequently.

  2. netaffinity says:

    Hi Brian,

    Love the article, topical and on the button as always. There is a real and definite shift in how 'search' is dealing with the real-time web. Google and Bing are struggling to present it in a real and meaningful way. I think there are great opportunities for new entrants to the search space and love the collecta concept.

    Keep up the good work.

    Will.

  3. What is interesting is Plurk fumbled and never made it with Twitter, yet had plurk allowed dofollow links would it have taken off and competed with Twitter?

Leave a Reply

Your email address will not be published. Required fields are marked *

Join the Newsletter

Sign up for the newsletter.

Stay Connected
Instagram
Twitter