27 September 2009

The Amazing Possibilities of Social Search

Search: http://www.digitaltonto.com/archives/427#more-427

A blog about Digital Business and more…

* Home
* About Greg Satell
* Why Digital Tonto?
* Newsletter

Digital Tonto
The Amazing Possibilities of Social Search
2009 September 16
tags: Duncan Watts, HITS algorithm, Jon Kleinberg, Network Theory, Page Rank, Search, Social Networks
by Greg

At the intersection of Social Networking and Search is an exciting frontier that is just beginning to be realized. Through more efficient analysis and subsequent comprehension of the relationships between information we will gain a greater understanding of the world around us and interact with it. The implications for both publishers and marketers will be powerful.

The concept of Social Search intuitively makes intuitive sense. We know a lot of people and know that some of them have specific knowledge and wisdom that is useful to us. However, locating the person in our networks with the specific intelligence to aid us in a specific task can be difficult.

In the digital world, the problem has been extended to more types of information including documents, graphics, audio and video. The more skillfully we are able to navigate through complex webs of data and concepts the greater value we can derive from the networks that we create.

Our Vast Networks

On LinkedIn we can all see our own network statistics. I have a few hundred contacts. However, my 2nd degree contacts (friends of my friends) number over 100,000. Amazingly, my 3rd degree contacts (the friends of my friend’s friends) number in the millions- as large as some countries – all of whom are just two introductions away from me.

Theoretically, my network on LinkedIn contains almost anything I would like to know, from where to get a good latte in Budapest to what are the latest developments in cancer research. The concept predates the internet, as people who want to search for information often intend to “ask around,” but online it has been expanded to an astounding degree. Everything we want to know is at our fingertips – if we can find it.

Yet, with so much information so close, how do we navigate and find what we need in our networks when we need it?? How can we benefit from information contained in our own private information treasure trove? As Duncan Watts wrote, “Searchability is, therefore, a generic property of social networks.”

Broadcast Network Searches

I have a friend I knew in high school with whom I connected on Facebook. She updates her status frequently so I have a pretty good idea what her life is like now, even though I haven’t actually seen her for 20 years and she lives thousands of miles away. (She always seems to be going to the pool or to the night club – lucky her!). It’s not extremely useful information, but I like seeing her updates, knowing that she is okay and enjoying her life.

Recently, my friend lost her cat and she used Facebook to try and find it. She described her cat and asked if anyone in her area had seen it on her network status. I felt a little strange about being contacted about this. I felt for my friend, but obviously couldn’t help her. I messaged her a few days later to ask if she found the cat and whether everything was okay but, uncharacteristically, she didn’t answer.

I don’t think she was being rude, or doubted my concern. She was probably overwhelmed. She wanted to find her cat and hundreds of people who could do nothing to help her were probably also contacting her to see if she ever found her lost cat. Broadcasting her search was enormously inefficient.

Directed Network Searches

Usually, we want to direct our search so that we can get there in as few steps as possible. We have a desire for some information, and we immediately scan our local network. (Our local network has nothing to do with geography, but rather network proximity – our 1st degree connections in social distance terms.)

For instance, if I wanted to know, as mentioned above, where to get a good latte in Budapest, I could think of a friend who lives in Budapest. Because he is my friend, I also know that he likes lattes and probably knows exactly the place. I would ask him, and he would tell me. 1st degree social searches are pretty simple.

The more interesting case is if someone who knew me, but didn’t know anybody in Budapest, was traveling and happened to be in Budapest and felt a sudden urge for a good latte. Their local search would divert them to me in Kiev to find a latte back in Budapest where they could actually drink it. It’s a little like going a few miles out of the way to get to a highway on-ramp that will take you to your destination.

What makes this latter search interesting is that I would then establish myself as a hub of information about Budapest (and probably for the rest of Eastern Europe as well). Future searches would probably also come my way.

Affiliation Networks

What we do in our social searches is identify what the network theorist Duncan Watts calls “Affiliation Networks.” We know what we are looking for and we know that it is associated with some other things. So, quite reasonably our search starts with something or someone that shares a common attribute with our target.

For instance, let’s say we wanted to find a stockbroker in Boston (as in Milgram’s famous experiment). We would think of stockbrokers we know and people in Boston that we know. Chances are, between the two groups, we would find someone who would be able to help us locate the stockbroker in question.

Amazingly, Watts found that with only two or three affiliations, a network search became much more efficient.

HITS Algorithm

The pioneer of this type of search electronically is Jon Kleinberg at Cornell. He has been at the forefront of both Network Theory and Search technology since the late ‘90’s. He had a very similar idea to Sergei Brin and Larry Page at Google, but with a small difference.

Google’s PageRank algorithm is currently the market standard (and was developed concurrently with HITS). A site is considered important if it has a lot of other sites linking to it (or if a site that links to it has a lot of sites linking to it). The more total paths that would lead someone to the site, the more valuable the site is considered. It’s a good idea and it’s based on the scientific meritocracy driven by cites (mentions in other papers).

HITS, however, has a crucial difference. While PageRank only takes into account the target of the search, Kleinberg felt that there were two elements that were important: Authorities (the target of our search query) and Hubs, which lead to the authorities. In our Budapest latte analogy, the coffee shop would be the Authority, while I, in my role assisting the search, would be a hub. If, for instance, the first coffee shop was closed I could be referenced for another possibility.

Two good examples of this type of search methodology are Amazon and Ask.com. Amazon regularly gives us affiliations (i.e. people who bought this book also bought…). Ask.com, using a version of Kleinberg’s HITS algorithm, regularly gives a list of “related searches” with our query results.

The Wonder Wheel of Social Search

So it would seem that Google has won the “Search Wars” with an inferior algorithm. Yet, Google shows us why they will probably continue to win with their new “Wonder Wheel” feature that utilizes the logic of the HITS algorithm.

Let’s say I wanted to know more about Social Search. I could do a Google search for Jon Kleinberg and get conventional Google results. After that it gets exciting!

I could then go to the top of the page, and click on “Show Options,” find Wonder Wheel on the left side menu, click it and a whole new world opens up. I can see that Jon is connected to Eva Tardos, who seems like a very nice woman and is doing some interesting research on algorithms herself. (Also, being Hungarian, she could probably help find the aforementioned delicious latte in Budapest).

I can also follow links to IBM research and someone named Amit Kumar, who also does exciting work on algorithms and apparently shares his name with a top Bollywood star. These links, of course, link to other interesting and exciting things. It’s an amazing (and fun!) way to research a topic.

In the future, we can expect the underlying logic of social search to continue to play a role in determining how people and information relate to each other. Using similar algorithms, we will be able to find commonalities among seemingly disparate groups of consumers and content, improving our ability to establish relevance between those that we market to and the information they seek. Consumer targeting, content management and overall web usability will benefit greatly as we learn to utilize Kleinberg’s concepts of Hubs and Authorities.