Business (And Social) Alternatives to Page Rank
Posted by Bob Warfield on September 5, 2007
Let’s assume you’ve managed to implement my wish list to enable an Emergent Enterprise 2.0 information community. You’ve busted through all the information silos, and someone can now do Enterprise-wide searches with the results filtered through an appropriate Trust Fabric based on what you want each person to be able to see. I want to suggest some possible changes in how your Search Engine may want to access the data. In particular, I want to suggest that Page Rank might be productively replaced with something a little more powerful which I’ll call Job Rank (simply because it sounds better than Role Rank, which is a better description).
To review, search engines start out by analyzing what words are on a page to determine the page’s relevance to your search. The Page Rank algorithm adds weighting to the keyword-based search by looking at how many other sites link to your page. The idea is that those links represent votes in favor of your page being even more relevant. There are some issues with Page Rank, but in general, it has worked out better than anything else so far.
Looking at the Enterprise Search problem, we can certainly apply keywords and Page Rank and get a decent result. However, we have more information available in the Enterprise. For example, we know quite a lot about each of the people who creates both the original content and any links associated with the content. For example, wouldn’t it make sense that somebody who participated in the creation of the content might know a little more about it than someone who didn’t? So therefore, perhaps we should weigh links provided by content creators a little more highly than other links.
What about content and links created by folks working in the department responsible for some area? We might, for example, access a tag cloud derived from the mission statement or brief discussion of the department to determine the department’s role. Or, so long as we’re analyzing text to determine how to think about someone’s role, how about feeding in each person’s resume plus a narrative description of their latest job role. That would establish a baseline for what the person should be an authority on. Lastly, how about document purpose and version or age. Certain documents are supposed to have the final say on various issues. Part of the strategy a business can follow is to make sure things like the final version of a product specification are accorded that status and the information is made available to the search engine’s ranking algorithm.
There is rich potential for domain specific search within a business, particularly if micro-domains are enabled. “Micro-domain” is my made-up term for being able to customize search on the fly to fit a small domain or topic. We used a micro-domain technology I developed to great effect when searching and categorizing eBay auction items. eBay was an extremely noisy search environment, meaning that the auctions were not that well written and could even be misleading sometimes. I hit on the idea that instead of using your typed in search to search the auctions, I would use it to search through a set of “meta-searches”. These metasearches were carefully constructed and tested searches that performed a much better search than most of what people typed in. A team of just 6 “taxonomers” were able to create well over 1 million high quality searches and we had an extremely powerful result that was much better than eBay’s search. Let me give an example of how this worked. Someone would type in “car stereo” and the Micro-domain would drop them at an intermediate place in a big tree, because “car stereo” is too broad. The intermediate place (chosen and designed by the Taxonomers and encoded in the Micro-domain language) was a simple multiple choice that asked if they wanted head units, amplifiers, speakers, or some other component. Choosing one triggered the next micro-domain search expression. For the car stereo example, we found that a list of brand names was extremely potent rather than more general terms. You can imagine how this would work for the Enterprise where a relatively small group of what some have called “tag gardeners” could keep creating Micro-domains to improve searches. Heck, I don’t know why that wouldn’t be a better strategy for the Mahalo’s of the world too.
Businesses have far less information to sift through than Google (the whole Internet). It should be possible for them to employ search methods that are far more powerful than what we can afford to use for the net. Yet, all too often, search inside the firewall works a lot less well. That’s just silly. I haven’t seen an Enterprise Search Engine that incorporates the features I’m talking about here, but wouldn’t it be exciting if you could get that kind of technology sifting through the information inside your firewalls? That would finally get to the promise of Knowledge Management we’ve all heard about for so long but never yet seen.
The title of the post alleges applicability to Social Networking as well. Why not? You can apply the same sorts of principles. You know more about the participants in a Social Network than Google knows about the average web page. The reason is that the Social Network adds a lot more structure than just flat web pages with links. A search engine that was aware of this structure could use it in various ways to generate better results. That’s really all I was driving at with the myriad of discussions around enabling Social Graph Search.