Take the Pepsi Challenge for Search
Posted by Bob Warfield on October 3, 2007
There’s a lot of chatter in the blogosphere about Yahoo’s new search features. Seems Compete figured out that people click on Yahoo’s results more often than Google’s, and inferred that Yahoo therefore has a better search engine. Mashable adds that Yahoo has higher customer satisfaction scores too, but ends without opining on the winner. I think the propensity to click is very interesting, but Google Operating System worries that it isn’t compelling. So, they put together a “Pepsi Challenge” for Search.
I thought that was very cool, so I immediately started playing with it. The page gives you the first search result from each of Google, Yahoo, and MSN Live and asks you to rate the quality of the result. Importantly, they don’t tell you which engine the result came from. I read somewhere (and lost the link, DOH!) that a test someone else performed that mislabeled other results as being from Google had people 25% more likely to pick the choice labeled Google. That’s the power of brand!
Try the challenge (page is here) for yourself. You vote on your first search, but you only get to give one vote. You can change it on subsequent searches. My first search was “Multicore Crisis”, which I’ve written about quite a lot, although someone else coined the term. I liked search service #2 best for that, and when I voted, so did the majority, with the following ranks:
Service #1: 35%
Service #2: 54%
Service #3: 25%
I know which engine was #2, and it isn’t Google, but don’t rush over to skew the results, take the challenge as it was intended without thinking too hard about which service is which.
The experience left me wanting to try Service #2 for a more extended time to see if I really prefer it. I remember having the same sort of a reaction when I first saw Excite and Alta Vista, both of which trumped the other players in their day with better results. And I remember being convinced Google had the best results and switching to them.
Will people really switch? I don’t know. I think it’s really hard. The experiment where brand could overcome a 25% difference in quality of result seems damning. Nevertheless, I am personally interested in getting better results. Today my primary strategy for that has been to switch search engines, but not companies. See my post, “Stop Googling and Search for Blogs” to learn more. It’s only if I fail there that I search the whole web. Try it: the strategy works well. Meanwhile, I may have to move Google to my third place search.