Wikipedia: Human-Powered Search
Has anyone else noticed this?
Often the best way to find a web page on specific topic is to use the external links in Wikipedia.
In the past year or two, I’ve noticed that Wikipedia very often ranked first in a Google search. So why bother even googling? I got in the habit of simply going to Wikipedia first. There sits a decent introduction for my query. If that was all it offered I might not keeping going there first. But again and again, the real meat in the Wikipedia was in the footnotes — outward-bound external links to the original sources, or other major experts.
In essence the gnomes at Wikipedia were hand linking my search topic to the best web pages on that subject. Now rather than page through Google results looking for the reliable expert, I go straight to Wikipedia – not for the boiled down summary — but to immediately exit to the prime sources. I use the Wiki article as a portal to frame, access, and point to the real stuff. Half of the value of a Wikipedia article is now the selective choice of external links.
I realized that the external links in Wikipedia achieve what the original Yahoo directory attempted, but could not keep up with. And it does what Mahalo and Squidoo are trying to do now: Have humans pick the best pages on a subject.
I wondered how Mahalo and Squidoo would fare when compared to Wikipedia external links so I decide to race them. Mahalo‘s mandate is to hand pick results for the 20,000 most common search terms and deliver them up when someone queries the term. Like Wikipedia, real humans evaluate the end pages and curate the external links. Squidoo is likewise hand-rolled. Rather than aiming for the most popular search topics, it is following the passions of enthusiasts (like Wikipedia) and completing “lenses” for subjects that its users care about.
I checked Google Trends to get a set of popular random search terms. I took the top ten search terms on Google yesterday and followed them in Wikipedia, Mahalo, and Sqiudoo. In each case I counted up the number of outbound links for each result page. I did not make an attempt to qualify each end link. In many cases the links from the 3 different portals would end up at the same final page. In many other cases, the final sites would not overlap. I simply counted how many links to prime sources were displayed.
The results:
A couple of things to note. Out of the ten most popular terms yesterday, I could only safely guess two of them: Steven Colbert and staph infection. The other eight were mysteries to me. Turns out that Jessica McClure and Baby Jessica were the same: the now grown girl who had fallen down a well and been rescued decades ago. Staph infection, superbug and mrsa were also nearly equivalent terms for methicillin resistant staphylococcus aureus, a mean staph superbug. E.g. daily was an actress, quadski is a sports machine, and avis ware, as far as I could tell, is a young female bodybuilder. The most popular term of the day did not even yield any meaningful results in Google itself. I still have no idea of what it is.
One more thing. This ain’t science. A sample of ten is pathetic. My counting is lame. Don’t take this little survey as proof of anything. It was an exercise I did for my own curiosity. The top ten searches on a different day might reveal opposite answers. But I learned something.
The more pop the term was the more results Mahalo displayed. The more obscure a term was the better Wikipedia was in referencing prime sites. Squidoo didn’t fare well in this popularity contest, so I went into my Google Search History to resurrect some random actual searches I had done in the previous week: Olympic tickets, rattlesnake bites, and recumbent bicycle. Mahalo had zero useful information, while Squidoo had plenty of very helpful stuff. These more utilitarian queries play into the strengths of Squidoo, which are user recommendations and expert advice.
But on these practical subjects, Wikipedia external links did as well. There might not be as many YouTube links, or book recommendations, but the reference links are solid and useful.
I believe there is huge value in human-augmented search. For many searches this will be the first step. From my experience the best tool for this kind of help is the many external links at the bottom of every Wikipedia article.