Grounding Google

WIRED:  'Google in danger from Wikipedia" read the headlines here

WIRED: 'Google in danger from Wikipedia" read the headlines here. Silicon Valley loves its King Lear family dramas, and none seems better than this tale of the young protected child turning on the figure who made its success.

Google may not feel threatened by the small-fry commercial project slowly kindling under the management of Wikipedia founder Jimmy Wales. But as a sign of the times, it may be the beginning of the end of the Golden Age of Google.

Google made Wikipedia. It might not have intended to, but the user-edited encyclopedia was made to fit into the Google-eye's view of how the web worked.

Google's search engine places higher on its pages sites that have a lot of incoming links. As a useful reference work, edited and touched by many web hands, Wikipedia is linked to from all over the internet.

READ MORE

That's not surprising: if you're editing Wikipedia, chances are you are also editing some other web document, be it your blog or some other hyperlinked piece of text. Wikipedia's status in Google was lifted high by this web of approving links, added by hundreds of thousands of other websites, all pointing to the one Wikipedia.

Google in its own way exploits the efforts of these worker bees, using their collated "votes" (or links) to guide its judgement of individual websites.

More controversially, having collected that knowledge from the net-authoring public, Google keeps the secret formulas that it uses to derive the rest of its magic from the rest of us.

As this column has noted previously, it processes a great deal more private data handed to it by its users - the content of e-mails, clicks on links, even visits to third-party websites, tracked via the ubiquitous Google Ads.

Jimmy Wales's new company, Wikia, aims to wrest some of that power away from the search engine giants and back, he claims, to those to whom it belongs.

The simple idea behind Wikia is that it's a collective search- engine in the same way as Wikipedia is a collective encyclopedia. Instead of Google going out and collating the world's information, Wikia's members use their own computers to learn about and share the structure of the internet.

One of the first big steps in this ambitious plan was Wikia's purchase of Grub, an early and ambitious distributed "web spider". Web spiders are what Google uses to explore and record the web; small programs that constantly scan every web page and click on every link they can find.

Google's spider - the famous "GoogleBot" - emerges from its nest on Google's vast network of servers, and returns to store its data on the thousands of terabytes of Google-owned storage.

If Wikia's spider is going to be based on Grub (and excuse me if any arachnophobes are becoming queasy), then the spiders will emerge from an equally large number of computers, distributed in the homes and offices of Wikia supporters.

This kind of radical decentralisation worked well for Wikipedia: individuals all over the world quickly built an encyclopedia that dwarfed Britannica in size (if not always accuracy).

Is Jimmy Wales right in seeing Google as the next Britannica: centralised, slow and no match for a hive of busy volunteers? He may well have chosen the toughest nut to crack with a decentralised approach.

To fix the hairy problems at the heart of search engine ranking, Google's researchers long ago embarked on implementing a branch of computer science particularly suited to large, somewhat centralised networks of computers.

Most of the expertise in this area is now employed by Google and most would balk at taking the optimisations Google can make by keeping all its eggs in one basket and throwing them away. Sometimes big centralised piles of paid professionals do beat a ragtag army of volunteers.

What might undermine Wikia's plan more than Google's high collective IQ, though, is motivation. While some complain of Google's exploitation, only slightly fewer complain about Wales making money on the reputation of Wikipedia - which, too, is built on the free labour of others.

Many argue and demonstrate by their actions that there's an inbuilt worth to copy-editing encyclopedia articles for free, but it's not clear what the overpowering benefit of Wales's project might be.

Beating the Google monopoly might be good for privacy and good for the market, but what's in it for the volunteers? Running a slice of a search engine for the pleasure of helping Wales win an argument? Perhaps not so appealing, even for those who love decentralisation.

The volunteer-written Linux may have shaken Bill Gates, but it was written more for love and fame than vengeance. What motivates the Wikia-lover?

The irony is that if anyone loves and values decentralised systems, it would be Google.

The decentralised knowledge network of all those millions of votes that Google bases its page- ranking system upon is what made Google.

Perhaps Jimmy Wales will discover that it still knows them well, and knows their limits better than Wales does. Or perhaps even the PhDs in Mountain View can be surprised and overpowered by the amassed might of the network.