11.21, Saturday 22 Jan 2011

Google appears to have a problem with its search results this month. They're filled with rubbish, and people are beginning to notice. Google acknowledge the problem. What I hadn't realised is that many of the rubbish results come from a small group of companies. Each of these companies is a "content farm" -- they identify what people are searching for online (eg, "winter tires") and then write articles relevant to that topic (eg, how to put snow tires on your car). Then they sell ads around those articles, and collect money. The biggest of these content farms is Demand Media.

This is the future of journalism!

It's so incredibly responsive. Imagine if Hollywood could turn out emo horror romance films immediately the vampire craze started. Or that, leafing through the Times or the Guardian or the Economist, the newspaper could magically sense your interests and create analysis and reportage dedicated just to you.

What's lame (of course) is that Demand Media's articles are no good. Here's their low-down on buying snow tires: Invest some time in comparison shopping. Prices for snow tires differ by retailer as well as region. Make use of the Internet, clubs and department stores in addition to tire dealerships. Well no shit. There's technically nothing wrong with what these articles say, but you get a vague sense of wrongness when reading them, like when you're talking to a super articulate idiot and you can't quite put your finger on why they're an idiot. But you know.

Google's search technology uses 200 signals to rank one result above another. Signals like "page rank" (how many important web pages link to this page) and title text are important. But these signals evidently aren't enough to weed out the dross.

One big new Google innovation is Social Search. This is the idea that your friends will have more relevant answers for you than the average of the entire rest of the web. Social is big! It's why Facebook is so exciting.

But there's another signal Google need: taste. The difference between a good article and a sophisticated spam article is no longer anything simple like number of linking, or quality of spelling. It's something weird and human. It's the quality that editors of newspapers have, and that every single person has very strongly in very individual areas, and some of it is personal and some of it is universal.

And I've no idea how they'll do it, but it's required. Search engines need to acquire a sense of taste.