WIRED:MANY YEARS ago, back when we did not despise anyone in the financial industry on sight, I heard a description of a working partnership at a Wall Street firm that sounded almost romantic.
A party guest told me how he worked in a firm where individual traders would work paired with a programmer colleague, who would sit beside his companion, nestled together in front of the banks of screens and terminals.
While one would do the trading, the other would, on the fly, construct entirely original programs to match and augment that trader’s instincts. The coder would work to create new ways to visualise the market, and automatically mine for new information and patterns in the endless data flow of stocks.
At the time, I rather liked the idea of two intrepid explorers fabricating the tools they needed to pan for gold in a new information frontier. I like the latest version of this pairing more.
Instead of tying financial whizz kids with coders, this new partnership connects investigative reporters to the same kind of data-miners. Instead of pure profit, these couples hunt for stories: tales of corruption, graft, and hidden motives, buried in the mountain of data that corporations, governments and individuals emit in the modern age.
Most reporters can’t afford to hire a side-kick coder, of course – especially in a world where technologists seem to have emptied journalism of many of its traditional revenue streams. The pattern that emerges in this new media is more of a hybrid.
Some of these data journalists are reporters who have learned to program themselves. Others are programmers who have chosen a career change as journalists.
Either way, large forces in the traditional world of journalism are betting that technologists and reporters work together in the future more than they work against each other.
The Knight Foundation is one of the biggest non-profit funders of the cutting edge of journalism. Based in the cities where US newspaper moguls the Knight brothers ran their papers, it has assets on close to $2 billion. For the last five years, it has handed out a total of $27 million to those experimenting in the building of tools for new forms of journalism in those cities in a competitive grant process called the Knight News Challenge.
More than 12,000 projects applied, but only 76 were funded. The last 16 were announced this week.
The winners this year continue the trend of trying to graft journalist and programmer together: or hackers and hacks, as one global meet-up between the two cultures calls it. Some awards were aimed at providing flexible tools and number-crunching for grassroots, citizen reporting.
Half a million dollars went to the Public Laboratory for Open Technology, which famously helped residents generate high-resolution “satellite maps” to chart the local extent of the Gulf oil spill, using helium balloons and digital cameras.
Others are aimed at providing assistance for professional journalists, including $475,000 to the Associated Press’s Overview project which will refine, mine and display large data sets from governments and other sources.
Knight has also been funding individuals, including a scholarship programme to retrain coders as journalists. Brian Boyer was a former Knight scholar who went on to win a $150,000 grant for his Panda project, a set of open source web tools his team is building to help reporters seek out patterns in data using the web’s already extensive array of analysis tools.
This new move to “hybrid data journalism” is certainly a search for how to do journalism in an internet world but, unlike those Wall Street profit-seeking duos, I don’t think it’s a question that revolves exclusively around finding the money. It’s more about working out how to do journalism better.
We live now in an environment where the lowest hurdles of journalism – expressing an opinion, doing some research, uncovering a new story – are made far easier for anyone to surmount. I can publish an opinion with a few clicks online. The research tools the average smart surfer has dwarf the most expensive databases hoarded by media companies 10 years ago. Hoards of online detectives can track their obsessions far better than the hired hand of a local reporter on their beats.
What remains difficult is filtering, scanning and picking out novel signals: the far-out stuff that the net, en masse, would never find on its own.
And I think that’s actually closer to the real reason why those traders teamed up with coders. It wasn’t about money. It was about competition. In order to get ahead, financial wizards worked not only to be smarter than the next person, but to be smarter than the market as a whole. The money emerged, in theory at least, from that.
There are dangers, clearly. What felled the financial industry is that, in the end, they were all pursuing the same goals, listening to the same signals, deriving the same conclusions. There’s often an element of groupthink among geeks, just as with journalists, and it would be a tragedy if the two were to combine to form one monoculture, chasing its own tail.
But if journalists really are in pursuit of novelty and innovation, technologists might be their best friend in creating the tools that might let them find it. To succeed, those tools need to be distributed widely, with no preconceptions about how they should be used, or who should use them.
The stories of the future are out there: but the further away they are from the stories we tell in the present, the more resilient, and more honest to the real world, they will be.
