The intersecting worlds of big data, transparency and journalism are changing rapidly.

Need a roadmap to the new terrain? Fortunately, the University of Oxford has teamed up with the Reuters Institute for the Study of Journalism to produce a new volume: Transparency in Politics and the Media: Accountability and Open Government. The volume brings forth a star-studded line-up of journalists and academics to give the latest and greatest lay-of-the-land for those who want to get up to speed on what’s what.

Though there’s no substitute for reading all the contributions, I’ve tried to distill some of the highlights and big picture “here’s where things are heading” insights into the three big areas the book covers: big data, transparency and journalism

Big Data

There is now more data than ever before, much of it government data. For example, in its first three years, data.gov published more than 450,000 datasets of public records. Computer processing capabilities continue to increase, and more and more resources are being devoted to figuring out what to do with this data. “Big data” writes Helen Margetts (in her chapter, “Data, Data Everywhere”),“has become the theme of current thinking.”

In a chapter on “Corporations and Transparency,” Joel Gurin and Beth Simone Noveck express plenty of enthusiasm for the potential: “What is revolutionary is not only the quantity of data but also how we can use computers to search, sort, compare, aggregate, visualize, and track data.” Gurin and Simone are optimistic about the ability of data and transparency to hold corporations accountable through what they call “smart disclosure” – giving consumers targeted information to help them make high-leverage decisions, and giving journalists tools to help citizens make those decisions.

But, as Margetts notes in her essay, at this point, the benefits for public policy “are potential rather than observable” One reason is that datasets are still often messy and unstandardized, inconsistent and not machine-readable. Sometimes government officials make excuses and are slow to comply with requests. When these shortcomings prevail, it is difficult for anybody to use the data for much of anything.

The solution? Keep asking. As Jennifer LaFleur argues in her chapter on “Data and Transparency: Perils and Progress”: “As journalists we must continue to request information.”

Transparency

In general, governments are moving towards more openness and transparency, making more of what they do public. However, several chapters describe continued government trepidation, and even occasional backlash. For instance, former British Prime Minister Tony Blair, once a champion of open government, came very much to regret supporting the FOI law. As he wrote in his memoirs, “For political leaders, it’s like saying to someone who is hitting you over the head with a stick, ‘Hey, try this instead’ and handing them a mallet.” But, as authors Benjamin Worthy and Robert Hazell respond, “Blair’s assertions are not borne out by the facts about who uses FOI.” They note, for example, that if it weren’t for FOI, journalists would not have exposed corruption in the MPs’ expenses scandal.

In a chapter entitled “Valuing Transparency in Government and Media,” Patrick Birkinshaw makes a strong case for FOI and FOS (freedom of speech) laws, calling them “essential features of democracy.” They are essential, he notes, because, “I want to know on what basis governors decide to act in the public interest and I want the press and media to assist in that vital right.”

Indeed, several chapters explore the key role that journalists play as the users extraordinaire of all the new information. In his chapter on “The Transparency Opportunity,” Paul Bradshaw, for example, summarizes interviews with 26 data journalists. He found “overwhelming agreement that independent and government open data, right to information laws and APIs had each changed the way they held power to account.”

Journalism

But while there is much potential for journalists to take advantage of the new transparency and open data regimes, the business model of journalism is changing rapidly. Fortunately, two giants of journalism, Charles Lewis and Leonard Downie Jr., are there to describe the changes.

In short: the old model of the for-profit newspaper is giving way to the non-profit NGO or university-based center. Citing the model of ProPublica (home to 34 investigative reporters, financed primarily by foundations), Downie writes that, “The most promising alternative so far is the expanding but still unstable ecosystem of new and varied nonprofit, internet-based news organizations – many of which are focused on accountability journalism.”

Lewis, meanwhile, cites organizations as diverse as Human Rights Watch, Global Integrity, and the New America Foundation. None are traditional journalistic outlets, yet all produce high-quality investigative work that has broad impact. Lewis calls for a collaborative model, asking for journalists to become more receptive to input from everywhere. He writes: “public interest, academic, NGO, journalistic, and other investigative-minded organizations and individuals increasingly must share their information, knowledge, and expertise.”

The takeaway

In sum, the 13 essays in the volume provide a nice mix, laying out the new and ever-changing landscape. The essays are for the most part sober and fair-minded: realistic in both the possibilities and the difficulties of the new world of transparency and open data, and the potential for journalism to harness it for accountability. Certainly, there are obstacles and trade-offs. But on the whole, the general tenor of the volume is optimistic. Transparency and open data have great potential for improving democratic and corporate accountability, and a new model of non-profit journalism that is up for the task is gaining momentum.