Tom Steinberg’s recent piece on Civicist raises familiar critiques of the open data movement, suggesting that advocacy is too incremental, political leaders are duplicitous and many basic problems have yet to be addressed.

Much of this critique is deeply familiar to the Sunlight Foundation, which has been a leader on both the adversarial and collaborative approaches to open data reform since 2006, giving us unique perspectives on the concerns Steinberg raises.

“Collaboration is threatening reform”

The threat of “overly-friendly collaboration between governments and transparency advocates” is often overstated. If advocacy can be split into two sides — the adversarial and the collaborative — then it’s reasonable to ask when the two approaches are rivalrous.

Steinberg suggests that the “oxygen” for advocacy is being hogged by collaborators, which neutralizes the outrage necessary for reform. In theory, one can imagine weak open data commitments being used for political cover. In practice, though, this defense suggests that open data commitments have far more political relevance than they actually do.

When the Obama administration’s opposition to strengthening the Freedom of Information Act came to light, no criticism was shut down through appeals to incremental progress. Obama claimed to champion FOIA while opposing Congress’s attempts to codify the same ideas.

Hypocrisy and scandal have their own distinct, self-evident merit; that’s why they’re politically powerful. Open data reforms have nowhere near this level of political salience. That’s part of why the first level of progress on open data has come so readily — our starting point was basically nil. Ten years ago, governments had basically no language on data in a modern sense, and that’s why vehicles like the Open Data Charter (garnering endorsement from the G8/7 in a shockingly short time), What Works Cities (building data-based governance approaches in dozens of American cities), Open Government Partnership (inspiring hundreds of commitments to open data and other topics in countries around the world) and many, many others have had such success spreading new policies, ideas and practices. When you’re starting with images of data in PDFs and senior leadership with no familiarity with data, there’s nowhere to go but up.

Collaborative initiatives like these are clearly just a starting point, and we do occasionally worry about whether they crowd out other approaches. I’m not particularly concerned about political capital, though, since scandal, hypocrisy and even vision for new reforms are all far more powerful forces than whatever modest defense against scandal incremental reforms might provide. I do see a more powerful concern, though, in the other resource modern reforms are often built upon: financial support.

The philanthropy and aid that supports the nongovernmental organizations that advocate for open data is certainly rivalrous, even if the amount of available funds is increasing with time. It’s not clear, however, that financial support is prioritizing collaborative approaches over the adversarial. This may be true from specific funders, but I don’t see that trend in the broader field. Philanthropy creates biases that reflect its peculiarities. The drive to measure outcomes prioritizes countable, predictable successes and an audience with executive power, like heads of state, governors and mayors. Grantmaking often coordinates awkwardly with the rhythms of politics, which are driven by elections, administrations, scandals or mass mobilizations. These biases, though, are either tractable problems, or reflect the inherent limitations of philanthropy. The Open Government Partnership grew around heads of state, and is learning to deal with legislative power; participation and social justice are hardly side issues in philanthropy.

If expending political capital isn’t much of a risk, and if philanthropy isn’t lulling open data into an irrelevant corner, the only other risk I see is in inspiration. If political power is seen as best displayed in an Open Government Partnership photo opportunity with heads of state, aren’t we all aiming a bit too low?

I doubt it. Open data activists love FOIA requests. Advocates and activists often started on a path to request, scrape or sue for data because they saw an opportunity to use it to illuminate corruption, reveal shady corporate entities, track human rights abuses or defend speech rights.

The desire to achieve greater of social justice, reduce inequality and drive social change fuel the spreadsheets, policy papers and visualizations that characterize the open data movement. The people strengthening systems using encryption, contributing to massive open source projects, experimenting with collaborative approaches to politics and designing new ways to analyze power are generally not satisfied by tepid incremental improvements, and we need not be too concerned that they will.

A collaborative approach to open data is also responsible for a canonical success in another context: the GTFS standard in public transportation. What Portland, Ore., and Google created has since been adopted around the country and beyond, enabling people to quickly and easily understand their public transit options in multiple ways. The importance of collaboration around setting standards is crucial to our shared future. Convening and discussion between governments, the private sector and civil society are driving more rapid change in more areas than people realize, from restaurant health inspection data to health reports for dialysis centers, nursing homes and hospitals.

The inside and outside approaches to government reform are often contrasted as though they are mutually exclusive. This is a false dichotomy. In our work, Sunlight has both delivered FOIA requests and lawsuits to government officials, and run massive advocacy campaigns on their behalf. Years of often adversarial effort can lead to collaborative processes, and collaborative spaces, in turn, can create the best opportunities for lawsuits, campaigns and basic truth-telling. Sometimes, the most radical advocate is the ignored public servant with a story to tell and some data to tell it, breaking down the line between collaborative and adversarial even further.

Open data advocacy isn’t perfect, and there’s clearly no shortage of work to be done. From expanding audiences, to building inclusion, to balancing the competing interests fighting both for and against disclosure, what “open data” should be, remains a work in progress. But it isn’t being destroyed by collaboration.

“Procurement has gone unaddressed”

Freeing data by adding conditions to government contracts is not just happening: It’s occurring systematically. Over the last five years, we have been asked about this concern by interested public officials so often that we’ve developed a templated response. Similarly, this is part of the Center for Government Excellence’s resource on the same topic.

While this isn’t a singular, global campaign, it is an important part of a systematic effort to build data into the governance processes of dozens of U.S. cities. Organizations like the Open Contracting Partnership are well positioned to scale this even further. It’s worth noting that we’ve seen far more demand from government officials than advocates for contracting language to create open data.

In the face of slowly winning a cultural fight, and as “contracts should require the creation of open data” is broadly adopted as common sense, the bigger barrier will likely be technical. This is part of why we’re so glad people like Waldo Jaquith, founder of U.S. Open Data and adviser to Sunlight, are chipping away at this very problem on a software level. They’re anticipating public officials’ and data managers’ needs and designing systems that bake in openness from the beginning.

“Real change only comes from scandals”

Steinberg is right that many of our most important reforms come from scandals. It’s no accident that one of the amendments to the Freedom of Information Act in 1975 that required a judicial review of executive claims to secrecy followed Watergate. These reforms, though, are also built on years of hard work, credibility, expertise, mistakes, course corrections and relationships. Sometimes newly elected leadership brings change, sometimes the differing priorities of party politics makes new things possible and sometimes the electorate itself shifts. In the U.S., we are going through all of these shifts at the same time, even as we are all adjusting to drastic shifts in technology and journalism that are changing how we think about information, data and public dialogue.

We have all seen how the combination of smartphones, online video and social media have fundamentally changed the dynamic of how nations see interactions between law enforcement and the governed. When the Justice Department finally acknowledged that it needed to improve how it collected data in response to The Washington Post’s Pulitzer Prize-winning journalism, the change demonstrated how data collection and publication can not only hold the use of power accountable but shift the balance towards more accountable systems.

In my time at Sunlight, I’ve repeatedly seen excited conversations become a scraper, a report, a new law, a newly funded project, a presidential speech or a new organization. It would be a terrible mistake to measure reforms and the impact of the broader civic tech movement only against the hundred-year flood that is FOIA.

Compared to our bedrock disclosure law, other advances can seem minor, but their cumulative impact will provide the foundation for all the fights yet to come.

And our data-based fights are just getting started.

The fundamental questions of public life are all increasingly mediated through questions of access to data. All this collaboration, outrage, experimentation and litigation are laying the groundwork for our new definition of publicness. Who and what gets counted, and who gets to know about it, may look incremental, but that’s because these questions have crept into every aspect of our politics (in addition to many other aspects of our lives). Radical questions about the commons, speech and representation are being decided on listserv fights about database licenses, in litigation, and, yes, in photo ops with heads of state.

The shared sense of purpose of the open data movement writ large, contested as it may be, is the social foundation on which these questions will be answered, one way or another. Consider the experience and experimentation we’ll need to answer the following questions with confidence:

• Who gets a voice in our decisions, and how will self-government adapt to the growing influence of a new class of oligarchs and corporations?
• What is the proper role of private platforms in securing public access to governments, and to each other? And government platforms? NGO-run?
• Who gets to see the algorithms that will shape an increasingly automated future?
• Do whistleblowers maintain legal protection in a world of data-scale vulnerabilities and abuses?
• How do we measure fairness in matters of public authority — who gets counted, who gets to count and how are results shared?
• When is regulatory monitoring data public, shared only with governments, or private, as a matter of law or practice?
• In a world of spreadsheets and sensors, how do we protect against the biases baked into our systems as they already exist?
• How will the administration of justice navigate the competing interests of privacy and public oversight?
• To what extent can public policy decisions be mediated through appeals to empirical data, and is our public policy apparatus prepared to take advantage of the information at its disposal?
• What are the contours of our civic space as it intersects with increasingly concentrated wealth and government authority?
• When can technology provide essential support for public participation, and when does it cause unintended consequences (or worse)?

As we live in a world where more and more of these questions are either answerable with data, or are going to be mediated through access to data, we’ll need all the approaches we can get: collaborative, adversarial, institutional and ad-hoc. Insofar as open data succeeds as a movement, though, these questions will cease to be “open data” questions, because the technical considerations will recede into the background, revealing the public policy questions at their core.