Getting Past Groundhog Day: Auditing the Government’s Open Data Policy Delivered at the American Council for Technology Daniel Schuman February 14, 2013

Available in PDF here.

In the movie Groundhog Day, the lead character, played by Bill Murray, finds himself living the same day, Groundhog Day, over and over again. Each morning, he wakes up to Sonny and Cher on the alarm-clock radio singing “I’ve got you babe,” and no matter what he does during the course of the day, when he wakes up again, it is that very same day, with Sonny and Cher.

When it comes to Open Data Policies, today is Groundhog Day. We’ve come a long way since the Paperwork Reduction Act, first passed in 1980, which called on each agency to ensure that the public has timely and equitable access to the agency’s public information; and things have changed since OMB’s Circular A-130, first issued in 1985, which calls for the protection of the public’s right of access to government information. We now have websites, bulk downloads, APIs, and a shiny new Digital Government Strategy. But when it comes to how agencies share access to the information they hold, we’re barely halfway through the movie.

At its core, the problem here isn’t with the intentions of the agencies, or of the government, but with the approach that’s been taken thus far. Newly announced open data policies usually pertain to particular data sets, cherry picked by the government. We often find ourselves in the circular discussion where the open-minded government officials say: “what information do you want,” and the public responds, “well, what do you have?”

It’s that key question — what do you have — which is why every day is Groundhog Day. The public doesn’t know what information the government holds. Other government officials often don’t know what information a particular agency holds. And sometimes, people sitting next to each other in an agency don’t know what data is being generated by their office mates. Even those responsible for the agency’s information needs — the CTO and CIO — are so busy putting out fires and their day-to-day responsibilities that this information is not readily at hand.

I suggest that there is a fundamental and irreducible set of questions: what information does an agency hold, where can it be found, who is responsible for it, and how is it shared? These questions can be answered with an agency-wide information audit and the creation of an information index. Agencies should survey their holdings and find out what information they collect, where it’s held, and how it’s held. And then they should put that list in an index that they release to the public.

The President’s Memo on Regulatory Compliance Data required something along these lines when it directed agencies to identify their activities and inventory their data. Of course, only one agency, the Department of Transportation, actually complied, with a fantastic list, released in May 2011.

DOT published a list of information already available to the public, a list of information that’s available to the public but where release practices could be improved, and information not currently available to the public. Each of these three categories was further broken down in a number of useful ways, including cataloging where and how the information is made available, and to whom.

This is something each agency should do. In fact, this is something each agency has been required by law to do — for example, through Congress’ establishment of a government information locator service in the 1990s — but this rule largely has been ignored. This is a shame and it’s a mistake.

All of us spend a lot of time discussing how information is made available to the public, but the first step is to know what information is gathered and retained by agencies. This empowers people inside and outside of government to be able to make real decisions about what information to release, to request, and to withhold. It was only through understanding that Bill Murray escaped from his never-ending Groundhog Day, and we will only move past our Open Data Groundhog Day when agencies gain the same kind of self-awareness that allows them to meet their own needs and the needs of others.