Enterprise data inventories should be open — not hidden from the public

city skyline in data drawing
Image credit: iStockPhoto

Over the past several years, the Sunlight Foundation has spent a lot of time talking about the need for governments to create comprehensive lists of their information holdings — and to release those lists to the public.

You can expect more of the same in 2015.

In our Open Data Guidelines, we argue that “for an open data policy to have a strong foundation, you first need to know what data you have — and so does the public.”

The federal government embraced the first part of this in 2013 when President Obama released his open data executive order. Project Open Data, the effort to implement the executive order, requires agencies to create and maintain “enterprise data inventories,” comprehensive lists of an agency’s information holdings. The enterprise data inventories share metadata about agency data assets, but do not themselves reveal the content of data sets that aren’t already public.

Unfortunately, these lists are not required to be made public. Instead, agencies only have to list the data that they are willing to make public. This is a problem because without access to full lists of agency data sets — or even just the names of those lists that contain otherwise private information — the public will not be able to understand the full scope of government information holding, and will be limited in the ability to hold it accountable.

Indeed, there is no clear way for the public to see what information the government has decided not to release, or why they have decided not to release it, shy of submitting a request under the Freedom of Information Act for each individual listing and fighting for its release. But how can you FOIA for something you don’t know exists?

One might be surprised how often proactive government actors ask — with much-appreciated sincerity — what data they should release next. Our stock response: How would we know?

Sharing an agency’s enterprise data inventory won’t expose any information that should reasonably be withheld from public view. It would, however, provide an easy way for interested parties to understand government disclosure decisions, while also showing the public what data these agencies actually hold.

We believe that there is a strong public interest in opening the enterprise data inventories. To that end, we submitted a FOIA request in December 2013 to the Office of Management and Budget (OMB) for all of the agency enterprise data inventories.

And then we waited.

Not content to sit on our heels while the gears of bureaucracy ground slowly on, we also began urging various agencies to consider releasing their enterprise data inventories to the public.

Eventually, OMB denied our request. Instead of citing a specific exemption, they told us to contact every agency individually.

At least 24 agencies were required to complete enterprise data inventories. It seemed to us that reaching out to at least 24 agencies instead of working through just OMB was a bit excessive, especially since the inventories are already required to be submitted to OMB each quarter. But, we did continue to evangelize our position to agencies when we had the chance and sent FOIA requests to four agencies (a slightly more manageable number), hoping that someone would decide to release this vital information.

So far, only one agency appears to have taken us up on the offer, the Department of Transportation (note all of the “non-public” data sets in that list). We are particularly happy to see this leadership from the DOT, which handles a host of potentially sensitive information about our nation’s infrastructure and transportation systems. It recognizes the positive power of making this information widely available.

The DOT is proof that agencies don’t have to be afraid of letting the public know what they know, even if the data itself must be withheld. We hope more agencies will recognize this and move to release their enterprise data inventories, but we recognize not all will do so. Sunlight will not hesitate to use our right to information under the FOIA to continue pushing for release of this vital information.

We have been fighting this fight since at least 2011 and aren’t looking to give up any time soon. We believe that the Obama administration has done a great public service with its open data efforts, but this is the next necessary step.