I wrote yesterday that we’d be going through agencies’ new open government plans to evaluate their compliance with the data-related requirements. Here’s what we’ve found so far:

Agencies have generally met the requirement to inventory their high value data that is currently available for download.

Many agencies, however, have not identified high-value datasets with specific goals for when they will be released.  This is the requirement I’ll focus on in this post, since it shows how differently agencies are interpreting some of the Directive’s requirements.

By my count, 18 of the 30 agency plans I checked fulfilled this requirement — to identify high value data for future publication, leaving 12 agencies that did not.  To follow my evaluation of each agency’s inventory of new data, here’s the Google Spreadsheet I used to record my observations.  (Commerce’s plan was linking to an error page as I checked through them.  Otherwise, I followed the agency list as available on the White House Open Government Dashboard, and used the helpful links to plans the GovLoop compiled here.)

The requirement in question from the Directive is in the section defining the components of the plans:

A strategic action plan for transparency that… (3) identifies high value information not yet available and establishes a reasonable timeline for publication online in open formats with specific target dates.

Importantly, the requirement is not a requirement for a strategic action plan for identifying information, but a strategic action plan that identifies information.  In other words, as I wrote yesterday, a plan to plan is insufficient.

Many agencies plans agree with this interpretation — requiring specific identification of data that they’ll be making public, with dates for when that will happen.  Some even linked their inventory of new data to the part of the Directive that requires it.

When agencies did fulfill this requirement, some did so questionably, and some quite admirably.  Some agencies are at the margins of compliance, like DOT, that thematically mentions kinds of data they’ll consider publishing, or the Department of Labor, that specifically mentions a number of new datasets, but qualifies them by saying they’re “examples of the types of data” that they will publish.

Agencies with the best lists of new data they’ll publish include the NRC, with a very detailed appendix, NASA, with precision, and HHS, with 14 new datasets.

Of the agencies that didn’t comply with this requirement, perhaps DoD is the most disappointing, since they encompass so significant a portion of our federal budget, and have an enormous number of datasets to choose from.

Despite the mixed compliance with this specific provision of the Open Government Directive, the Directive itself continues to deliver an enormous seachange in how our federal government relates to its information, and to the public at large.

If some agencies have not delivered this particular requirement, that means it’s time for everyone — agencies, the White House, and the public at large, to work towards a comprehensive accounting for our government’s information.  If fully implemented in good faith, even DoD’s plan for an eventual inventory of their data could prove to be as fundamentally transformative as anything else the directive requires, releasing an empowering deluge of vital national information.

Having mixed or varied compliance with key components does make it harder to evaluate agencies’ performance.  It isn’t a trivial task to differentiate between legitimate struggling with complex problems (on one hand), and meaningless bureaucratic defense against fundamental change (on the other).  Clear expectations are one antidote against this problem.  Another is continued effort on the part of both the White House and all the agencies.

As White House and agency officials keep referring to these plans (and the Directive) as Version 1.0.  This is entirely appropriate for an ongoing, complex task like remaking our federal government’s relationship to public information.  This appeal to an iterative approach, and to “living documents,” however, can also be a sign of softening expectations.  To live up to the potential of 21st Century disclosure rather than settling for less, we’ve all got a lot of work to do.

Comprehensive, aggressive inventories of public data are a great place to start.