Sunlight Foundation

Obama's Open Government Directive, Two Years On

Tomorrow is the two year anniversary of the Open Government Directive (OGD), the signature transparency policy issued by the Obama administration on December, 2009.

The transparency issues facing the administration, both before and after the 2009 policy, fall well beyond the control of the OGD, which is, after all, an OMB memo. Nevertheless, the OGD is Obama's single broadest attempt to create transparency across the executive branch, and the most high profile attempt to live up to Obama's campaign rhethoric on transparency.

Over the last two years, Sunlight has become familiar with the inherent limitations of directives and declarations such as the OGD, as we've learned that the difference between an aspiration and a mandate can be a huge gap. Announcements about new transparency policies imagine the best possible impact, while implementation often looks to the minimum requirements.

So to mark the two year anniversary of the OGD, we decided to look at implementation of the Open Government Directive.  Since much of the OGD is written in broad, aspirational language, we decided to review how well agencies have lived up to the commitments they created for themselves in their open government plans.  The OGD required agencies to publish these plans, which were all posted and revised during 2010, and often included deadlines and goals for agencies to release data and tools.

Building on the work that OpentheGovernment.org did reviewing all the agencies' plans (we participated in that review), Sunlight has pulled out all the deadlines from the agencies' plans, and checked to see whether the goals were met.

The results are decidedly mixed.

In some cases, agencies' goals were clearly met.  Many of the datasets planned to be released are now available on data.gov, and the projects and tools that agencies described are underway.

Often, however, agencies have failed to live up to the standards that they set for themselves as a result of the Open Government Directive.

The Commerce Secretary never put up a schedule. The Office of Science and Technology Policy only put up 4 years of budget data. And the Department of Justice apparently decided that none of the data they identified for public release was fit for publication on Data.gov.  Perhaps most egregiously, the Department of Homeland Security rescinded part of their plan to post a schedule for new data to be released, and released a new version of the plan scrubbed of that milestone.

Any broad declaration or aspirational policy is going to face complex challenges, as we've noted. But the agencies applied these goals to themselves. Far too often, agencies are failing to live up to the transparency goals that they themselves created. And this review only the concrete goals that agencies set. Many agencies didn't even go through the hassle of setting detailed deadlines for themselves. The Defense Department won't show up on most of our evaluation, because they hardly set any deadlines at all, and mostly committed (doc file) to talking about openness and doing some untrackable internal reviews, despite their size and obvious public importance.

We've expressed disappointment in the Open Government Directive before, pointing out, for example, that the Open Government Directive requires:

A strategic action plan for transparency that... (3) identifies high value information not yet available and establishes a reasonable timeline for publication online in open formats with specific target dates.
Cass Sunstein (director of OIRA) once called me Oliver Wendell Holmes for insistently pointing out that agencies who said they'd make some future plans failed to live up to this requirement -- that a plan to make plans was an obfuscation, and that if the White House didn't take this seriously, they'd create the wiggle room that would let agencies evade the White House's best intentions. If he meant that I was being too much of a realist, then I stand by that assessment even more firmly now, as most agencies have clearly failed to comply with that passage.

And that cuts to the heart of the OGD. Openness without information is emptiness.  If some agencies won't even share the plans they've made for publishing new information, how far can their commitment to openness possibly go?

The Open Government Directive has caused a lot of good.  And it has also often failed to live up to its promise, the administration's rhetoric, and agencies' own self-imposed compliance plans. We should remember that Presidential rhetoric and bureaucratic commitments are not the same thing as results, especially as even more administration work happens through broad, plan-making executive actions and plans.

Transparency proclamations are valuable, but the path to transparent government runs through a thousand fights over information. The OGD may have moved the default slightly towards openness, but it doesn't win those fights alone.

We'd like to invite you to review our evaluations of agencies' progress (searching through all the agency sites and plans can be tricky), and to help us think about where the OGD should go from here.

Data released via the Open Government Directive has been put to good use

Policy Fellow Matt Rumsey wrote this post. 

As part of its Open Government Directive, the Obama Administration took steps to make a wide variety of federal data publicly available online. The Open Government Progress Report, released in December 2009 lists open government projects and transparency milestones that support the goals of the OGD. Included among these are a number of important data-sets.

These data-sets have been utilized by journalists, bloggers, organizations, and citizens. They have informed investigative stories and think tank reports, contributed to unique and useful visualizations, inspired projects dedicated to helping the American public make better use of data, and helped to shine sunlight on previously hidden areas of government.

As part of the OGD, the Department of Treasury released IRS Statistics of Migration Data. The data tracks how tax return filers moved around the country and has helped illustrate migration patterns through space and time. The data has been well utilized by a variety of sources. Forbes took the data and created an attractive and easy to use interactive map. Nielsen found that migration from the Northeast to the South and Southwest was correlated with underwater mortgages, while Ad Age focused their analysis on the migration of people as well as money. Additionally, Brookings Institution scholar William Frey used the data as part of a report on recent shifts in migration trends.

The Occupational Safety and Health Administration began to publish employer-specific information about occupational fatalities. This data, intended to help employers identify dangerous conditions and "take steps to improve safety and prevent future accidents," has also allowed investigative journalists to expose unsafe conditions and tell heartbreaking stories. The Center for Public Integrity and Huffington Post took advantage of the OSHA data in a series of articles that expose unsafe working conditions.

Data posted on recovery.gov proved useful in a number of ways.  It allowed interested parties access to information to assess stimulus programs. Data available on the site contributed to reporting on the collapse of Solyndra, helped bloggers in Wisconsin outline benefits to their state, and allowed academics to create novel tools to analyze individual stimulus projects. The nature of recovery.gov also made it easy for critics to weigh in on the shortcomings of the tracking process itself, by identifying areas where data is missing or incomplete.

Data.gov, a central repository for Federal government information, has attracted millions of viewers and inspired major cities, several states, and even other countries to launch sites of their own. Information in the database was used to create some unique and useful applications. AnalyzeThe.US aims to "enable anyone to develop an intuitive picture of the complex flow of resources, money, and influence that affect how our government functions."  Similarly, DataMasher pulls from data.gov and other sources and allows users to compare state level data on various issues.

This post represents a small sampling of the ways that data released via the OGD has been used.  It has proved to be a valuable resource for journalists, researchers, and average citizens.

           

White House Announces Leading Practices Winners

On Thursday, the White House announced the winners of their Leading Practices initiative, that they first outlined in April.

The Leading Practices were designed to highlight examples where agencies have risen above the expectations set by the White House, and proactively attained a higher standard of transparency. (I participated in helping to establish the leading practices standards.)

The winners are a collection of some of the best transparency work being done at federal agencies, with HHS taking the slot for transparency (quite deservingly). These winners are a nice counterpart to the White House page on Open Government Highlights.

As I wrote when the Leading Practices were first announced, though, there is a bittersweet element to this congratulatory platform. As the White House rightly points to the great work some agencies are undertaking, we can't help but wonder whether there is an analagous effort being undertaken with agencies who are struggling with (or blowing off) the Directive's requirements.

While we can hardly expect the White House or OMB to publicly chastise any laggard agencies, we do have to wonder how much of a private stick exists to go along with this public carrot.

Federal agencies drop their data IOU notes

Federal Agencies Drop An IOUCowritten by Laurenellen McCann

When federal agencies released their open government plans earlier this month the thing I was most excited for was new data. While the Open Government Directive didn't specifically require agencies to submit previously unavailable data, 75 new data sets have been promised for public release.

Some of these new data sets have never before been seen by the public. For others you needed to purchase expensive proprietary software. A few only covered a few years and are now being expanded - some to the turn of the 20th century!

There's no single good methodology for determining "what's new" just like there isn't a single good methodology for defining a "single continuous data set". To be clear this is a list of new data sets that are to be released on or after April 7th, 2010. Many government agencies released new data earlier in the year as well. This isn't an exhaustive 2010 list - it's a look into the future.

We've saved you the trouble of going into each plan individually by publishing the spreadsheet below. You can also download the information in a variety of formats by clicking on "Menu" in the upper left corner and then "Download this data." View the data full screen by opening up the "Views" menu:

Open Government Plans: New Data after April 7th, 2010

This spreadsheet is meant to serve as a resource for citizens, journalists and government officials to get a heads up on what data the entire federal gov. has committed to publish. It might come as a surprise that of the 31 agencies that published their Open Government plans on April 7th, only 16 are responsible for proposing the 75 new sets of data. The key word is new. Most government agencies promised to release data tools that were actually aggregators, dashboards, or other web services that run on information you can already find on Data.gov or other agency web sites. Others counted their recent releases of information that is already public and published every year. These data sets did not meet our "newness" criteria. In order to account for what information is genuinely new a data set had to be newly released -- that is, the data must have never before made available to the public online in a freely readable format. It also had be named, evidence that the agency releasing the data is actually initiating the process of opening up this information. Where possible included hyperlinks to additional information about the data set - in some cases it's even the link to the page where the data will be published. This spreadsheet is meant to serve as a resource for citizens, journalists and government officials to get a heads up on what data the entire federal government has committed to publish. Our Reporting team has already been writing about new data coming out of the Open Government plans and pointing out places where work still needs to be done. We're also releasing a single file download for all the federal Open Government plans. Instead of having to round up all 31 files individually, you can just download this single ZIP and get them all in one go. If you're looking for additional information about the Open Government plan of a particular agency, the White House has has published a list of hyperlinks. Beth Noveck, Deputy CTO and Director of the Open Government Directive has published a look into the horizon from her perspective. She also submits her highlights from other sections of the Open Government plan. Our focus has been the data section of the transparency plank - agencies were also asked to develop plans on participation and collaboration. Any ideas on other ways we can make the Open Government Directive more useful to you is welcome in the comments. Be sure to sign our transparency pledge to keep government accountable to their promises! Stay tuned for the week before May 1st - the White House will be releasing their own assessment of the Open Government plans. Photo credit: “Federal IOU” by Laurenellen McCann Photo model: Nicko Margolies

What Open Government plans could learn from retail management

retail

After working several depressing retail jobs in my teenage years, I used to think that it was a kind of job I would never wish upon anyone. After reviewing the open government plans of 29 federal agencies, I'm starting to take a second look at the lessons I learned at those jobs.

For example, it gave me a deep appreciation for the need to conduct occasional inventories of the store: a listing of every single piece of merchandise under the store's roof. In my assessment, the majority of the open government plans failed to provide clear inventories of the "high-value" (a problematic term, as we've discussed before) data.

Department of Commerce - Data Inventory

Most plans gave a general narrative of the type of data that was out there without actually creating an invoice of said data, hyperlinks, citations or even a spreadsheet - in other words, no inventory!

Given the importance of inventories in retail, it shouldn't be a surprise that the Department of Commerce (DOC) provided one of the best data inventories. A screenshot of the inventory including a link back to their open government plan can be found at right.

To give credit where it's due, the General Services Administration (GSA) also had a pretty solid inventory [PDF] (page 55). It's not surprising since the GSA is responsible for acquisition solutions of supplies for many government organizations.

Last week, we devoted a fair amount of digital ink to highlighting the shortcomings of the data in open government plans so I wanted to make sure we continue showcasing the awesomeness of certain aspects of particular agencies' plans. The kudos to the DOC doesn't stop with their data inventory. Clear organization and concise writing typified the DOC’s “What Commerce Will Do” section. It also helps that the plan is written in plain English.

I urge you to read that section in its entirety [PDF] - it starts on page 4. The real star of this section is the National Oceanic and Atmospheric Administration otherwise (NOAA). Factoring out my automatic positive association with the name the new NOAA data being released is absolutely great.

Whether it's digitizing weather station data from the 18th and 19th centuries or making public for the first time soil moisture observation data, the new data from NOAA will improve climate studies and help business make better economic decisions. NOAA was already putting huge amounts of data online, even before the Open Government Directive. Recognizing that the data is sometimes hard to find, NOAA is also expanding the scope and functionality of its Climate Services Portal to help citizens and scientists find the data they need.

The Sunlight Foundation has been focusing its eye on the transparency plank of the open government plants, specifically on data transparency. We'll continue to do so this week but it's important to note that transparency is only one of three Open Government Directive planks: the others are participation and collaboration. Agencies were also asked to come up with an open government flagship initiative. Heather West of the Center for Democracy and Technology has a great post on Govfresh highlighting certain flagship initiatives.

We'll continue to dig deeper into the transparency portion of the open government plans and link to other evaluations going up round the net. If you see a perspective on the plans we've missed drop it in the comments below!

Photo credit: "Discoveryland Retail Packaging" by Flickr user Design Packaging.

White House Announces Leading Practices

CTO Aneesh Chopra blogged an announcement yesterday laying out the Administration's next steps after the April 7th milestone.

The Adminstration’s transparency initiative, though, was designed to fail well. This isn’t the same old OMB memo. If it were, we’d all be calling it OMB Memo M10-6, its real name.

There are several announcements here packed into one.   Chopra announced the updated White House dashboard, and set a May 1st goal for a White House evaluation of agencies' plans.  He also invited everyone to dig into open government plans, and suggest improvements and offer feedback.  This is an important invitation, and one we're going to continue to take advantage of.  The plans from April 7th can really be a version 1.0 if they get updated and improved, which takes effort and feedback.

Third, and the focus of this post, Chopra announced "leading practices" guidelines, that set out goals beyond the minimum goals defined in the Directive.  I'm happy to have had a small role in shaping the policies the Open Government Working Group created, which basically means discussing ideas and approaches for defining excellence in transparency.

These guidelines are a big deal.  I wrote yesterday about how it's difficult, right now, to distinguish between meaningless bureaucratic language (on one hand), and authentic, legitimate planning as it occurs within a huge institution.  Are agencies stalling, or are they bringing about change in the ways they know how?

These leading practices make that distinction matter less.  Even if some agencies don't take the Directive seriously, these guidelines were built to move.  And by move, I mean forward.  The Directive, and the enthusiasm it has created among the agency officials who have caught the transparency bug, could just peter out, if the administration issued the Directive and said "Great! We're done!"

That's clearly not what is happening, however.  In addition to the day one Memo, Executive Orders, multiple Memoranda, the public consultation, the Directive, and agency plans, agency officials are being encouraged to think of ways that the Directive hasn't yet gone far enough, and then those results are being published on WhiteHouse.gov.

We're certainly disappointed in the way that many agencies' plans complied with the Directive's data-related requirements.  The Adminstration's transparency initiative, though, was designed to fail well.  This isn't the same old OMB memo.  If it were, we'd all be calling it OMB Memo M10-6, its real name.  (That's not an overstatement, many of us refer to A-130 as though it were a fun nickname.)  Instead, the "OGD" (Open Government Directive) is shorthand for what is, to me, several hundred conversations, plans, meetings, processes, and official documents.  Most importantly, it's not something that has ended.

Criticism of the Directive, and its results, aren't met with a "well, you should know we Worked Hard on this, and have some complex things to balance!"  Instead, they're met with "that's what we're looking for, more please."  That's a very important difference.  Defensiveness results from a static process, one that is considered done.  Welcoming criticism, though, is the hallmark of an evergreen work in process, where getting credit isn't as important as getting to the right answers, no matter how long it takes.

These leading practices could be interpreted as an admission that the Directive's requirements don't go far enough.  And in a way, that's true.  But it's also by design.  Technology has changed what's possible quickly enough that no one has all the right answers.  Making the government's work truly public (read: online) will take laws, regulations, guidance, procedures, and experimentation (in order from strict to loose), all backed up with lawsuits, criminal penalties, admonishments, encouragement, recognition, and prizes (from negative to positive).

By pursuing an evaluation process that focuses on the positive, the White House is, in part, choosing to pair the public carrot with the private stick, as is to be expected for government reform efforts.  They're also giving up on any claims to having all the best answers out of the box.  In place of that, they get something more important -- a process that, if they continue to pursue it openly, will lead most reliably to the best answers.

Open Government: idling in the driveway

Sigh. I feel like a disappointed parent.

When the details of the Open Government Directive were announced early last December I was unbelievably excited. Seriously. My long time hope that one day government would get “it” about the importance of putting public information online appeared to have arrived. Government data was going to become available as a default and that was going to start with an “inventory” (government's word) of the “high value information” (also their words, though less than ideal because who would ever agree what that means?).

Agencies were supposed to do two things with respect to releasing data: create an inventory of the “high-value information” currently available for download and identify high value information not yet available along with establishing a reasonable timeline for publication of that data online. It was that latter requirement that I salivated over. Certainly there are other important aspects of Open Government -- participation and collaboration are values we hold dear at the Sunlight Foundation. Car IdleBut yesterday was the day when the rubber was supposed to hit the road on data. For many agencies, they didn't even get out of the garage.

There are some very interesting data that's going to made available, almost immediately (and John Wonderlich, our Policy Director, has a post on it) but some agencies avoided the requirement entirely, some decided to say they'd make a plan to plan how to identify and release data, and others mentioned it but didn't explain how they would achieve it.

First, our quick review shows that a little more than half of the 30 agencies' plans we reviewed (18) specifically identified new data to be released -- 12 did not. (This includes some independent agencies.) The total number of data sets identified to be released -- approximately 89.*

89 data sets identified for release - across the entire federal government!? I'm speechless. I was looking for inventories of data (this is the Directive's word, after all) -- actual audits of what data each agency collects and dates of when new information would be made available. That is not what we got.

The Department of Health and Human Services (HHS) was among the best - identifying 14 new data sets to be released - and this is crucial data. While maintaining the privacy and identity of patients HHS will be releasing critical data about Medicare: everything from inpatient hospital stats to prescription drugs and hospice care. During an era on increased responsibilities for HHS this data is absolutely critical to keeping HHS effective and accountable.

Few agencies rose to the high water mark of HHS. Part of the problem might be attributable to cultural barriers and the illusion that some bureaucrats hold that this is "their" data vs "all of our" data. Part of the problem might have been time to pull the information together.

Maybe, a bigger part of this problem is a loophole in the Open Government Directive itself. By asking agencies to only inventory their "high-value" data it gave them an instant out for just about anything. Despite the White House's good intention in defining high-value as: "increase agency accountability and responsiveness; improve public knowledge of the agency and its operations; further the core mission of the agency; create economic opportunity; or respond to need and demand as identified through public consultation."

With a definition like that "high-value" could mean literally anything: if you collect a piece of data that is not to "further the core mission of the agency" why are you collecting it?

When you define a concept too broadly you end up not defining it at all. If we could roll back the clock on the Open Government Directive we would ask agencies to first list all data they collect and then create sub-lists of:

  • data that is currently public but not online
  • data that is currently public and online
  • data that is not currently online but that will be put online and when
  • for everything else, explain why it won't be put online
This would give us an instant picture of what the online (and therefore, public) landscape of federal government looks like and is an invaluable data set in its own right.

HHS, NASA, Education, National Archives and Records Administration and the Office of Personnel Management were the high water marks.

Defense, Homeland Security, Justice, State, Interior, Treasury, Veterans Affairs, US Agency for International Development and the Social Security Administration did not identify any new data to be made available - no inventories either.

Yes, I appreciate the extraordinary hard work put into the Open Government Directive by all those in the agencies and those spearheading it at Office of Management and Budget and the White House, and I wouldn't suggest that evaluating these plans based on just one of a couple dozen appropriate criteria is a totally fair reading of how successful this exercise was, but I have to look at it from what I feel is key for government accountability - data. That's my lens on the world.

We'll continue to evaluate agency plans all next week.

NOTES:

*We arrived at the 89 number via a very generous methodology. It all depends on how you define a "data set". Our complete inventory using a more exact methodology will be available soon.

Photo credit: "Idling" by Flickr user N1NJ4.

Agency Compliance with Data Requirement Mixed

I wrote yesterday that we'd be going through agencies' new open government plans to evaluate their compliance with the data-related requirements. Here's what we've found so far:

Agencies have generally met the requirement to inventory their high value data that is currently available for download.

Many agencies, however, have not identified high-value datasets with specific goals for when they will be released.  This is the requirement I'll focus on in this post, since it shows how differently agencies are interpreting some of the Directive's requirements.

By my count, 18 of the 30 agency plans I checked fulfilled this requirement -- to identify high value data for future publication, leaving 12 agencies that did not.  To follow my evaluation of each agency's inventory of new data, here's the Google Spreadsheet I used to record my observations.  (Commerce's plan was linking to an error page as I checked through them.  Otherwise, I followed the agency list as available on the White House Open Government Dashboard, and used the helpful links to plans the GovLoop compiled here.)

The requirement in question from the Directive is in the section defining the components of the plans:

A strategic action plan for transparency that... (3) identifies high value information not yet available and establishes a reasonable timeline for publication online in open formats with specific target dates.
Importantly, the requirement is not a requirement for a strategic action plan for identifying information, but a strategic action plan that identifies information.  In other words, as I wrote yesterday, a plan to plan is insufficient.

Many agencies plans agree with this interpretation -- requiring specific identification of data that they'll be making public, with dates for when that will happen.  Some even linked their inventory of new data to the part of the Directive that requires it.

When agencies did fulfill this requirement, some did so questionably, and some quite admirably.  Some agencies are at the margins of compliance, like DOT, that thematically mentions kinds of data they'll consider publishing, or the Department of Labor, that specifically mentions a number of new datasets, but qualifies them by saying they're "examples of the types of data" that they will publish.

Agencies with the best lists of new data they'll publish include the NRC, with a very detailed appendix, NASA, with precision, and HHS, with 14 new datasets.

Of the agencies that didn't comply with this requirement, perhaps DoD is the most disappointing, since they encompass so significant a portion of our federal budget, and have an enormous number of datasets to choose from.

Despite the mixed compliance with this specific provision of the Open Government Directive, the Directive itself continues to deliver an enormous seachange in how our federal government relates to its information, and to the public at large.

If some agencies have not delivered this particular requirement, that means it's time for everyone -- agencies, the White House, and the public at large, to work towards a comprehensive accounting for our government's information.  If fully implemented in good faith, even DoD's plan for an eventual inventory of their data could prove to be as fundamentally transformative as anything else the directive requires, releasing an empowering deluge of vital national information.

Having mixed or varied compliance with key components does make it harder to evaluate agencies' performance.  It isn't a trivial task to differentiate between legitimate struggling with complex problems (on one hand), and meaningless bureaucratic defense against fundamental change (on the other).  Clear expectations are one antidote against this problem.  Another is continued effort on the part of both the White House and all the agencies.

As White House and agency officials keep referring to these plans (and the Directive) as Version 1.0.  This is entirely appropriate for an ongoing, complex task like remaking our federal government's relationship to public information.  This appeal to an iterative approach, and to "living documents," however, can also be a sign of softening expectations.  To live up to the potential of 21st Century disclosure rather than settling for less, we've all got a lot of work to do.

Comprehensive, aggressive inventories of public data are a great place to start.

Agency Plans and Data

As I wrote earlier today, Sunlight is going to be particularly focused on data transparency in the new Open Government Plans, which are expected to be released today.

We're focused on this element of the plans -- how agencies inventory and plan to release data -- because it's important, and because the government has repeatedly failed to do it effectively in the past.  Broad data access, and the agency plans intended to create it, lie at the heart of the empowerment and accountability the Directive is intended to create.

Asking "What is knowable about this institution and its work" is extremely powerful.  The government has never provided a sufficient answer to this question, despite past (and current) laws and initiatives that require it.  That's why we plan to hold the agencies and the administration to the highest standard -- to create broad, systemic awareness and access to government information that has up to now been elusive.

The clarity and detail agencies use to inventory and plan for future data release will signify, in part, whether specific agencies view the directive as an administrative exercise, or a transformative initiative. First, here's what the directive requires:

A strategic action plan for transparency that (1) inventories agency high-value information currently available for download; (2) fosters the public’s use of this information to increase public knowledge and promote public scrutiny of agency services; and (3) identifies high value information not yet available and establishes a reasonable timeline for publication online in open formats with specific target dates. High-value information is information that can be used to increase agency accountability and responsiveness; improve public knowledge of the agency and its operations; further the core mission of the agency; create economic opportunity; or respond to need and demand as identified through public consultation.
First and foremost, we're interpreting this provision to mean that agencies' plans (released today) must contain an inventory of data currently available for download, and must contain specific plans for data to be released in the near future, with specific timeframes.  A plan to release a plan for a procedure for an inventory won't cut it.

The directive doesn't require a comprehensive list of all public information holdings.  The first requirement, "A strategic action plan for transparency that (1) inventories agency high-value information currently available for download;" has two significant qualifiers.  First, that the inventoried information be "high-value", and second that it be "currently available for download."  I expect this to be interpreted to mean "those datasets that are available now on Data.gov," although a strict reading would imply other data available in bulk as well.

This first requirement echoes past initiatives, like the Federal Information Locator Service, which was replaced by the Government Information Locator Service (44 USC 3551).  This requirement has been almost completely ignored, although the GILS database still exists.  It also echoes similar requirements from the e-government Act of 2002, though they are more vague and ineffectual.

We proposed a definitive solution in the POIA bill, which would require agencies to publish machine-readable lists of all of their public information holdings, and is enforcable through a private right of action.  These and similar requirements, created to generate a comprehensive public accounting of public information holdings, will only succeed if a high standard is set and agencies are held to it.  True success in this enterprise would be wildly transformative, and empower governance, oversight, and reuse of government information.

The second requirement deals with future action.  Agencies must create a "A strategic action plan for transparency that ... (3) identifies high value information not yet available and establishes a reasonable timeline for publication online in open formats with specific target dates."

This requirement means that agencies must identify information for future publication.  Again, plans to plan are insufficient.

This requirement also has two major qualifiers.  First, list of data to be released in the future need not be comprehensive.  It applies specifically to high value information.  High-value information (defined in the beginning of this post) is a useful list of ways that data can be valuable, but does little to strengthen this requirement.  In fact, in weakens it, by adding a significant qualifier to the set of datasets agencies must plan to release.

Similarly, "not yet available" could be interpreted to mean "not available at all to the public," or could mean "not yet available in a structured or bulk form."  I expect agencies to interpret this to mean both definitions, although they should distinguish between data that is gettting a format upgrade, and data that will be released for the first time.

Despite those two qualifiers, this provision gives agencies their chance to shine.  Their role as important stewards of some aspect of the national interest involves our vital information, and the Open Government Directive is intended to create access to that data.  We're hoping to be impressed by what agencies identify for publication.  The scope of what is knowable should grow.

In fulfilling this second provision, agencies should be as comprehensive as possible, be clear about what they're releasing (is it new or not?), and it should reflect agencies' priorities.  There are easy and hard datasets to release.

In creating the "high-value data" standard for what agencies should release, the administration attempted to set some priorities for what should be made public.  Ultimately, that "high-value" definition doesn't serve that purpose well.  It's a great list of ways that public data can be valuable, but doesn't really help agencies to decide where to put their effort first.

Sunlight is particularly interested in what we're thinking of as accountability data.  Accountability data is information about an agency and its functions, or about those entities that report to the agency, that allows those entities or the agency to be held accountable.  While that's a somewhat tautological definition, the distinction is important.  Some information has enormous transformative potential, and empowers oversight and accountability.  This information also can be the most embarrassing, or the hardest to relinquish control over.  That's where we want to see agencies' attention -- on fixing tough problems.

Up to now there has been a bit of a priorities vacuum, where publicizing information on the Internet has happened in an ad-hoc manner, in response to laws (albeit unreliably) or often because it serves an agency's or department's self-interest.  As we move beyond a request-based FOIA world of data access, and achieve more affirmative publication, we're going to need new distinctions about what can be public, and new priorities for where to focus our energy.  Accountability data is our first priority, and should be agencies first priority as well.

That's where we're headed.  For now, we're anxiously awaiting agencies' plans.  We'll be looking through their data plans in the coming hours, and evaluating them in the terms I've described above.  Whatever agencies deliver today, we'll be working to create broad systemic access to their information, and to create the tools and practices necessary so that we can all benefit from it.

OGD /Open Pages Come to Life

Meeting or surpassing the White House's deadline, the 20 agencies we monitored launched their /open pages by this past Saturday in accordance with the Open Government Directive. (See Sunlight Lab's /open page tracker; also ProPublica's transparency tracker).

The White House also revealed its Open Government Dashboard, which monitors 29 agencies for compliance with the OGD. The timely creation of this Dashboard fulfills another promise contained in the OGD.

What's notable about the White House's Dashboard is that it helps hold agencies accountable by identifying the 4 agencies that have yet to fully comply with the OGD: the Nuclear Regulatory Commission, the Office of Personnel Management, the Council on Environmental Quality, and the Office of the US Trade Representative, in addition to indicating the 25 that have fully complied. These 4 agencies have yet to release all three high-value data sets. The Council on Environmental Quality receives an additional warning light for having failed to assign a senior official to ensure data quality.

We are in the preliminary stages of looking at what's on these /open websites, with four questions initially springing to mind:

  • Are they making progress towards developing their Open Government Plan?
Both Intellitics and the General Services Administration have put together links to each agency's pages (and RSS feeds) for gathering feedback and engaging in discussion with the public about the Plan. GSA has also added additional means of contact, specifically email and postal addresses.

In addition, GSA has there is an additional tool available: a wiki -- the open government playbook -- that aggregates a lot work done by the government and those outside the government on transparency. Its purpose is to "serve as a useful directory to [OGD] resources" -- and invites everyone from government officials to members of the public to contribute. It is a great idea. (We had previously gathered all of our OGD resources on our separate wiki page.)

We'll have a lot more to say in the upcoming days and weeks, particularly as we dig into the /open pages and the OGD Plan.

Before moving on, I must note that the White House took a risk in publicly setting a deadline for creating these /open pages. Meeting this deadline may not result in a lot of favorable media attention, but had agencies failed to do so, the White House may have been subject to a lot of criticism. The architects of the OGD deserve credit for taking a risk, for being willing to risk public failure in order to make something good happen. At first glace, they made it happen. Congratulations.

We're going to take a hard look at these webpages to see how well they satisfy the details of the OGD, what improvements should be made, and evaluate the emergence of the Open Government Plan over the next 60 days. There's a lot more to do, but Saturday marks an important milestone.

Updated: the wiki is linked to from GSA, but is not run by them. I've been lead to believe that Lucas Cioffi is one of the volunteers leading the wiki effort.

« Previous
1 2 3