I recently stumbled upon a Federal Drug Administration report issued in response to President Obama's Regulatory Compliance Memo. The May 2011 report chronicles the agency's transparency efforts since 2009 as well as outlines goals for making additional data available.

A look into the FDA's website found mixed results as to the implementation of their stated goals at the time of this writing. The goals were to post the following on their website:

• Information about all the inspections the agency conducts related to market products, rather than the current disclosure of piecemeal information
• A significantly expanded set of enforcement letters
• FDA evaluations of importers
• Information about each case the Department of Justice files on FDA's behalf at the time the case is filed and when the case is resolved.

We were able to find the information specified in the first three goals - import filer evaluation outcomes, enforcement letters, an inspections database, a current list of press releases when DOJ cases are resolved, and a list of cargo thefts. However, we were unable to find any information regarding ongoing criminal investigations or recently filed cases.

The report details all the regulatory compliance data the FDA manages and breaks out which pieces are already publicly available. The list of publicly available data includes import refusal reports, recalls, annual enforcement statistics, and warning letters. The report also describes the agency's transparency efforts to date, which include forming an internal Transparency Task Force, posting a number of datasets on Data.gov, and launching a redesigned webpage and phone app.

Last month I looked at the report the Department of Transportation issued in response to President Obama's January 18, 2011, Regulatory Compliance Memo. I found that while their report was impressive, there were only three DOT datasets listed on Data.gov. In less than a week after my post DOT posted numerous data sets and Data.gov lead Jeanne Holm publicly thanked Sunlight for pointing out the error.

Only two other agencies have chosen to make their reports publicly available - the Environmental Protection Agency and the Department of Commerce. Neither of these two plans are as clear or as comprehensive as the DOT plan, and their approach to Data.gov is very different.

The EPA has 1,671 total datasets on Data.gov, and all but one of the seven resources they list in their regulatory compliance report as being either on Data.gov or in the process of being put there are easily findable on the platform. The datasets were put up between February and March of 2011, and each has had between 84 and 1,010 visitors. The resources are just links to EPA websites hosting the data, however, which means that the data cannot be downloaded in bulk or manipulated within Data.gov. Lastly the EPA's FY2011 goals are too broad to effectively measure at this time, unfortunately.

Although the EPA report is not as precise or as detailed as it could be, the Commerce report does not even mention Data.gov. The agency has just 48 datasets on the platform; none of which are even mentioned in the report.

Neither of these plans provide us with substantive ways to measure the amount of data agencies are posting on Data.gov. I'd like to have a look at all the other plans, but unfortunately those agencies have not decided to make them public.

I am so pleased to see the Department of Transportation and Data.gov being so incredibly responsive to the concerns we brought up regarding the lack of DOT data posted on the Data.gov platform. While we were initially disappointed that the data sets had not been posted in a timely manner as promised, it's nice to see those in charge accepting the responsibility and taking immediate steps to fix it.

Not only has DOT posted numerous data sets less than a week after we initially brought the data discrepancy to light, but we have been publicly thanked and thoughtfully responded to by people from Data.gov and DOT.

While speaking on a panel at the Open Government Data Camp 2011 in Poland, Jeanne Holm, the Communications and Collaborations Lead for Data.gov, publicly thanked Sunlight for pointing out their error and giving them the necessary push to get the DOT data uploaded.

Daniel Morgan, a contractor who works on DOT and Data.gov, responded via Twitter, indicating that even more data sets are on their way.

This case study has demonstrated the impact and improvement that can happen literally over night when government entities are responsive. One not-so-simple investigation by an outside watchdog was all it took, and DOT and Data.gov significantly increased the accessibility of 80 DOT datasets in the space of 24 hours.

Judging on the page views as recorded by Data.gov, the 80 new datasets collectively received 73 views in the first 24 hours they were online, and 335 views within a week. That right there is a tangible, meaningful, measurable impact. And it's only been one week.

The other three data sets, which have been on Data.gov since February 18, 2011, have collectively had almost 10,000 views. Now there are 80 data sets. Were the ratio of data sets to page views to remain the same, that would suggest that within the next eight months the new data sets will have gathered 270,000 page views.

I'm not necessarily saying that will happen, but it could not be any more clear. People are using Data.gov, and we need to ensure that it remains a top priority for funding in FY2012.

Yesterday I wrote about my investigation into whether the Department of Transportation had put their data on Data.gov as promised months ago, and found, to my surprise, that they actually were not following their own guidelines or deadlines, and were not in compliance with an executive memo.

This was extremely disappointing, as we initially hailed DOT as setting a high example and being an impressive leader with the timely, clear report they issued in response to President Obama's Regulatory Compliance Memo.

In the 24 hours after my post, DOT has increased the data sets located on Data.gov from three to 83, which is much closer to the 89 total data sets they have listed in their report. My initial reaction is simply: wow, those people over at DOT are certainly some fast responders! 

I am excited to take a closer look and see what the overlap looks like between those now on Data.gov and those listed in the report. Stay tuned for updates!

The Department of Transportation has been a leader in complying with recentopen government initiatives, so I decided to use the agency as a case study to measure the amount of data agencies are posting on Data.gov. I found some pretty surprising things - namely, that they are not in compliance with a presidential memo, their own reporting pledge, or even their self-imposed deadline.

Backing up for a minute, President Obama's January 18, 2011, Regulatory Compliance Memo mandated, among other things, that:

First, agencies with broad regulatory compliance and administrative enforcement responsibilities, within 120 days of this memorandum, to the extent feasible and permitted by law, shall develop plans to make public informationconcerning their regulatory compliance and enforcement activities accessible, downloadable, and searchable online.... Second, the Federal Chief Information Officer and the Chief Technology Officer shall work with appropriate counterparts in each agency to make such data available online in searchable form, including on centralized platforms such as data.gov, in a manner that facilitates easy access, encourages cross-agency comparisons, and engages the public in new and creative ways of using the information.

The DOT was among the first agencies to release the required report, which they did in May 2011. The report is succinct, clear, and public (unlike some of the other plans), with detailed charts identifying information already available to the public (47 data sets), information already available but where release practices could be improved (16 data sets), and information not available to the public at all (26 data sets). The charts conveniently include exact URL locations for the information that is available. We even based some of our suggestions for the Open Government Partnership's U.S. National Action Plan on DOT's plan.

But what if you were looking for this data without the help of the "Department of Transportation Draft Preliminary Plan for Implementation of The President's Memorandum dated January 18, 2011 Regulatory Enforcement and Compliance Data May 2011" report?

You might try looking on Data.gov. While in theory that should be a good choice, you would not be successful.

There are just three DOT data sets listed on Data.gov, not one of which overlaps with the 89 data sets listed in the DOT report. While perhaps it could be argued that not all of the 89 information sources listed in the report are actual data sets, this is still a striking discrepancy.

Confusing? You bet.

The report states that the "six DOT modes that have broad enforcement and compliance authority: 1) FAA; 2) FMCSA; 3) FRA; 4) NHTSA; 5) OST; and 6) PHMSA as higher priority for upgrading and including in data.gov." When I looked at each of these data sets in turn, there were no data sets available for any of these "higher priority" modes. Of the six, only NHTSA had any data sets available.

The report also states that:

As part of this effort, we linked up modal enforcement and compliance personnel with our DOT Data.gov working group to: A) Determine if the information or data sets comply with DOT Order 1351.34, Departmental Data Release Policy, by ensuring data are identified and registered in the Department’s S/DAG Metadata Registry (our choice of a central platform, http://www.data.gov.)

I went to check on DOT Order 1351.34, and quickly found out that it is not available anywhere online, despite being a public document. After a lengthy process I was able to retrieve a copy through using DOT's online contact tool.

• The Reference Desk responded, sending me a direct link to the only instance on regulations.gov where "DOT Order 1351.34" was found. This turned out to be just a link to the DOT Regulatory Compliance Report, where I had originally found the order referenced.
• Another back and forth provided me with a link to the order, accompanied by a comment from their Reference Desk, "I think I got it. Took a while..." This link turned out to be internal-only and would not load.
• Another back and forth and the Reference Desk explained that they had a PDF they could send me which was labeled "public," but "that might just mean that the entire DOT is allowed to see it," so they had to check.
• After checking internally, they sent me a PDF of  the order.

And let me say a huge thank you to those people over at DOT's Reference Services, for patience and perseverance on my behalf. 

But it should not have been that hard.

According to the order:

Data may be released through DOT Web sites. When data are released through a DOT Web site, such data must also be disseminated through one of the e-Government initiatives identified in Section 34.4.5.2 below.

The order then lists options for the e-gov initiatives where data can be disseminated:

• FedStats.gov,
• Data.gov,
• Geodata.gov,
• USASpending.gov, or
• ITDashboard.gov

As the report clearly states, DOT chose Data.gov to host its regulatory compliance data. The report also sets a deadline for when data sets should be placed on Data.gov: "the end of Fiscal Year 2011." That's September 30, 2011.

It is now October 18, 2011 - nine months after the memo was released, five months after the DOT report was released, and almost a month past DOT's self-imposed deadline. Based on the standards established in President Obama's Regulatory Compliance Memo, DOT's Regulatory Compliance Report, and DOT Order 1351.34, it seems clear to me that by now there should be many more data sets posted on Data.gov.

Open government is not always easy, and it takes perseverance, patience, and even courage to stick with it and get it right. And time and time again we have seen that it is worth the effort. If DOT is the best we've got, then all I can say is that we most certainly have a ways to go.

As part of the international Open Government Partnership, the US will soon release a National Action Plan about government transparency, intended to stretch "the country beyond current practice", "with the active engagement of citizens and civil society."

As the White House considers what to include in the plan, Sunlight is suggesting two specific actions for the plan.

1. The White House and the Office of Management and Budget should allow agencies to publish any open government recommendations, draft laws, and plans that they have, without fear of reprisal. Any gag orders or restrictions on sharing transparency proposals should be lifted, and agencies should be encouraged to share their draft ideas.

2. Every agency should publicly index and audit their regulatory data at least as well as the Department of Transportation has, and exceed the requirements of the Presidential Memo on Regulatory Compliance Data.

Open Sharing of Plans

The US National Action Plan should demonstrate what it means for a country to engage in good faith in the complex business of reforming transparency laws.  American transparency laws should be a source of pride, often setting informal international standards for disclosure.  They are also imperfect, and are the subject of intense scrutiny and analysis.

Some of the most informed expert views on any transparency law come from the government officials tasked with implementing those laws.  Unfortunately, those expert views are kept away from the public.  Many agencies that implement open government requirements have drafted legislative changes and guidelines for reform, but they are kept from sharing them publicly by OMB.

As part of the National Action Plan, the White House should demonstrate that transparency laws should be the domain of public discussion, not political control and review by central political staff. OMB should encourage the publication of draft transparency laws.

The complexity and importance of our transparency laws means that their expert review should not be subject to central control and gag orders.  Anyone responsible for implementing an open government law (in any country) should be permitted (and encouraged) to speak publicly about how those laws might be improved.

Indexes and Audits

Open Government Directives and proclamations can do an enormous amount of good. Ultimately, though, their main limitation comes from their vague language and the lack of enforcement mechanisms.  Aspirations can set a direction, but they're rarely enough to keep us on track.

In terms of new data and information, the Open Government Directive from OMB caused an initial burst of new information, but ultimately didn't fundamentally alter the way agencies choose what to release.  That's why we were so excited when President Obama followed up the OGD in January 2011 with a closely related Presidential Memo.  If the first Directive fell short because of a lack of specifics, the Presidential Memo showed that there are meaningful ways to direct agencies to create a public list of their data, and audit how well it's made available.  Specifically, the memo asked agencies to review the data that they collect from those entities they regulate, and make plans for how to improve it.

It's now most of a year later, and the most exciting result we've seen is the draft plan from the Department of Transportation.  It's worth a read, especially the appendices. It's a well designed guide to data collected by the different parts of the agency, organized by whether or not it's public, complete with citations of the laws or regulations that cover the information collection, and including ideas for how data publication could be improved.

While this may seem like basic stuff, it's actually been nearly impossible to get this sort of guide out of agencies, as similar policies have been pursuing for decades. Understanding what is knowable about an agency's work is one of the most powerful ways to approach oversight, and this plan gives any member of the public a great start.  The plan also demonstrates the work that CIOs ignore far too often (as they focus more on technology procurement): active stewardship over public information.

In the National Action Plan, the White House should recommit agencies exceeding the requirements of the Presidential Memo on Regulatory Compliance Data.  Every agency should create a public list of their data (and a plan to improve it) that is at least as comprehensive as what DOT prepared.  Every agency should be a responsible steward of its public information, and the first good faith step in taking that responsibility is to publicly define what datasets the agency is responsible for, and to publicly define the steps that should be taken to improve them.

Ultimately, this indexing and auditing requirement should extend to other fields of information in addition to regulatory compliance, but since some of our public protections have sometimes eroded as a result of atrophying, ineffective disclosure, it's a good, focused place to start.

By including these two changes together, the White House can demonstrate its commitment to openness, showcase the strong foundation for transparency that exists in American government, and also demonstrate that good-faith reform involves including the public closely in a complex, long term process.