The White House blog recently wrote about Obama's trip to India and mentioned that US-based organizations could learn from Indian organizations using technology to improve accountability and transparency. I agreed. Now is a great time for the US government to recognize that there are  transparency policies all over the world that we Americans could implement or, at a bare minimum, learn from. Here are just a few foreign governments that have policies we wish would improve what we have state-side:

You get a dataset! And you get a dataset! Everyone gets a dataset!

There is always progress to be made and the presumption to make data public and online (with teeth!) is an important cultural shift we hope to see soon. Just last week the United Kingdom took an unprecedented step to publicize all government spending over 25,0000 pounds. As governments around the world tighten their belts we think making the books fully transparent will allow citizens to be better informed about where their tax dollars go and how to move forward. Here in the US there is the Data.gov site (which could be greatly improved) and we are encouraged that the culture is shifting as we see folks like the United Nations, the World Bank, Russia, Spain, Finland, Australia and many others hopping on board.

Publicly Funded Research Papers Available to the Public

The Congressional Research Service, often referred to as 'Congress' think tank', is a well-respected non-partisan branch of the Library of Congress that regularly publishes reports exclusively for members of Congress and their staff at a budget over $100 million. The Sunlight Foundation and others have long advocated for these reports to be public (meaning online), but they remain inaccessible to the general public.

Many foreign governments have publicly-funded think tanks similar to CRS, but they make the reports free to the public and accessible online. The United Kingdom has the House of Commons Library Research Papers, Canada has a nice list with categories on the site of their Library of Parliament Research Publications, and Australia publishes their reports (going back to 1993!) on the Parliament of Australia's Parliamentary Library website. Australia even has official research reports published on the state level by the websites of Victoria and New South Wales.

Imagine that.

Creating Better Disclosure Surrounding Resource Management

The US could learn a thing or two from other resource-rich countries about disclosing online searchable production, leases, costs, audits, and safety reports. This important non-proprietary information keeps the public informed about the safety and financial status of our natural resources. We hope the Bureau of Ocean Energy Management Regulation and Enforcement (BOEMRE) that replaced the Minerals Management Service (MMS) will take the necessary steps that many countries have already taken to improve online reporting in this sector.

The Revenue Watch Institute and Transparency International recently rated the top 41 oil, gas and mineral producing countries countries in terms of their government disclosure record [pdf link]. The United States came in at 11th place, behind Russia, Mexico, Kazakhstan, Azerbaijan, Ecuador and others. This ranking assessed revenue transparency more than safety records, but it is an important metric to recognize how much the US government could continue to learn. Let's see less of this and more online disclosure like Angola.

Expanding and Enriching Visitor Logs

The Prime Minister of the United Kingdom has a portion of the official website dedicated to transparency initiatives including some substantive items that we would love to see in the US. We appreciate what the White House has done with releasing visitor logs, but a glance across the pond shows that Number 10 is posting details of meetings, hospitality, gifts and overseas travel across all departments and high level staff. Impressive stuff when you compare it to the White House offerings.

Online Disclosure Forms

The Australian equivalent of the Federal Election Commission, known down under as the Australian Electoral Commission (AEC), has a great online system to research financial documents relating to the elections.  It includes a nice financial disclosure and donor search function that is quite similar to the FEC version (both obviously don't hold a candle to Sunlight's illuminative version), but after some more research I discovered that they allow those who have to file* to do so through disclosure forms online!  We didn't venture beyond the sleek registration page, but it gave us goosebumps to see other countries approaching our vision of real-time online disclosure. We would like to see this type of online filing possible for lobbying, elections or even meetings - it would certainly ease the eyes of our reporters who often have to dig through .pdf image files.

• In Australia the political system requires candidates and Senate groups, registered political parties and their associated entities, and donors and third parties to lodge disclosure returns. Swoon!

It would be impossible to ignore that each country listed in the items above has a unique political system, but these examples serve as great starting points for policies that could work here, now. The Sunlight Foundation will continue to encourage dialogue on these important issues and hope that the US government learns from non-profits and governments all over the world.

Just as the Administration needs to incorporate transparency into their response to the failures of the Minerals Management Service, Congress does as well.

Since even before the oil spill began, Congress has been working on bills that would change the laws that govern offshore drilling. While there may be others, I’m particularly interested in the CLEAR Act [PDF], HR 3534, since it would amend the law I’m most familiar with, the Outer Continental Shelf Lands Act, which could redefine how the government (and the public) relate to offshore energy information. You can find the version the House posted here. There is a similar Senate bill, S. 3516, which is far shorter.

...transparency requirements often get overlooked entirely in the context of finance or energy, since they’re so specialized and complex. This just isn’t acceptable.

There are a number of transparency requirements that should be included in these bills. I’ve reviewed the entire House CLEAR Act, and suggested a number of amendments to improve transparency below.

Important reports, energy statistics, and oversight information needs to be online for disclosure to be as effective as possible. Public access requirements shouldn’t be an afterthought, especially when transparency is part of the foundation on which our entirely regulatory system is built.

As we all watch what happens when that system fails, we should turn our interest to what we can control, and one part of that is to help Congress define what we can know about resource extraction.

I should note that Sunlight isn’t endorsing any particular approach to regulating oil drilling, but if Congress acts on these bills, they should better address the transparency requirements associated with them.

While we’re certainly not experts in the energy industry, transparency requirements often get overlooked entirely in the context of finance or energy, since they’re so specialized and complex. This just isn’t acceptable. Also, if I’ve missed something, please let me know in the comments.

To see my annotations inline with the bill click the image below. (The same comments are in the extended entry below, as well.)

I've also included the text of my commentary separately below. All page numbers refer to the pagination on the document, rather than the file. Page 31, line 14: If the Secretary needs special authority to hire officials outside the regular process, and if that presents enough of a risk that all of those hires need to be reported to Congress, then shouldn’t those officials in critical positions also be reported to the public? Reporting to Congress isn’t always the same as reporting to the public, and it seems likely that the need for public oversight here outweighs the small privacy interests of the officials so appointed.

Page 45, line 22: This is a spectacular provision. Proprietary information (like building codes) are often incorporated by reference into laws, and if those laws are to be followed by everyone, then they should be accessible by everyone. See Carl Malamud’s vast work on the topic.

Page 50, line 1: If the Secretary is preparing a comprehensive report on the “Federal offshore oila and gas fiscal system” and submitting it to multiple committees, it should be made public as well, with any appropriate redactions if they are necessary.

Page 55, line 18: Why aren’t the Exploration Plans required to be public and freely available online? It appears that they are posted now. They do get bonus points for the bulk download option. Why not write this into the statute?

Page 59, line 6: The drilling permits aren’t required by statute to be made available. Despite widespread reporting about their issuance, I haven’t found them online. If they contain sensitive information, redacted versions could be published, which MMS did for many other documents. This could be required in the statute.

Page 60, line 10: The exploration permits should be required to be public. They (like many of these documents) are full of the kind of planning and assurances that were apparently flaunted at the old MMS.

Page 67, line 4: Why aren’t the Environmental Studies required to be public?

Page 67, line 14: It seems clear to me that research on data gaps regarding the use of dispersants should be required to be made public.

Page 75, line 24: The bill doesn’t require the Development and Production Plans to be made public online, even though they’re intended to be a public document, and are already required to be available at the regional offices by MMS (now BOEMRE) regulations. They should be required to be posted online. For additional research on the plans, see this post.

Page 79, line 10: The Oil and Gas Information Program doesn’t seem to incorporate public scrutiny at all. Perhaps it could strengthen its work by incorporating online transparency.

Page 89, line 16: This requirement looks good, but could be strengthened by requiring bulk access, and requiring that the datasets be registered on Data.gov.

Page 97, line 8: It seems to me that the Interim and Final Reclamation Plans should be required to be made available to the public.

Page 109, line 13: This data should be registered on Data.gov, and made available in bulk.

Page 120, line 16: If natural gas reporting needs to include collection of BTU values, perhaps that information should be available to the public as well as to the Secretary. The same goes for the “MMS-2014” form altogether, although I would expect redactions to be necessary there.

Page 145, line 1: This is good, but could also include bulk access and registration on Data.gov.

Page 146, line 21: This seems to be a drafting error -- redundant phrase (compare (G) and (H)).

Page 160, line 8: There is no good reason for the committees to be exempted altogether from the Federal Advisory Committee Act, especially given MMS’s history. See this post to see how we successfully pushed for this exemption to be narrowed in the financial reform bill.

Page 184, line 17: This is good, again, could have bulk acess and registration on Data.gov required.

Page 185, line 3: Shouldn’t this report be required to be made public as well as reported to Congress?

Just as there is now a new cap being fitted over the leaking oil well is patching a mechanical failure, we are left wondering whether the now-defunct Minerals Management Service (MMS) will be fixed as well.

Online transparency needs to be a part of the administration’s response to their own failure, and so far, it isn’t.

The American public has been engrossed by the physical effort to stop the leaking well -- a public display of underwater engineering with divers, oiled sea life, and animations of complex drilling machinery.

There will also be a response to the regulatory failure, but the governmental response isn’t likely to have a live camera on the lobbyists, ethics rules, or complex regulations that will result from the disaster.

We may need it, though.

Just as public attention on the images of spilling oil have driven the policy response to this disaster, only real public scrutiny will guarantee effective government oversight. Online transparency needs to be a part of the administration’s response to their own failure, and so far, it isn’t.

In President Obama’s prime-time Oval Office address, he described this regulatory failure:

One place we’ve already begun to take action is at the agency in charge of regulating drilling and issuing permits, known as the Minerals Management Service. Over the last decade, this agency has become emblematic of a failed philosophy that views all regulation with hostility -- a philosophy that says corporations should be allowed to play by their own rules and police themselves. At this agency, industry insiders were put in charge of industry oversight. Oil companies showered regulators with gifts and favors, and were essentially allowed to conduct their own safety inspections and write their own regulations.

While Obama does acknowledge the regulatory failure, he leaves out the role the media has played in illuminating these failures, and, perhaps as a consequence, omits public oversight from his plans for reforming the MMS.

The stated plan for MMS reform is to hire a new director:

When Ken Salazar became my Secretary of the Interior, one of his very first acts was to clean up the worst of the corruption at this agency. But it’s now clear that the problem there ran much deeper, and the pace of reform was just too slow. And so Secretary Salazar and I are bringing in new leadership at the agency -- Michael Bromwich, who was a tough federal prosecutor and Inspector General. And his charge over the next few months is to build an organization that acts as the oil industry’s watchdog -- not its partner. So one of the lessons we’ve learned from this spill is that we need better regulations, better safety standards, and better enforcement when it comes to offshore drilling.

Michael Bromwich and his new role are presented as a suitable proxy for the public interest. If Obama wants to distinguish the administration from the industry response to this kind of failure, however, renaming the MMS and shuffling in new leadership aren’t going to do the job.

That’s what we’ve gotten so far, though -- MMS will now be the Bureau of Ocean Energy Management Regulation and Enforcement (BOEMRE), and Elizabeth Birnbaum has now been replaced by Michael Bromwich.

If this experience has taught us anything, though, it’s that we shouldn’t accept public officials as a replacement for public scrutiny. If MMS under Salazar saw problems that “ran much deeper,” and “the pace of reform was just too slow,” as Obama noted, then why should BOEMRE operate any differently? Should we just trust Bromwich’s reputation as a change agent?

In responding to such a disaster, the administration can either be associated with the mistakes or their solutions.

The MMS website, with its message of a transition, can either represent a band-aid on a hemmoraging oversight agency, or it can be the beginning of our collective ability to hold industry and government accountable.

To start with, new leadership should post the documents and data online from the old MMS. Contingency plans, leases, Development and Production Plans, inspection reports, ethics waivers, and other essential information should be affirmatively posted online for everyone to see. MMS’s failures should be examined, scrutinized, and used to propel real reform within the new agency.

Any of these reports and data that are publicly available, but not online, should be freely posted on the new site, as soon as possible.

The administration should consider whether the strict restrictions on stimulus lobbying may be appropriate for a bruised and ineffective regulator like BOEMRE. If merit-based decision making needed special protection for the stimulus spending, it certainly needs it here.

If online transparency is to be a tool for strengthening trust in government and dispelling corruption, then it should be a tool for fixing our oversight of offshore drilling. The Obama administration can start by affirmatively posting the accountability data they have relating to the old MMS, and by guaranteeing that the MMS transition isn’t just another failed blowout preventer.

Powerlessness in the face of disaster is dispiriting. Powerlessness in the face of regulatory failure is fixable.

Despite our widespread ability to communicate online; to see, as a society, to the murky bottom of the Gulf in real time, we're still in a suspended state of irrelevance to this slow motion disaster -- transfixed and dazed, as Micah Sifry points out.

Unfortunately, the people formerly known as the audience are fundamentally still functioning as the audience.

This is true, in part, because the current situation is a failure of complex machinery, which is difficult for most of us to constructively relate to. We try, suggesting enormous rubber shower curtains, or pointing out the absorbency of hay. And perhaps the international community of oil rig engineers is collaborating well now -- if so, I hope that story gets told.

Mechanical failure is palatable; regulatory failure is inexcusable.

There is, however, another failure here too, that takes less specialized skill to relate to. That's the regulatory failure that has led to current situation. We have a complex set of regulatory mechanisms set up to keep this from happening, and they have failed, and miserably so. We're only really relating to that regulatory failure only through traditional investigative journalism -- to its credit, but also at all of our peril.

Every day brings a new kick in the stomach, as the New York Times, McClatchy, and many many others illuminate new parts of this failure.

And each time I read one of those stories, I feel the same way -- amazed that it happened, and also amazed that we're only finding out about it now.

For all of the national discussion about offshore drilling, how has no one reviewed the required plans before now to realize their apparent fakeness? How were federal regulators able to let the industry fill out their own inspection forms, to be later traced in pen?

How often are we letting them languish, unread, unexamined, and unchallenged, in the “regional field offices” of our public neglect?

Many of these "public" inspections, in turns out, were only public in a very limited sense, opening the door to neglect and abuse. The same holds for the plans companies need to submit before they drill -- apparently public, but effectively out of reach, and, consequently, filled out thoughtlessly, failing to create accountability. (My initial research into those reports is attached at the the end of this post, and inspired me to write this.)

Many of these public accountability mechanisms rely on outdated techniques despite their central role in our regulatory system. Putting an important report in a regional field office doesn't make it effectively available for public inspection. As we've now seen, it makes them effectively hidden -- waiting in obscurity for weeks for real review, even in the face of the country's largest environmental disaster.

That's the spirit in which we've helped prepare the Public Online Information Act, or POIA, introduced by Rep. Steve Israel in the House -- to require public reporting to be truly public, by forcing their publication online.

While the live view of the gushing oil is valuable, creating some accountability, and having what is probably an important psychological impact, we need a live view of the other mechanisms that failed us here too -- our public protections in the form of regulations. I'm not talking about cameras on the helmets of oil rig inspectors (although the image is appealing at the moment), but about key public reports going online so that they serve their intended purpose.

These reports aren't obscure, or pointless either. They're absolutely central to the way we regulate industry activity, and that's true across regulatory contexts. The SEC, FCC, FEC (etc) regulatory agencies each collect public information, not as a byproduct of their work, but as a central approach to doing their jobs. Read the Federal Election Campaign Act. Or the Outer Continental Shelf Lands Act (as I did below) -- they're largely accessible even if you don't have a law degree -- and you can see that these public reports were created as the pillars of public regulation. Public reporting empowers public regulation.

How often are we letting them languish, unread, unexamined, and unchallenged, in the "regional field offices" of our public neglect?

The administration, as they piece together the Interior Department and the Minerals Management Service, should use public scrutiny to everyone's advantage, and start posting MMS information online -- if it's public, post it. There is probably much, much more to be examined than the recent news stories indicate.

As they retroactively illuminate our regulatory failure, the daily gut check from our newspapers includes phrases like "according to our review of certain MMS documents." We should, instead, be saying "according to documents submitted yesterday by inspectors," or, "according to a permit to drill submitted earlier this week by , and spotted by , a reporter for the Daily ___".

Mechanical failure is palatable; regulatory failure is inexcusable.

That it takes a national disaster to spur us into effective oversight means we've got a long way to go before the public can effectively hold the government, and by extension, regulated industry, to account.

What follows are my notes as I researched the OCS Lands Act, and the reports it creates ineffective public access to:

Here's a summary of what I found regarding the MMS reports I haven't been able to find online anywhere yet. 1. The Outer Continental Shelf Lands Act (OCS Lands Act) sets policy regarding leasing and drilling. pdf: http://bit.ly/d1T83m US Code (43 USC 1331 - 1356a) -- http://www.law.cornell.edu/uscode/html/uscode43/usc_sup_01_43_10_29_20_III.html 2. 43 USC 1351 mandates that detailed "Development and Production Plans" (DPPs) be submitted to the Secretary (almost def. the Interior Secretary). I suspect that this detailed report would be immensely valuable to read, especially if one were done of the Deepwater Horizon well. I don't know whether that well would be covered by the OCS Lands Act, or whether it's different, although I suspect it'd be covered. Even if the current spill isn't covered by this requirement, it's likely that it'd be covered by other similar requirements. 3. The same law mandates that these reports are available to the public, after appropriate redaction, and within 10 days of receipt. ("online" isn't mentioned.) Googling for them gets me nowhere. 4. The Regulations promulgated under the OCS Lands Act require that these DPPs be made available to the public at the "MMS Regional Public Information Office", by the Regional Supervisor. 30 CFR 250.204 (PDF: edocket.access.gpo.gov/cfr_2001/julqtr/pdf/30cfr250.204.pdf ) 5. The MMS site says that these required reports will also be called a Development Operations Coordination Document, or DOCD. http://www.gomr.mms.gov/homepg/regulate/regs/laws/postsale.html#d+p 6. Googling for DOCDs and DPPs (the two names for the same required report) hasn't been productive. 7. There are numerous other reports on the MMS site that could conceivably be some sort of excerpts of the DOCDs or DPPs, but they appear to be something else. 8. Guidance from MMS on how to prepare and submit those documents: http://www.gomr.mms.gov/homepg/regulate/regs/ntls/ntl03-g17.html 9. MMS's information page: http://www.gomr.mms.gov/homepg/pubinfo/piindex.html (The DPPs don't appear to be here either.)

Yesterday, the Inspector General of the Department of the Interior released multiple reports revealing widespread corruption in the Mineral Management Services agency, which handles mineral extraction, leases, and royalties for the Department of the Interior. The allegations show employees receiving illegal gifts, graft, filing false statements on ethics forms, using illegal drugs, and having sex with both subordinates in the agency and with agents of oil and gas companies with business before the agency.

Here are some of the allegations:

Lucy Denett, former associate director of minerals revenue management: accused of steering a contract to one of her aides after he retired.

Gregory Smith, former director of the royalty-in-kind program: accused of doing outside consulting work that included using his position to help the company paying him gain access to clients doing work with the royalty-in-kind program; billing Mineral Management Services for trips made in conjunction with his outside consulting work; accepting over $1,000 in gifts from oil and gas companies; using cocaine with a subordinate; having sex with two subordinates, where one episode is clearly a sexual assault.

Eight other employees: Socialized with and received gifts from companies with business before the royalty-in-kind program. Two of these employees are also alleged to have used drugs and had sexual relations with various agents of oil and gas companies with business before the program.

Here's CNN reporting on the report:

The IG reports are available at ProPublica where Paul Kiel is providing running coverage.

Since 2001, when President Bush took office, the Department of the Interior was beset by problems arising from the appointment of officials who previously worked in or with the industries that the Department is intended to oversee.

Both the Secretary of the Interior, Gale Norton, and the Deputy Secretary of the Interior, Steven J. Griles, came from the extraction industries. Norton worked for a law firm that lobbied for a variety of companies, including oil, gas, and metal companies. Griles previously worked for a natural resources company and later provided public relations advice to a variety of extraction companies doing business with the government. Both Norton and Griles wound up caught in the Jack Abramoff lobbying scandal. Norton resigned her post as the scandal encroached into the Department of the Interior, while Griles wound up pleading guilty.

Ethical standards trickle from the top on down. Some of the officials involved in this current scandal expressed the opinion that they "didn't think ethics rules applied to them because of their 'unique' role in the agency and that they needed to socialize with industry representatives for 'market intelligence.'" The Mineral Management Services scandal has been brewing for a long time and highlights a lack of oversight that occurs when a Department is staffed with individuals who are used to making money from the business they are charged with regulating.