Gov2.0 Presentation: An Open Government Scorecard

by

I was absolutely delighted to present at the Gov2.0 Summit here in Washington this morning.  Thanks to Tim OReilly and his whole crew for the invitation and all the hard work they put into the event.

I have to admit it was a real  challenge to present an “Open Government Scorecard” in the 10 minutes I had allotted to me but as I prepared my remarks I found that the time constraint really made me focus.  There were a number of disappointments that I have in the President’s record that I didn’t mention, such as his calling for  — but not acting upon —  a centralized database for earmarks, or his promises to push for lobbying reform that has never materialized — but I hit some of the major concerns that I and my colleagues at Sunlight have with the Open Government Directive, Recovery.gov, Data.gov, and USA Spending.gov. (Tomorrow I’ll talk more about our ongoing analysis of the data on that site.)

And rather that summarize my remarks, I’m presenting them in full below. Let me know what you think in the comment section below.

From the Executive Memorandum on transparency issued by President Obama on the first full day in office, to the full-blown Open Government Directive, to establishing new policies regulating lobbying of the Executive Branch, no organization has been more excited, enthusiastic or optimistic about the advent of Gov 2.0 for data transparency than Sunlight. In many respects, this Administration has gone further and faster toward creating a transparent government than any that’s preceded it.

But now, 20 months later, the drive for transparency appears stalled. * The Open Government Directive — the Administration’s 10-month-old manifesto on government transparency — can either become a dated relic destined for the ashbin of history, or a transformative commitment to a new era of openness. Right now it’s teetering someplace in between.

As you may recall, the central thrust of the directive is its insistence that all government departments create and implement their own open data plans, aimed at releasing “high-value” data to the public. The plans that resulted, however, were little more than aspirational. In the first of those plans, 12 out of 30 agencies didn’t identify any data for future publication and altogether only 75 new data sets were promised.  75 data sets? That was hugely disappointing.

Enforcement of these plans has always been ‘soft.’  And now even faces greater uncertainty with the departure of OMB Director Peter Orszag and WH Ethics Counsel Norm Eisen. The Directive is only as strong as its enforcement. * To be sure, there have been some meaningful first steps from agencies and the White House.  HHS and NASA both immediately spring to mind as aggressively pursuing new projects that are on the cutting edge. The White House itself has taken some meaningful steps in posting its staffers’ ethics filings online, requiring extensive stimulus lobbying disclosure, and posting the Visitor Logs online for the first time. But these aren’t well-established policies, and exemptions to publishing this data are unclear and unstated. All of these initiatives need a steady hand and a clear commitment to mature into permanent, reliable, effective policies.

And one central Obama campaign promise, to “create a centralized Internet database of lobbying reports, ethics records, and campaign finance filings” isn’t even on the drawing boards. * Data.gov started with enormous promise. A single catalog for all government data is very exciting in concept. As it has evolved, we have gotten a progressively better website: a more attractive design, and somewhat improved organization of the data sets. But it’s still a pretty mediocre data repository and the types of data available remains an enormous concern.

It turns out that the government has some interesting ideas about what counts as “high value” information.  The Department of the Interior seems to feel that population counts of wild horses and burros are “high value” but records of safety violations like the ones that seem to have led to the Upper Branch Mine disaster are not.

We want to see data that can be used to hold government – and the entities that report to it – accountable: records and data that would allow the public to assess the effectiveness and efficiency of federal programs, policies and initiatives; the competence and integrity of its employees and contractors; its management of public resources. * Recovery.gov, the government’s signature project to demonstrate transparency around economic stimulus spending is hard consider it more than a qualified success. And the idea of collecting job creation data at the most local level was overly optimistic from the start – and ended with Vice President Biden making excuses for the data’s poor quality on the Daily Show. * Now, let’s take a look at USASpending.gov, the project that then-Senator Obama and Senator Tom Coburn championed into law back in 2007.

USASpending was created to provide the public with information about how the federal government spends our tax dollars.  It was launched in late 2007, but it’s already gone through three redesigns, each one flashier than the next, with gradients and maps and now a sort of GapMinder-style visualization tool.  It’s pretty impressive…. looking. Unfortunately, its data is almost completely useless….

To demonstrate just how flawed, Sunlight is announcing today a new project today called Clear Spending.

This project – the result of the hard work of Sunlight’s Kaitlin Lee and Kevin Webb — is an ongoing analysis of the grants data in USASpending. We started with a methodology for evaluating data quality that has been used by the GAO.

It’s really pretty simple.

Federal spending gets reported in two places. Individual transactions are sent to USASpending.  There’s also a separate, somewhat redundant process where total spending for each program is reported to the Catalog of Federal Domestic Assistance, or CFDA.  It’s possible to tally up the USASpending records by program, and compare those totals to the ones in the CFDA.  When the two don’t match, we know there’s a problem.

That’s the heart of the method. We also looked at whether the reporting occurred on time, and whether all of the reports’ mandatory fields were filled out.  Then we automated those three tests, so that we could run them against all ten million rows of the USASpending database.

The results aren’t pretty. We found just under* 1.3 trillion dollars of broken reporting in 2009 alone. That’s fully half of the spending for that year. Some of the numbers are too big, some are too small, some are missing completely. Others don’t have the detail that’s required, or were reported months later than the law demands.

The data powering USASpending is broken. You can’t trust any aggregate numbers you get from the site — answers to questions about federal spending that rise above the micro level. When we say things just don’t add up, we mean it.

The government has known about this problem, and they say they’re working to fix it.  But what we’ve actually gotten is a series of redesigned websites, each one with data just as unreliable as the one before it.

We are beginning to worry that the Administration is more interested in style than substance. * If we settle for a superficial kind of approach, Gov 2.0 will be remembered as a failure. Government has learned to say the right things — now we need government to actually get serious about technology and transparency. *

There’s a lot of work left to do. These are not easy tasks, or certainly not glamorous ones. But these are the types of challenges that we must undertake if the promise of Gov 2.0 is going to be realized.

For starters, we have to take on some of the responsibility for making this happen ourselves – I mean ‘us’ as in the community of Americans who are concerned about accountability. And that’s a lot of us. And we wear all kinds of political labels from conservative to liberal, to libertarian to the labels of Tea and Coffee Party activists.

Our job is to hold the Administration’s feet to the fire – bureaucrats aren’t going to act just because someone asks nicely.  Government isn’t going to change how and when it makes data available – even when a few good people on the inside want it to – because of a directive.

It’s not going to happen until laws are changed, or Executive Orders are issued, or until enforcers are given real power and the President himself makes it a priority.

And finally, we need to admit that Gov2.0 isn’t happening until citizens are truly actively engaged in helping to demand and co-create it.

We can still have a government that is truly open, participatory and collaborative. But it won’t happen until we push for it.

Thank you.

*We updated this number – at launch it was just north of $1.3 trillion. You can see why here.

Categorized in:
Share This:
  • janice parsons

    Andrea,
    You sound like you have years of experience with entrenched governmental behaviors but you haven’t lost your optimism or passion. Your viewpoint was interesting to read. Funny, but the entire country is crazy with frustration over the fact the left hand doesn’t know what the right hand is doing….not to even mention the prohibitive costs of all this!
    I’d love the government to Race to the Top. It’s about bloody time!
    thanks

  • Andrea Schneider

    TLK,

    Thanks for the nod. I know there are excellent people within the different agencies with great ideas and talent. We also have many which will resist change, not admit they “don’t know how”, have deep grooves in the earth against innovation or anything which could upset their current sense of “power and control”.

    Inevitably, new directors are political appointments, sometimes good, sometimes not so good, sometimes terrible. Unless they are really strong and know what they are doing, they can be manipulated by a senior staff inherited from the prior administration.

    The incentive to change, innovate, raise standards of practice and seriously address new initiatives or directives can be minimal. It takes confident leadership to open up the system, make it safe and make it happen.

    The Open Government Directive needs us to dig in together and establish clear strategic, operational and tactical goals against which results will be measured. Parallel to this we need the Executive Branch to conduct transparent organizational assessments, perhaps merging the best practices in business and government, to develop and standardize the tools we use. We do not lack the resources. There is no excuse.

    Once agency assessments are conducted, it will be apparent where the agency needs help. We can structure inclusive capacity building initiatives, and
    I don’t mean one-day workshops, which will institutionalize the behavior and standards we want to establish. Not so different than what is happening with the Education Initiatives of the Administration right now.

    Sustainable organizational change takes time, this will not happen overnight. If we truly want data that counts, and we want it over time, we need to be very serious about how we approach getting it in a sustainable way. Our methodology has to match up.

    The agencies have to integrate these standards for data collection, and the other equally important parts of the Open Government Directive, into their grant announcements and expectations. We don’t have to watch and wait for them to fail.

    We have the information and know-how to translate all of this into action in a clear and sustainable way. We have the talent in this country, both public and private, to make this happen.

    A question: If we boarded an airplane and the pilot got on the intercom and said, “I will try and fly this plane”, would you stay on? No. This is no longer about trying, this is doing something hard with confidence, collaboratively and support.

    Congress also has to understand the benefit to themselves and to their constituents. Has the case been made in such a compelling way, past rhetoric, that it’s clearly a mandate?

    Sunlight is right, we need public participation and they are working on it with Public=Online. Again, we can easily make a case that each aspect of the Open Government Directive will save a lot of money, reduce redundancy and the duplication of spending, over and over again. Who doesn’t want to save taxpayer money?

    I could tear my hair out over the grants I’ve seen, targeting similar issues, but funded through different agencies. Can you imagine what this mess can do to transparency and clean data collection? While I’m getting bald, how about all the work sitting on shelves somewhere, tucked away, out of site out of mind? Trust me, there is rich data and very successful programs, just put aside, once the funding cycle is over. Back to the drawing board. New cohort, new funding, sometimes the same stakeholders.

    Why can’t we create a “Race to the Top” for our own government? Why can’t we bring the best and the brightest together to create action plans which work? Why can’t we tap the creativity inside our federal institutions, regardless of title?

    Americans are creative, smart and anxious to put our best feet forward. It’s embarrassing to see our Congress act like they do or our Executive Branch not put teeth into their own great ideas.

    If we can be brave enough to ask the harder questions and challenge, as Ellen and Sunlight has done, we will get somewhere. There are a lot of groups working on parts of the whole, from different points of view and different advocacy positions. The great thing about the Open Government Directive, in full, is it can be hard to argue with, on a partisan basis, if done right, explained well with demonstrated results. Given the inpatient mood of the country right now, we can create methods to demonstrate outcomes on a comprehensive scale. We all know real change takes time and many, many wins over time to be sustainable and normative.

    I do think it’s problematic, even if we have the very best data, to move it into action. It’s a current problem many are working on. What we do about this is critical or else we just have a lot of good information with nowhere to go.

    Lastly, I think the confusion between Gov2.0 and the Open Government Directive, which is the umbrella concept, is not helping us keep things clear and simple. If we aren’t clear, it’s very hard to ask or expect others to be clear. We have our own hard work to do. Luckily, we have many more ways to work together because of technology.

    Thanks Sunlight for creating a forum to express ideas. It’s truly refreshing and even possible we will be heard and make a difference!

  • John Keck

    I think you are underestimating the amount of effort necessary to actually pull this off technically. Under the best of circumstances a project this big is extremely difficult, time intensive and error prone. The data was not sitting around already organized and formatted and now it is just a matter of will. In many cases a whole new work flow and process re-engineering is required.

    Also, the government doesn’t have the A-team working for them. These projects are not being executed by Google or Amazon. There are some talented folks but what’s being build is primarily done by 2nd tier contractors.

    It’s going to take years to get this right.

  • TLK

    Andrea adds excellent insight to what Ellen articulated in her speech. Like all hard problems, it has multiple causes. I’m working on getting my agency’s data released in a way that is useful. The question is, what do we do about it? Where can we continue this conversation??

    Some additional things to consider:

    * Underlying data quality is such a big problem. Not only is there in several cases not enough resources to put towards GOOD data, but in some cases the people doing the data aggregation and management don’t know how to do it properly. Yes, we can hire contractors to get these skills, but these contractors are useless without a competent person giving them real technical direction, navigating them through the vast government bureaucracy and giving useful feedback on their work. This isn’t happening. Lots of people in my agency don’t want to see themselves as bureaucrats, so they shrug off these types of responsibilities, produce bad products, and then blame the system for it.

    * In some cases, there are great datasets, the problem is the people with these datasets don’t understand how to make it accessible. they think “accessible” = “dumbing data down” so they resist it. Or they truly, honestly believe that this: http://www.epa.gov/waters/data/index.html or this: http://www.data.gov/geodata/g602222/ is accessible.

    I cannot tell you how much I appreciate Sunlight for turning up the heat on this issue. I just hope that the govt will listen. Like Ellen said, “Government isn’t going to change how and when it makes data available – even when a few good people on the inside want it to – because of a directive. It’s not going to happen until laws are changed, or Executive Orders are issued, or until enforcers are given real power…” As one of those good people on the inside, I want change just as badly as you do. Keep up the good fight!

  • Andrea Schneider

    This is a very well stated presentation. Much of this has needed to be said for quite awhile.

    There are a few things I’d like to mention. One, I think transparency and Gov 2.0 is one aspect of Open Government. Important, but so is collaboration in it’s many forms, accountability, beyond qualitative data, and participation, aside from websites alone.

    I sincerely doubt, when the Open Government Directive was announced, that anyone in the Executive Branch ever did an analysis of organizational capacity to implement the directive.

    It’s one thing to want something to happen, it’s another to know how to do it and do it well. I am a firm advocate of the Open Government Directive, yet I fail to see where there has been any investment in “teaching” the agencies what each of the parts of the whole even means or establish any standards of practice.

    Grant management and outcomes is very disappointing, for so many internal and external reasons. Nothing has changed in a really long time, neither in the process nor expectations of the granting agency or the grantee.

    Serious grant data is often an afterthought in reporting. Then what happens to it after the reporting. A lot of grantees or anyone else ever hears a word.

    There is rarely consistency between each agency, agency requirements, qualitative and quantitative evaluation or expected results.

    Often the direction comes from the grantee, again ignoring their need for capacity building as a core building block to success. Having evaluated millions of dollars of federal training and technical assistance programs, it is stunning how much doesn’t happen, stick or is sustainable.

    Reporting can be superficial because data collection is expensive, as is meaningful evaluation.

    While we are collecting data, I want to know what we are collecting on collaboration between federal agencies, how we are reducing redundancy, increasing effectiveness, and lowering costs?

    How many agencies look at what has been funded in the past? Have they used that data and those reports as a foundation for future initiatives?

    How many agencies have gone through all the thousands of reports previously submitted, where millions of dollars have been spent, to mine for best practices and discoveries? How much money could we save by taking the time to look?

    How many times has Congress funded the same thing over and over again, in multiple federal agencies?

    Ellen, as always, I really appreciate your opening up this topic so honestly. It’s a great jumping off point…