On Monday the House of Representatives delivered, as promised, an electronic dump of House Expense Reports. We, at Sunlight Labs had a plan. We knew it was going to be a huge PDF, but we have all the infrastructure in place. We had plenty of bandwidth, knew when the data was coming out, roughly how it was going to look, and that it was likely we wouldn't be able to parse it all with computers. "We'll use TransparencyCorps," we thought, to get that last mile out of the data, so that eventually we'll end up with a parseable database.
Continue readingIs There a Local Hackathon in Your Area?
Just two weeks after announcing the Great American Hackathon, there are a dozen events planned across the country. The great govtrack.us creator Josh Tauberer is organizing one in Philly. The big-data folks at Cloudera are hosting an event. ThoughtBot is holding one in Boston, as is Steven Clift in Minnesota. And there's a handful across the country too-- in towns like DC, Atlanta, and New York. Today, Mozilla announced that they were getting in on the fun.
This is a great opportunity not just to make a difference but to meet new friends. So if you're near one-- create an account and RSVP to an event. Plan on making a difference with your skills that weekend.
And if you're not-- now's your chance too! Start organizing and creating community in your neighborhood. Get developers together. Find them in your area. Reach out and start building a community to open up our government. Check out our organizing guide and start your own.
See you in person or virtually on December 12-13th!
Continue readingOpen Data We’re Thankful For
While this is a little late-- late's better than never for giving thanks. And this year, we've got a lot to be thankful for. Open Data in Open Government is making leaps and strides. The Vice President is talking data quality in government on the Daily Show. ABC News along with Recovery.gov's controversy have brought government data into prime time. It's been a long time since transparency like this has seen this kind of attention.
At this time of Thanksgiving here in the United States I wanted to give thanks for the new and changing government datasets that we have now. Some are truly amazing.
Continue readingMighty Tiny Thomas
This post is from Sunlight's Policy Counsel, Daniel Schuman who normally blogs on the Sunlight Foundation Blog.
A few months ago, I posted a request on Sunlight Lab's wiki for someone to build a web tool that would make the links on Congress's legislative website, THOMAS, permanent. Although it seems odd, when users look up legislative information on THOMAS, such as bills and committee reports, they usually cannot bookmark or share the links because the URL goes dead after a couple of hours.
I had little faith that someone would answer my request to build permalinks, but Asa Hopkins has done so with a great new program called tinyThom.as.
Continue readingRecovery.gov’s Success
We spend a lot of time talking about how Government does a lot wrong with data. And we harass them and complain a lot to the extent that even I get on my own nerves. But the fact is, the people and programmers working on these projects on the inside are neither malicious nor incompetent. The problem isn't people, but a weird system of priorities and incentives that often leaves the citizen short-handed. After all, transparency isn't even an inkling the constitution (yet!) and I'm fairly certain that the framers of our constitution weren't really considering data portability when they drafted the Bill of Rights.
Continue readingFederalReporting.gov: Recovery.gov’s Dirty Little Secret
The press would have you believe that Recovery.gov is an $18MM website that collects loan, contract, and grant data from recipients and shows it to end users. But that's only half true in a lot of ways. That price isn't exactly correct, and that's not all Recovery.gov's function.
Read more to find out what we mean.
Continue readingGet your act together, Data.gov
On May 21st, we launched Apps for America 2: the Data.gov Challenge-- the very same day that Federal CIO Vivek Kundra & Company launched data.gov. On May 26th, Kundra announced that there were hundreds of thousands of data sources just around the corner.
It is now November 13th, 2009. Right now the Raw Data Catalog in data.gov stands at an even 600 feeds. What's worse, the data is chunked up into small little bits, making 600 not a particularly exciting number. For instance, nearly half the datasets (293/600) in the raw data catalog are toxics release inventory datasets, broken up into individual states and outlying territories further broken up into individual years, from 2005 through 2008. This isn't living up to expectations, or even keeping in line with public statements.
Continue readingWhy you shouldn’t use Dropbox to host images on your blog
Recovery.gov’s Systemic Failure
The new Recovery.gov-- which we've written about and even nearly bid on-- has certainly taken the government huge steps forward in terms of disclosing information, but it is not without controversy. The press is questioning the program, pointing to wasteful spending or bad data. The White House fired back with a "reality check"(their words) saying that few of the reports have gone through the "extensive three-week review" and that the data might be particularly misleading at this point.
Chat with Knight about the News Challenge on Monday
59063
