A couple of weeks ago I had the chance to go to Georgia and hang out with the folks at the Tbilisi chapter of Transparency International. It was a great opportunity to learn about a part of the world that I was completely unfamiliar with, to share some technical knowledge, and -- somewhat unexpectedly -- to gain some perspective on the work we do at Sunlight.
Quantifying Data Quality
You've already heard me complain about data quality -- how it's a bigger problem than most people realize, and a harder problem than many people hope. But let's not leave it there! Perfect datasets mostly exist in textbooks and computer simulations. We need to figure out what we can do with what we have. In this and other posts, I hope to give the developers in our community some idea of how they can deal with less-than-perfect data.
The first step is to figure out how bad things actually are. To do that, we'll use some simple statistics -- those of you with a strong stat background can skip to the next entry in your RSS reader (or better yet, correct my mistakes in comments).
More About the Door
But the electronics are just one part of the system. As I mentioned at the end of that last post, my colleagues did an impressive job of springing into action and building out the systems necessary to turn an SSH-accessible script into a useful interface. Here's how they did it. Continue reading
Our Door Opener (A Science Project)
Life in the labs has been pretty good since we moved into our current offices. Before, we were spread out over two floors: my team was upstairs in a stuffy law office sublet, and the rest of our colleagues were stuck in a homey but increasingly cramped and run-down space four floors below. Since moving everyone to the third floor we've found ourselves with plenty of room, lots more light and a nicer kitchen. It's just a more pleasant working environment in general.
But there's always room for improvement. For one thing, the new space came with new locks -- ones with really expensive keys. Issuing keys to the entire staff wasn't practical, and coordinating door-opening responsibilities in a way that accommodated team members' occasionally odd schedules was inconvenient. Fortunately, the space also came with the button you see to the right.
Located near the reception desk, this button opens an electronic latch on the front door. Pulling the assembly out of the wall revealed the system to be about as simple as possible: the button simply connects two wires. Bridging them with a screwdriver fired the latch (from their small gauge and uninsulated connections, it was obvious we weren't dealing with dangerous voltages, but please don't start pulling cable from your walls unless you know what you're doing).
Connecting two low-voltage wires electronically isn't a particularly hard trick, so I decided it'd be fun to spend some evenings building a system to expose that switch to our network.
Data Quality Deserves to be Tackled on Its Own
Last week Clay wrote about how we'll be evaluating /open pages released under the OGD. The post ended with a series of considerations that we think are important: completeness, primacy, timeliness, accessibility, machine readability, availability without registration, being non-proprietary, freedom from licensing restrictions, permanence and obtainability.
One thing is conspicuously missing from the list, though: quality.
Continue readingGrading the New Recovery.gov
Recovery.gov relaunched yesterday, and we've spent some time playing around with the site since then. The verdict? Well, it's hard to say — the site's a bit broken. There are 404s all over the place, most gallingly on the data download page. Parts of the site seem like they work, but don't: the select boxes on the front page that provide filters for the map don't actually affect its behavior in any way. It's hard to see these glaring bugs alongside the totally-unnecessary link to Facebook and not groan (am I supposed to play Scrabble with Chairman Devaney?).
Continue readingTaking a Look at @2gov
Last Friday I had a chance to meet with Dave Binetti, the man behind @2gov, a new service that aims to make it easier for Twitter-users to route messages to their elected representatives. The idea is pretty simple: register with the site, then include "@2gov" in your tweets. It'll grab your tweet, look up your previously-recorded location, then run your message through a classification engine that determines what issue(s) it concerns and to whom it should be delivered.
It's a neat idea, and although the interface is elegantly simple, it's clear both from meeting with Dave and from his announcement on the Sunlight Labs list that there's some serious horsepower under @2gov's hood. There's the classification engine, for one thing, which is being hand-tuned, but which Dave says is going to remain closed-source, making it not all that interesting from my perspective. And there's a voter-verification system based on exhaustively-collected voter rolls, which shows an impressive dedication to making sure that the service isn't merely spamming legislators. The whole thing's modular, too, opening the possibility of other, non-Twitter interfaces in the future.
What's most exciting to me is the data powering the site.
Continue reading- « Previous
- 1
- …
- 9
- 10
- 11
59063
