New programming tools and technologies have opened up a whole new way for people to explore government data online. We are just at the beginning of an explosion of civic resources built using this new technology. Here’s my story of building useful civic tools over the past two years.
I believe data is only useful when the people who need it can use it to make decisions. Some data is important for everyone to know and making it transparent means not just releasing it to the public, but giving the public tools to understand it. Too often, even though data is “public”, it doesn’t help the public’s understanding of the issues most affecting them.
One example of this is the U.S. Budget. The U.S. Budget has always been published, but it is published in a format that normal people couldn’t make any sense of. As a result, people are forced to make political decisions based on only vague ideas about how the U.S. spends and makes money.
My first civic project was to build a visualization to bring transparency to the U.S. Budget and you can see the result here. It is an interactive visualization of the U.S. Budget, going back 35 years so that you can understand exactly how the U.S. spends money, as well as get historical context for the spending you see.
After working on the budget visualization, I wanted to work on another civic problem where important data existed, but regular people weren’t able to use that data to make decisions. I was inspired by the many people working on campaign finance reform and wanted to try and build a tool to help.
All data work happens in stages. It is evolutionary, not revolutionary. New work is only possible because of the foundation that others have built. At the beginning there is no data, so the most important thing is to start collecting it. Next you need to make it accessible, even if only to people who can program. After that, you can make it accessible to everyone. The last stage is to make it useful for everyone. Campaign finance data has challenges at each of these stages.
The Center for Responsive Politics cleans Federal Election Commission campaign finance data and publishes it online. They act as a source for many researchers and journalists who need information on political fundraising. They also make their data available as a bulk download for those who have the programming ability to work with that kind of data. These raw data dumps power the Sunlight Foundation’s Influence Explorer project.
The challenge with all those data files, is that it takes a lot of programming knowledge to make them useful. I wanted to take those data dump files and turn them into a public database. That process is finished and open sourced here. Now, anyone who knows SQL can use this data.
However, most people can’t use SQL, so if the goal is to eventually turn this data into something that normal people can use to make decisions, we need to move on to the next stage.
A few people and organizations are working on this problem and each is approaching it from a different angle. Influence Explorer from the Sunlight Foundation focuses on connecting multiple data sources, incorporating federal and state data and showing information on fundraising, lobbying, earmarks and more.
I’m currently working on a project that takes a slightly different angle and goes deep into just the opensecrets federal campaign finance database. The project is an interactive visualization to explore politicians similar to how the budget visualization let people explore the budget. You will be able to look at lots of data at a high level and then zoom in to see more details when things seem interesting. With the help we received from a Sunlight Foundation OpenGov Grant, the project should be completed in 2015.
So, why should you care about this story?
I used to think that to have an impact was an all or nothing thing. Either I was completely devoting my career to fixing political issues, or I wouldn’t be able to accomplish anything. What the last two years have taught me is that there is a lot of space between all and nothing.
You, as a normal person with a normal job, can do important civic work in a few hours a week. This applies to both programmers and non-programmers. If you want to contribute, find a topic you care about, get in touch with an organization that deals with that topic and ask where they need help. If that doesn’t work and you don’t know where to start, feel free to reach out to me on twitter and I will send you to some projects that could really use your help.
Solomon Kahn is a Sunlight Foundation OpenGov Grantee and the Director of Analytics at Paperless Post. He is also an Edmond J. Safra Network Fellow at Harvard University. He can be found online at solomonkahn.com or on twitter @SolomonKahn
Interested in writing a guest blog for Sunlight? Email us at email@example.com