Visualization

 

How the Parties Flip-Flopped on the Debt Ceiling

Because of some the work we've done before on last minute negotiations and divided government, Sunlight prepared the following graphic that visualizes the recent history of US House votes on the debt ceiling, based on public voting records and a CRS report. The bars indicate the "yea" votes.

 

Sunlight Foundation infographic on the votes to raise the debt limit.

We'll have more commentary forthcoming, but here are a few initial thoughts on what this graphic makes clear:

  1. Opposition to raising the debt ceiling is often partisan, with opposition coming from either party, based on who is in the White House. Many House Republicans have voted for raising the ceiling, just as President Obama voted against it when he was a Senator.
  2. Divided government has necessitated support from both parties to raise the limit.
  3. There is a significant untold story about the Gephardt Rule, a House Rule which enabled the limit to be raised with little public record. The role this rule played in setting up the current showdowns has been insufficiently examined.
  4. Good access to congressional data and reports enables this kind of analysis; it could be improved.
  5. Each of these votes was a predictable consequence of budgets that were passed before them, demonstrating another facet of political hypocrisy.

Daniel Schuman, Zander Furnas, Caitlin Weber and Matt Rumsey contributed to this post.

Open States Source Visualized

Open States recently reached a milestone in that we now support 40 states (and DC and Puerto Rico) and at our current pace we'll reach our goal of all 50 states by sometime early next year. It is only due to the fantastic support of our community and indviduals who have showed up at hackathons or just started contributing on their own that this goal is now in sight.

I thought it might be fun to look back on how the project has grown, and luckily gource is a piece of software for visualizing the history of a repository can help do just that. Watch below to enjoy a visually stimulating look back through the last two and half years of commits to Open States. You'll see flurries of activity around our hackathons, the drastic increase in activity from 2009 to 2010 and how 2011 so far takes up more than half the video, and some of the big refactors that we've made along the way to scale the project to a size well beyond what we initially conceived of.

Read more

Data Visualization Fellowship

We've got a new job listing up, and I hope you'll have a look. If you do, you'll see that we're doing something new. This position came about because we decided that we wanted to create more and better data visualizations -- they're interesting, people like them, and they're a great opportunity to experiment with new technologies.

But as we started thinking through how to staff this position, we realized we didn't really want someone who was an expert in d3, or processing.js, or any other presentation technology. Don't get me wrong: finding someone with those skills for this position would be great. But we already have a bunch of talented front-end developers and designers. I think we can present answers in beautiful and compelling ways; what I could really use are better questions.

So, like I said, we're looking for something a little different. The listing says "quantitative social scientist," but you could easily substitute the "data scientist" buzzword that the tech industry seems to be embracing. Whatever you call it, what we're looking for boils down to this: we need someone with the ability to understand the questions that can reasonably be asked of our data; someone who knows the questions that people have asked of the data in the past; and who is be able to find some decent answers of her own. At Sunlight, those questions are likely to be about the U.S. government and the entities that try to influence it. Once you've got an interesting answer, we'll throw all the Javascript and CSS at it that you could ever want.

So please have a look, and if you know folks who you think would be a good fit, pass the link along to them. And if you yourself are thinking about applying, please don't be scared off by the specific requirements -- they describe what we think an ideal candidate would be, but we know that we're likely to find some surprises. This fellowship is a bit of an experiment for us, but I'm excited about the possibilities it represents.

Read more

Data Visualeggzation

Months ago, Josh and Tim bought an Egg-Bot kit from Evil Mad Scientist Laboratories. Despite the obvious utility of this piece of office equipment, it fell into disuse not too long after assembly. But with the year's premiere egg-decorating holiday fast approaching, we decided to dust off the Egg-Bot and see if we couldn't put it to good use during our team's weekly lunch meeting. Things kind of spiraled out of control from there. We blame the sugar high from eating all that candy.

Read more

OpenGovernment Is A Finalist For SXSW Accelerator

We've been really pleased with the enthusiastic response that OpenGov has received since its launch. Today's brought one more bit of good news: the site's been named a finalist in the Accelerator competition of this year's SXSWi. You can find the full details over at the OG blog. Congratulations to David, Carl and the rest of the PPF team!

And while I've got you thinking about state legislative data, check out what Gabriel Florit's done to visualize data from Open States. It's very neat stuff, and a great early example of the kinds of things that we think the project will make possible.

Read more

IBM introduces Many Bills

IBM Research recently released Many Bills, a new companion project to their wildly successful Many Eyes visualization tool set. Many Bills is billed as a visual bill explorer. More accurately, Many Bills is a web based, color coded visualization of 2009 U.S. congressional legislation. At its very heart the site tries to reveal what the different parts of a bill may be about by using computer learning to analyze and categorize the text of a particular piece of legislation. Because the analysis comes from a computer, they assign confidence scores to each section based on the likelihood that the categorization is correct.

While the default view can be a bit overwhelming at first, I've found that the "minified" view is a great way to review a series of legislative revisions/versions at a glance. More importantly, it also provides a quick and easy way to see where potentially unrelated legislation has been tacked on to a bill. For example, it is possible to see where the unrelated provision allowing visitors to U.S. National Parks to legally carry licensed, loaded firearms was inserted into the Credit Card Act of 2009. Just imagine if we could look at pending legislation in this "minified" color coded categorical view!

Sunlight recently spoke with the creators of Many Bills and offered a variety of feedback and suggestions for future features and options. We think there is a lot of potential for this project and we are really looking forward to the evolution of the Many Bills product. Congratulations to our friends at IBM!

Services, Resources and Tools for Mapping Data

Services, Resources and Tools for Mapping DataLong ago, putting together a map of data points would be the sole domain of a skilled GIS practitioner employing an application like ArcView. These days, particularly with the advent of Google Maps, Yahoo Maps and OpenStreetMap, et al., there are a multitude of options for an individual to employ in displaying data geographically. Of course, there are, and will always be, technical options that require some level of programming chops. Fortunately, the pool of drop dead easy implementations that anyone can throw together with ease has grown a lot over the last few years. Then, there is the growing middle ground, lying somewhere between easy but rigid and difficult but flexible. Personally, I tend to hover in this netherworld, leveraging existing code, services or tutorials when possible but occasionally finding myself diving into the more technical areas when necessary and learning a lot in the process.

For those of you out there who might be interested in mapping data, I've put together a collection of links to a variety of services, code samples, resources and tutorials I've found useful in the past. These links range from new services that barely require anything more than a spreadsheet to complicated frameworks that require a great deal of technical knowledge. This is by no means all encompassing and if you happen to have additional links you'd like to share, feel free to leave them in the comments. Services

  • Spatial Key "SpatialKey gives decision makers quick, easy, and cost-effective access to location intelligence so they can better achieve their objectives."
  • Many Eyes Many Eyes displays data associated with geographic regions using either colors or bubbles.
  • Maker "Anyone can build complex, data-rich maps."
  • Tableau "With Tableau Public you can create interactive graphs, dashboards, maps and tables from virtually any data and embed them on your website or blog in minutes."
  • Mapspread "Import your data, geocode it, share it with coworkers and friends, create thematic maps, create interactive applications."
  • Zeemaps Custom maps from lists.

<img src="http://assets.sunlightfoundation.com/images/blog/posts/mapping_coding.jpg" alt="Some Coding Required" class="aligncenter">
<ul id="nobullets">
<li><a href="http://cartographer.visualmotive.com/">Cartographer.js</a> (javascript)

A great javascript library that lets you create thematic maps by leveraging Google Maps.

  • Protovis (javascript) "A graphical toolkit for visualization" that can be used in conjunction with Google Maps.
  • MapBox "MapBox is a suite of open source tools to create beautiful custom maps in Amazon's cloud. "
  • ModestMaps (python, actionscript) "Modest Maps is a BSD-licensed display and interaction library for tile-based maps in Flash and Python."
  • ClearMaps (new, from SunlightLabs) "An action script mapping framework"
  • Mapfluence "Mapfluence is a suite of easy to use web mapping, visualization, and analytic APIs..."
  • Mapnik "Mapnik is a Free Toolkit for developing mapping applications."
  • TileCache "TileCache provides a Python-based WMS-C/TMS server, with pluggable caching mechanisms and rendering backends."
  • <li><a href="http://code.google.com/p/gheat/">gheat</a>
    

    "A map tile server for a heatmap layer" on top of Google Maps.

    <img src="http://assets.sunlightfoundation.com/images/blog/posts/mapping_resources.jpg" alt="Resources" class="aligncenter">
    <ul id="nobullets">
    <li><a href="http://openlayers.org/">OpenLayers</a>
    

    "OpenLayers makes it easy to put a dynamic map in any web page. It can display map tiles and markers loaded from any source."

  • Raphael (javascript) "Raphaël is a small JavaScript library that should simplify your work with vector graphics on the web."
  • ColorBrewer "Color advice for Cartography"
  • Geocoder "Convert multiple addresses to GPS coordinates"
  • Google's Geocoding service
  • <li><a href="http://www.batchgeocode.com/">BatchGeoCode</a>
    

    Batch geocode addresses in a spreadsheet with ease.

  • Mapstraction "Mapstraction is a library that provides a common API for various javascript mapping APIs to enable switching from one to another as smoothly as possible."
  • </ul>
    
    <img src="http://assets.sunlightfoundation.com/images/blog/posts/mapping_tutorials.jpg" alt="Tutorials" class="aligncenter">
    <ul id="nobullets">
    <li><a href="http://flowingdata.com/2009/11/12/how-to-make-a-us-county-thematic-map-using-free-tools/">Flowing Data: Thematic Maps</a> (choropleth with Python, SVG)</li>
    <li><a href="http://geography.uoregon.edu/GeogR/topics/maps.htm">Maps in R</a></li>
    <li><a href="http://blog.blprnt.com/blog/blprnt/7-days-of-source-day-1-goodmorning">GoodMorning</a> (processing based)</li>
    <li><a href="http://flowingdata.com/2009/11/05/make-thematic-maps-with-cartographer-js/">Thematic Maps with Cartographer.js</a></li>
    <li><a href="http://mapbox.com/documentation/adding-tiles-your-site/using-mapbox-tiles-google-maps-api/">Using MapBox tiles with Google Maps</a></li>
    <li><a href="http://vis.stanford.edu/protovis/ex/oakland.html">Recreating Oakland Crimespotting with Protovis</a> (See: <a href="http://oakland.crimespotting.org/">Oakland.Crimespotting</a>)</li>
    <li><a href="http://econym.org.uk/gmap/">Google Maps API Tutorial</a></li>
    <li><a href="http://gmaps-samples.googlecode.com/svn/trunk/spreadsheetsmapwizard/makecustommap.htm">Generate a Google Map from a spreadsheet</a></li>
    

  • Quickly visualize and map a data set using Google Fusion Tables
  • This TED 2010 talk is also worth watching to see the future of online mapping.

    2009 in Capitol Words + Colors

    CapitolWords.org is a Sunlight Foundation site that "visualizes the most frequently used words in the Congressional Record, giving you an at-a-glance view of which issues lawmakers address on a daily, weekly, monthly and yearly basis." As 2009 began winding down, I started poking around at the data for the year to see if I could come up with an interesting way to visually summarize the year in Congress. With a total of 58 unique words commanding the top slots over the course of the year, I thought it might be interesting to color code and present them as a sort of "at a glance" Congressional calendar. As you can see below, the word Health pretty much dominated the latter half of the year, while the first half of 2009 was dedicated to a variety of topics including Land, Energy and Credit. It is also easy to see when Congress does a lot of its work on the floor and when they tend to take breaks throughout the year (April and August, apparently).

    If we look a little deeper, for example using the word "Credit", we can see that there was a lot of use of the word in May '09, which happens to correlate nicely with the signing of the Credit CARD Act of 2009. Of course, not all of the top words are directly related to a particular bill, and that's part of the fun of Capitol Words. Hover over the various boxes/days and see if you can find any interesting correlations, or just marvel at the year that was 2009.

    Some notes:

    • I didn't set a threshold for inclusion, choosing to use the exact data that Capitol Words provides in the API, so you'll see that some days the top word was only spoken a handful of times. Such is the nature of the Congressional Record and CapitolWords.org.
    • The "calendar" was generated using Processing and uses jquery QTip to show the metadata for the day.
    • Aside from assigning "Health" a blue value, the other 57 words were randomly assigned colors from a palette generated using ColorSchemeDesigner.
    • visually/conceptually inspired by colorjack.