Sunlight Foundation

IBM introduces Many Bills

IBM Research recently released Many Bills, a new companion project to their wildly successful Many Eyes visualization tool set. Many Bills is billed as a visual bill explorer. More accurately, Many Bills is a web based, color coded visualization of 2009 U.S. congressional legislation. At its very heart the site tries to reveal what the different parts of a bill may be about by using computer learning to analyze and categorize the text of a particular piece of legislation. Because the analysis comes from a computer, they assign confidence scores to each section based on the likelihood that the categorization is correct.

While the default view can be a bit overwhelming at first, I've found that the "minified" view is a great way to review a series of legislative revisions/versions at a glance. More importantly, it also provides a quick and easy way to see where potentially unrelated legislation has been tacked on to a bill. For example, it is possible to see where the unrelated provision allowing visitors to U.S. National Parks to legally carry licensed, loaded firearms was inserted into the Credit Card Act of 2009. Just imagine if we could look at pending legislation in this "minified" color coded categorical view!

Sunlight recently spoke with the creators of Many Bills and offered a variety of feedback and suggestions for future features and options. We think there is a lot of potential for this project and we are really looking forward to the evolution of the Many Bills product. Congratulations to our friends at IBM!

Services, Resources and Tools for Mapping Data

Services, Resources and Tools for Mapping DataLong ago, putting together a map of data points would be the sole domain of a skilled GIS practitioner employing an application like ArcView. These days, particularly with the advent of Google Maps, Yahoo Maps and OpenStreetMap, et al., there are a multitude of options for an individual to employ in displaying data geographically. Of course, there are, and will always be, technical options that require some level of programming chops. Fortunately, the pool of drop dead easy implementations that anyone can throw together with ease has grown a lot over the last few years. Then, there is the growing middle ground, lying somewhere between easy but rigid and difficult but flexible. Personally, I tend to hover in this netherworld, leveraging existing code, services or tutorials when possible but occasionally finding myself diving into the more technical areas when necessary and learning a lot in the process.

For those of you out there who might be interested in mapping data, I've put together a collection of links to a variety of services, code samples, resources and tutorials I've found useful in the past. These links range from new services that barely require anything more than a spreadsheet to complicated frameworks that require a great deal of technical knowledge. This is by no means all encompassing and if you happen to have additional links you'd like to share, feel free to leave them in the comments. Services

  • Spatial Key "SpatialKey gives decision makers quick, easy, and cost-effective access to location intelligence so they can better achieve their objectives."
  • Many Eyes Many Eyes displays data associated with geographic regions using either colors or bubbles.
  • Maker "Anyone can build complex, data-rich maps."
  • Tableau "With Tableau Public you can create interactive graphs, dashboards, maps and tables from virtually any data and embed them on your website or blog in minutes."
  • Mapspread "Import your data, geocode it, share it with coworkers and friends, create thematic maps, create interactive applications."
  • Zeemaps Custom maps from lists.

<img src="http://assets.sunlightfoundation.com/images/blog/posts/mapping_coding.jpg" alt="Some Coding Required" class="aligncenter">
<ul id="nobullets">
<li><a href="http://cartographer.visualmotive.com/">Cartographer.js</a> (javascript)

A great javascript library that lets you create thematic maps by leveraging Google Maps.

  • Protovis (javascript) "A graphical toolkit for visualization" that can be used in conjunction with Google Maps.
  • MapBox "MapBox is a suite of open source tools to create beautiful custom maps in Amazon's cloud. "
  • ModestMaps (python, actionscript) "Modest Maps is a BSD-licensed display and interaction library for tile-based maps in Flash and Python."
  • ClearMaps (new, from SunlightLabs) "An action script mapping framework"
  • Mapfluence "Mapfluence is a suite of easy to use web mapping, visualization, and analytic APIs..."
  • Mapnik "Mapnik is a Free Toolkit for developing mapping applications."
  • TileCache "TileCache provides a Python-based WMS-C/TMS server, with pluggable caching mechanisms and rendering backends."
  • <li><a href="http://code.google.com/p/gheat/">gheat</a>
    

    "A map tile server for a heatmap layer" on top of Google Maps.

    <img src="http://assets.sunlightfoundation.com/images/blog/posts/mapping_resources.jpg" alt="Resources" class="aligncenter">
    <ul id="nobullets">
    <li><a href="http://openlayers.org/">OpenLayers</a>
    

    "OpenLayers makes it easy to put a dynamic map in any web page. It can display map tiles and markers loaded from any source."

  • Raphael (javascript) "Raphaël is a small JavaScript library that should simplify your work with vector graphics on the web."
  • ColorBrewer "Color advice for Cartography"
  • Geocoder "Convert multiple addresses to GPS coordinates"
  • Google's Geocoding service
  • <li><a href="http://www.batchgeocode.com/">BatchGeoCode</a>
    

    Batch geocode addresses in a spreadsheet with ease.

  • Mapstraction "Mapstraction is a library that provides a common API for various javascript mapping APIs to enable switching from one to another as smoothly as possible."
  • </ul>
    
    <img src="http://assets.sunlightfoundation.com/images/blog/posts/mapping_tutorials.jpg" alt="Tutorials" class="aligncenter">
    <ul id="nobullets">
    <li><a href="http://flowingdata.com/2009/11/12/how-to-make-a-us-county-thematic-map-using-free-tools/">Flowing Data: Thematic Maps</a> (choropleth with Python, SVG)</li>
    <li><a href="http://geography.uoregon.edu/GeogR/topics/maps.htm">Maps in R</a></li>
    <li><a href="http://blog.blprnt.com/blog/blprnt/7-days-of-source-day-1-goodmorning">GoodMorning</a> (processing based)</li>
    <li><a href="http://flowingdata.com/2009/11/05/make-thematic-maps-with-cartographer-js/">Thematic Maps with Cartographer.js</a></li>
    <li><a href="http://mapbox.com/documentation/adding-tiles-your-site/using-mapbox-tiles-google-maps-api/">Using MapBox tiles with Google Maps</a></li>
    <li><a href="http://vis.stanford.edu/protovis/ex/oakland.html">Recreating Oakland Crimespotting with Protovis</a> (See: <a href="http://oakland.crimespotting.org/">Oakland.Crimespotting</a>)</li>
    <li><a href="http://econym.org.uk/gmap/">Google Maps API Tutorial</a></li>
    <li><a href="http://gmaps-samples.googlecode.com/svn/trunk/spreadsheetsmapwizard/makecustommap.htm">Generate a Google Map from a spreadsheet</a></li>
    

  • Quickly visualize and map a data set using Google Fusion Tables
  • This TED 2010 talk is also worth watching to see the future of online mapping.

    2009 in Capitol Words + Colors

    CapitolWords.org is a Sunlight Foundation site that "visualizes the most frequently used words in the Congressional Record, giving you an at-a-glance view of which issues lawmakers address on a daily, weekly, monthly and yearly basis." As 2009 began winding down, I started poking around at the data for the year to see if I could come up with an interesting way to visually summarize the year in Congress. With a total of 58 unique words commanding the top slots over the course of the year, I thought it might be interesting to color code and present them as a sort of "at a glance" Congressional calendar. As you can see below, the word Health pretty much dominated the latter half of the year, while the first half of 2009 was dedicated to a variety of topics including Land, Energy and Credit. It is also easy to see when Congress does a lot of its work on the floor and when they tend to take breaks throughout the year (April and August, apparently).

    If we look a little deeper, for example using the word "Credit", we can see that there was a lot of use of the word in May '09, which happens to correlate nicely with the signing of the Credit CARD Act of 2009. Of course, not all of the top words are directly related to a particular bill, and that's part of the fun of Capitol Words. Hover over the various boxes/days and see if you can find any interesting correlations, or just marvel at the year that was 2009.

    Some notes:

    • I didn't set a threshold for inclusion, choosing to use the exact data that Capitol Words provides in the API, so you'll see that some days the top word was only spoken a handful of times. Such is the nature of the Congressional Record and CapitolWords.org.
    • The "calendar" was generated using Processing and uses jquery QTip to show the metadata for the day.
    • Aside from assigning "Health" a blue value, the other 57 words were randomly assigned colors from a palette generated using ColorSchemeDesigner.
    • visually/conceptually inspired by colorjack.

    Visualizing Twitter's Political Zeitgeist

    Back in mid-November, I had the pleasure of attending and presenting at the kickoff of Medialab Prado's Visualizar '09 event in Madrid, Spain. The theme of this year's Visualizar seminar was Public Data, Data in Public, a topic near and dear to Sunlight's heart. The program kicked off with a day of public proposal presentations followed by a day of presentations from a variety of visualization visionaries, including Aaron Koblin and Stamen. Following the 2 day seminar, the real work began as the guest tutors and collaborators got to work on their proposed projects. Two weeks later, they presented the fruits of their labors. One such project to come out of this year's Visualizar '09 is New Political Interfaces, from Barcelona's Aer Studio. This projects presents a pair of flash based visualizations focused on politics and is based on data pulled from Twittersphere.

    Their first visualization compares various topics to determine correlations between what is being discussed by politicians, bloggers, and the traditional media outlets. Consider it a sort of visual political zeitgeist, through a twitter lens. new_political Their second visualization looks at the decay rate of a retweet over time, revealing the short attention span of the political twitterati. npi_retweet

    While easy on the eyes and visually compelling, neither of these visualizations need necessarily be focused specifically on "politics." Nonetheless, they provide an interesting view into the political realm and allow for a new and novel approach to comparing political "sentiment" as communicated via twitter.

    Local Spotlight: Mapping Texas

    Texas Watchdog is a great organization working toward state level transparency in Texas, they just released a map with all Texas lawmakers on it with  links to all kinds of great information about them.  It is a great visual and an example of how information digging doesn’t have to be reserved for spreadsheets.

    Where do Texas’ lawmakers make their money? Who are they in business with? Texas Watchdog has the details (Looking for a specific lawmaker’s 2009 ethics form? Click here to skip down to a list of legislators: Senators / House members)

    Texas Watchdog presents its 2009 map of Texas’ 181 lawmakers, including their state ethics forms and useful links that will allow the public to research each lawmaker’s campaign contributions, the bills they pushed in the last legislative session and contact information so the average citizen can let their voice be heard. The map is intended to be an update of our 2008 map. The legislators’ 2008 ethics forms are integrated into this new map, but if you want to see how this map is an improvement over last year’s, we encourage you to look at the old map.

    Texas legislators and many other state officials are required by law to fill out annual forms, formally titled “personal financial statements,” to the state Ethics Commission detailing their sources of income. Required categories include for whom they work, what stocks and bonds they own, and their business interests. State officials are supposed to file by April, but they are allowed to seek a one-time extension of the deadline until June. Readers will note that a few lawmakers have more than one ethics form for 2009.  That’s because they turned in their form and later turned in a corrected version. We’ve tried to present all the versions of the 2009 forms here.

    Read More Here

    The Max Baucus Health Care Lobbyist Complex

    Mapping Max Baucus' Health Care Lobbyist Complex. Click image for full visualization.

    As the chair of the Senate Finance Committee, Sen. Max Baucus is at the center of the congressional effort to craft health care reform legislation, a top priority of President Barack Obama. The Baucus-headed Finance Committee has been singled out by advocates and news organizations as the toughest obstacle for the President's health care priorities. Containing more moderate and conservative members may not be the only reason. The committee is packed with lawmakers who have close ties to the health care and insurance industries, receiving large campaign contributions as their former staffers turn around to lobby for the very interests whose issues -- in this case health care -- they previously worked on. Baucus, as chair, stands out in particular.

    Lobbying disclosure filings for the first quarter of 2009 reveal that five of Baucus' former staffers currently work for a total of twenty-seven different organizations that are either in the health care or insurance sector or have a noted interest in the outcome. The organizations represented include some of the top lobbying organizations in the health sector: Pharmaceutical Manufacturers and Researchers of America (PhRMA), America's Health Insurance Plans (AHIP), Amgen, and GE Health Care.

    The former staffers turned lobbyists include two former chiefs of staff, David Castagnetti and Jeff Forbes, and one former legislative assistant, Scott Olsen. Other former staffers working with health care portfolios include Angela Hoffman and Roger Blauwet.

    The overall health and insurance sectors haven't just been kind to Baucus' staffers, but they've also aided his campaigns handsomely over the years, especially in his barely contested 2008 reelection campaign. In 2008, Baucus received $1,148,775 from the health sector and $285,850 from the insurance sector. For his career he has received $2,797,381 from the health sector and $1,170,313 from the insurance sector.

    The accompanying visualization shows the connections from Baucus to his staffers-turned-lobbyists to their health care sector clients, which, in some cases, overlap. Most of the organizations are directly involved in the health care or insurance industries. A couple, the Business Roundtable and Wal-Mart, may seem to fall outside of the realm of health care, however both are playing key roles. The Business Roundatble is lobbying heavily on the issue and Wal-Mart is a big seller of prescription medications and has a large stake in the outcome.

    All data comes from OpenSecrets.org.

    Earmark My Words

    What do top earmarkers talk about in Congress? Does our money go where their mouths are?

    In the case of the top ten earmarkers for FY 2008, the top words they used from 2007-2008 (110th Congress) do often align with their duties in either the Appropriations Committee or in bringing home the bacon to their home state. Six of the top ten use appropriations-related language in their top words and three use their state's name in their top words.

    The top ten earmarkers for FY 2008 were, in descending order with top word in parentheses, Rep. John Murtha (Billion), Rep. Jerry Lewis (Appropriations), Rep. C.W. "Bill" Young (Defense), Rep. Pete Visclosky (Indiana), Rep. David Obey (Billion), Rep. Norm Dicks (Million), Rep. Marcy Kaptur (Trade), Rep. Harold Rogers (Kentucky), Rep. Ike Skelton (Military), Rep. Chet Edwards (Veterans). Only one of these lawmakers (Rep. Skelton) is not on the House Appropriations Committee.

    Three of these lawmakers -- Reps. Lewis, Murtha and Visclosky -- are either under federal investigation or have been mentioned in connection to an investigation in relation to their earmarking practices.

    Check out the following word cloud visualization to see what these earmarking lawmakers are talking about. Below the visualization is a list of the Appropriations committee assignment for the nine lawmakers on the committee.

    Appropriations Committee Assignments
    Rep. John Murtha Chairman, Defense Appropriations Subcommittee
    Rep. Jerry Lewis Ranking Member, House Appropriations Committee
    Rep. C.W. "Bill" Young Ranking Member, Defense Appropriations Subcommittee
    Rep. Pete Visclosky Chairman, Energy and Water Appropriations Subcommittee (currently surrendered position); Defense Appropriations Subcommittee
    Rep. David Obey Chairman, House Appropriations Committee; Chairman, Labor, Health and Human Services, Education, and Related Agencies Appropriations Subcommittee
    Rep. Norm Dicks Chairman, Interior, Environment, and Related Agencies Appropriations Subcommittee; Defense Appropriations Subcommittee; Military Construction, Veterans Affairs, and Related Agencies Appropriations Subcommittee
    Rep. Marcy Kaptur Defense Appropriations Subcommittee; Agriculture Appropriations Subcommittee; Transportation, HUD Appropriations Subcommittee
    Rep. Harold Rogers Ranking Member, Homeland Security Appropriations Subcommittee; Defense Appropriations Subcommittee
    Rep. Ike Skelton Not on Appropriations Committee
    Rep. Chet Edwards Chairman, Military Construction, Veterans Affairs, and Related Agencies Appropriations Subcommittee; Financial Services and General Government Appropriations Subcommittee
    Note: Earmark data comes via Taxpayer.net. Word data comes from CapitolWords.org. Only House lawmakers were used due to less than stellar earmark disclosure by the Senate. And thanks again to Kerry Mitchell for the visuals.

    Fun with Lines and Dots and Open Source Code

    Anthony Mattox, a very talented student at Maryland Institute College of Art (MICA), recently released some processing code called WikiWeb to visualize the connections between pages at Wikipedia.

    I read about the project and thought, "I wonder if that would work with OpenCongress's wiki?" I grabbed the open sourced creative commons code, changed a few URLs to point to OpenCongress's Wiki and voilà.

    wikiweb_opencongress

    It is running in the browser as a Java applet, so you will need to "trust" it before it will "work." Don't worry, it doesn't do anything nefarious. It just needs your trust before it can go to OpenCongress and fetch pages from the wiki.

    Once it loads, you'll want to click the dot in the middle "Main Page" and select the first option on the left of the menu which will "load links" and then you're off. Any dot (which represents a page of the wiki) can be expanded to show the other pages it links to.

    Just something fun to demonstrate the value of open source, Creative Commons and of course Processing. Thanks to Anthony for making the code available!

    « Previous
    1 2