In a welcome turn toward transparency, House Appropriators are now set to webcast all public hearings starting this week. As late as last week, only 2 of the 10 meetings for this week were scheduled to be online. All committees except Appropriations have webcast the vast majority of their proceedings since the Republican leadership's passage of a House rule at the start of the 112th Congress requiring webcasting "to the maximum extent practicable."

Of special note is the webcasting of several proceedings that will take place inside in the Capitol building itself, in room HT-2. This tiny room is often too small to admit all interested members of the public, especially with the large staff entourage that often accompanies testimony by legislative branch agency leadership. (Here's what it looks like.) C-SPAN does not usually cover the committee's proceedings, so the only way to see what happens is to go and hope you get in. Whatever issues that had existed with webcasting from this room apparently have been resolved, and a new precedent has been set in favor of transparency.

We'll be watching.

How are House and Senate candidates' war-chests faring this election cycle? The animations below show who has been pulling further ahead, and who has been closing fund-raising gaps as the races mature.

Itemized contributions, which are reported to the FEC along with the date the contributions were made, have allowed us to reconstruct a detailed accounting of the funds available to candidates over time. Beginning with the first quarter of 2011, we looked at all itemized contributions made to candidates in competitive races (rated by Cook as Likely, Lean or Toss up within the last month) reported to the FEC up through the end of the second quarter of this year (the most recent filing deadline). This does not include Independent Expenditures supporting or opposing candidates by outside groups, nor "dark money" expenditures from 501 (c) organizations like Crossroads GPS.

These animations show the evolving status of the funding landscape in competitive Senate and House races through the second quarter of 2012. The Democratic funding advantage is displayed on the y-axis; bars with positive values indicate that the Democratic candidate leads in fund-raising, while a negative value indicates that the Republican candidate leads. Races are colored according to the party which previously held the senate seat, either as an incumbent running in this cycle or a retiring senator.

Money in Senate races over time:

The time series animation above highlights a few trends. The first is that the party that currently controls the seat tends to lead in funding. This is unsurprising, given incumbents’ inherent fund-raising advantages. North Dakota and Hawaii are interesting exceptions to this. (Races can be highlighted by clicking their check-box on the right of the graph.) A partial explanation is that both races are for open seats. So current democratic candidates do not benefit from true incumbency advantages in fund-raising potential.

Another notable pattern we see is that challengers who join the race later tend to fund-raise faster than the incumbent once they join the race. Massachusetts is a particularly strong example of this effect, where Democratic challenger Elizabeth Warren got into the race comparatively late, but once she did, she caught up with incumbent Senator Scott Brown (R) quickly. This also occurred somewhat in Virginia, Hawaii, Ohio and Nebraska.

Below is a similar time-series chart for competitive House races. This graphic also displays the Democratic funding advantage in each race, based on itemized contributions reported to the FEC through the second quarter of 2012. Once again color of the bar indicates which party controlled the seat in the prior Congress. New seats created by redistricting in which neither party has historical control are denoted by green.

Money in House races over time:

The major trend visible above is that in House races, funding gaps between parties seem to grow over time. Once again, candidates trying to hold a seat (e.g. a democrat running in a district in which the seat was controlled by the democratic party in the last cycle) tend to do better in terms of fund-raising than candidates trying to flip a seat.

This effect is not determinative in all cases, however. In in the AR-04, AZ-01, CA-09, CA-47, IN-02, MD-06, NC-13, and the OK-02 candidates challenging for a seat previously held by the opposing party are out performing the candidate from the incumbent party.

We also see a "rebound effect" similar to what was observed in the Senate races, in which money comes in faster later on to overcome initial leads in some of the races. The FL-22, IL-08, MA-06, NC-08, and the NV-03 are all interesting to watch in this regard.

As these races continue and more data is available, graphics of this kind will become increasingly informative. Ultimately we will be able to pair this time-series funding data with time-series polling. This will provide a window into how changes in funding advantages and changes in polling leads are correlated.

The media's magnifying glass is concentrating attention on actions by the House Appropriations Committee that could stall progress on the public's access to legislative information. The Sunlight Foundation and our allies continue to push Congress to stop dragging their feet and join the 21st century by allowing developers access to open legislative data to build the tools to keep citizens informed about what their government is doing.

Please find and call your Representative at 202-224-3121 or write to reinforce the American public's hunger to read and follow legislation. Here are some excerpts from recent media coverage on this important transparency issue:

Roll Call reports on Republican House leadership's strong support for bulk access and quotes Rep. Crenshaw misunderstanding the issue of authentication:

“The Speaker pledged to make the 112th Congress the most open and transparent Congress in history and to make legislative data available online and in bulk,” said Michael Steel, spokesman for Speaker John Boehner (R-Ohio). “He continues to look for the best way to do that.”

“Facilitating public access to bulk legislative data ... has been and will continue to be a priority for this committee,” echoed Salley Wood, spokeswoman for the House Administration Committee. But lawmakers’ hands would be tied until a task force could be convened and report back on its findings, according to the House report language.

“We wanted to create a system where we could have this available but also make sure we protect the authenticity and integrity of all this information,” said Rep. Ander Crenshaw (R-Fla.), chairman of the Appropriations Subcommittee on the Legislative Branch.

Folks with computers -- notably, professional and citizen journalists -- would be able to take information about massive numbers of bills and analyze them in myriad ways -- if Congress would allow such information to be downloaded from THOMAS in bulk.

It won't. And, according to a new draft report from the House Appropriations Committee, it won't be allowing bulk data downloads from THOMAS anytime soon.

Instead of taking a step towards greater transparency, the committee got hung up on whether people would know if the data they're seeing on the Internet were accurate and really from Congress -- "authentication," they call it.

The report retains language decried by transparency opponents that would indefinitely postpone public bulk downloads of legislative information in XML. Good government groups, including the Sunlight Foundation, have pressed for the Library of Congress to release the bulk data used to track legislative developments in the library's THOMAS website, arguing that they could do a better job of presenting information.

Open government advocates are up in arms over what appears to be another attempt by government bureaucrats to stall the move to enable bulk data downloads of legislative information online.

Slashdot opens the issue for conversation to their community:

The House Appropriations Committee is considering a draft report that would forbid the Library of Congress to allow bulk downloads of bills pending before Congress. The Library of Congress currently has an online database called THOMAS (for Thomas Jefferson) that allows people to look up bills pending before Congress. The problem is that THOMAS is somewhat clunky and it is difficult to extract data from it. This draft report would forbid the Library of Congress from modernizing THOMAS until a task force reports back. I am pretty sure that the majority of people on Slashdot agree that being able to better understand how the various bills being considered by Congress interact would be good for this country.

Legal Informatics also has a nice collection of blog posts on this issue.

Follow the latest developments here.

Does information about legislation belong to Congress or to the American people? This basic question is at the heart of a fight over how Congress releases data about what it does. Americans increasingly use the Internet to make sense of the world around them, and open data opens up Congress in a way that's never been possible before.

In the pre-YouTube pre-iPhone pre-Amazon days, Congress built a website -- THOMAS -- to let citizens follow legislation from home. THOMAS was revolutionary ... in 1995. But the Internet continued to develop, becoming more sophisticated and interactive, allowing web developers to easily share the data behind their websites with others. It's why we can book flights on Travelocity, check the weather on our phones, and follow legislation on OpenCongress and GovTrack.

Unlike Travelocity and the National Weather Service, Congress doesn't share the data behind THOMAS with anyone. Instead, web developers must reverse-engineer the website to transmute its pages into usable data, like assembling a puzzle from thousands of ragged pieces without a picture on the box as a guide. This slow, difficult, and time-consuming process isn't perfect, but it's responsible for how most Americans follow what's happening in Congress.

The better approach is for Congress to publish the data behind THOMAS. Government regularly does this elsewhere, and "bulk data" is responsible for clever new uses of information developed by citizens, journalists, and even the government itself.

In upcoming days, the House is likely to pass legislative language that pays lip service to releasing THOMAS data while putting the idea in a deep freeze. This would be a disaster. But it's not too late. Tell your representative that you want Congress to publish legislative data now.

PS. For more information and the latest developments, go here. 

In the last 18 months, the House of Representatives has made significant strides towards greater openness and transparency in congressional deliberations, but significant work remains. The Legislative Branch Appropriations Bill for 2013, which was marked-up by a subcommittee last week, presents a major vehicle for the House leadership to make good on its promise to implement common-sense transparency measures this session.

While there are many issues that can be addressed a number of different ways, Sunlight will be looking at  the full committee markup to see if the bill:

-- Provides bulk access to THOMAS data

-- Fully funds the Office of Congressional Ethics

-- Requires Publication of  CRS Reports online

-- Publishes the Constitution Annotated online as it's updated in XML

-- Reinstates the Office of Technology Assessment

-- Makes reports to Congress available online

-- Publishes House spending information in an appropriate format for the data

Improve Public Access to THOMAS Data

THOMAS was created by Congress to make legislative information freely available to the public, but the Library of Congress has not kept up with best practices. One such practice -- "bulk access" -- would ease the development of new tools and technologies by publishing THOMAS data files online, promoting accurate and timely information dissemination. Congress has expressed its support for bulk data as have many organizations, but the Library continues to stall despite a 2008 memo describing how easy it would be to implement.

At the recent legislative subcommittee hearing, Rep. Honda mentioned that text has been inserted into the committee's report that would in some way address the bulk data question. The last time this happened, the language was watered down sufficiently so that the Library of Congress successfully evaded its obligations over the last half a decade. We hope the bill will contain these two provisions:

(1) Congress directs the Library of Congress to implement bulk access to THOMAS within 120 days of passage

(2) Congress directs the Library of Congress to immediately create an advisory committee on improving public access to legislative information that is composed of people inside and outside of government.

Fully Fund the Office of Congressional Ethics

The Office of Congressional Ethics is the House of Representatives' independent ethics watchdog. It came into existence in March 2008 after a series of corruption scandals prompted congressional leaders to explore creating a transparent, outside enforcement entity. While OCE is not as robust as originally contemplated, it plays a crucial role in ethics oversight. Last year, the office survived a counterproductive effort by nearly 100 members of Congress to significantly reduce its funding. This year's appropriations bill maintains OCE's funding at $1,548,000, which is the same level as last year. We believe that OCE should be strengthened, but at a minimum, its funding should be sustained at least at this level.

Publish CRS Reports Online

Congressional Research Service reports undergird the public's understanding of Congress, but CRS no longer directly releases the reports to the public. As a consequence, while many reports used by citizens, courts, and government employees are on the internet, they are often out-of-date, and a fair number are available only for a fee or not at all. By comparison, sister agencies like CBO and GAO regularly publish reports online. For more than a decade, organizations and members of Congress have urged that CRS reports be publicly available, and CRS concerns have been refuted by a former counsel to the House of Representatives. The reports are already digitized and available on Congress's intranet; it would take a trivial effort to publish them online.

During the markup of the 2012 Appropriations Bill, Rep. Leonard Lance introduced an amendment that would have required the Clerk of the House and the Secretary of the Senate to maintain a website containing CRS Reports and Appropriation products while protecting confidential advice from CRS. Similar legislation has been introduced by Rep. Quigley. We hope that House Appropriators will move to make these reports more readily available to the public.

Release the Constitution Annotated Online

The Constitution Annotated (or CONAN) is a continuously-updated 100-year-old legal treatise that explains the Constitution as it has been interpreted by Supreme Court. Maintained by CRS and printed by GPO, a hard copy is published (and put online) only once a decade, with printed updates every two years. However, CONAN is updated frequently, with those updates available on Congress' internal website. In November 2010 (18 months ago), the Joint Committee on Printing directed that the continuously-updated version of CONAN be made available online as a searchable PDF, but it still is not. Many organizations have asked that the underlying document be published online in its original (XML) format, which is more user friendly than a PDF, and would take minimal effort to release.

This upcoming year, the Constitution Annotated will be up for its once-a-decade print edition. With at least 4,870 statutorily mandated copies, at an estimated cost of $226, the House and Senate will pay over $1.1 million for a document that will go out of date almost immediately. We suggest that some of these costs may be recouped by asking House offices if they wish to receive a print copy, as a continuously updated web version is already made available to all congressional offices. Regardless, we urge that the web version that is already made available to congressional offices also be made available to the American people in its web friendly format. While publishing the document as a PDF would be a small step forward, the best use of taxpayer dollars to maximize usability would be to publish it in XML, the format in which it is prepared.

Other Provisions

Sunlight support additional measures in the Legislative Branch Appropriations bill. Those provisions include:

The reinstatement of the Office of Technology Assessment, as proposed by Rep. Rush Holt last year. OTA provided Congress http://sunlightfoundation.com/blog/taxonomy/term/Office-of-Technology-Assessment/ with the “means for securing competent, unbiased information concerning the physical, biological, economic, social, and political effects” of technology.

Inclusion of the Access to Congressionally Mandated Reports Act, which would would gather together all reports to Congress from federal agencies in one place. It requires that they be published online by GPO in bulk, in open formats, and in a timely fashion, so that people can easily learn about the work of the federal government. The legislation would not require any additional appropriation, and would bring much needed transparency and coordination. It has already passed the Committee on Oversight and Government reform, was introduced in the Senate, and is awaiting action by the House.

Avoiding decreasing funding levels for the House of Representatives and certain legislative support agencies below the subcommittee proposal. Funding for the House has already diminished by at least 10% over the last two years. This raises the concern that congressional staff may become more susceptible to influence from lobbyists, and that support entities (like GPO, the Clerk, and the Library of Congress) that have transparency roles will be less able to fulfill their missions.

Publishing the House Expenditure Reports in a data-friendly format such as CSV. The quarterly reports contain all spending by the House of Representatives, and are currently published online as a PDF. Starting in 2009, then Speaker-Pelosi began publishing House Expenditure Reports online, which was a significant step forward in making them available, as they had only been published in giant books. Unfortunately, publishing columns of data in a PDF does not allow for the data to be analyzed. Simply put, we're only halfway to House spending transparency. The Sunlight Foundation goes through significant effort to scrape the data from the PDFs and put them into spreadsheets, but this should really be done by the House. It would increase accuracy and timeliness -- and so long as the House releases the information, it should do so in the most useful way possible.