Late last month we applauded the House Appropriations Committee for appearing to improve on its dismal record of webcasting hearings. As we noted, webcasting is an effective way to allow interested parties to access public hearings that are often essentially private because of space constraints.

Unfortunately, that praise was premature. We've been monitoring the committee's progress since then, and were hopeful that we could close Sunshine Week by announcing that it had continued to webcast the vast majority of its hearings. That has not been the case.

During the last week of February, when the committee appeared to be turning a corner, every hearing listed on its schedule was webcast live. That trend ended almost as soon as it started.

In the first two weeks of March the committee only webcast 8 of the 24 hearings that it has held. Even worse, only 4 of its 17 open hearings over the past week have been webcast.

That means that during Sunshine Week the House Committee on Appropriations only managed to webcast 23% of its open proceedings.

This was unacceptable in 2012 when a Sunlight analysis found that the Appropriations Committee was the only House committee that did not webcast the vast majority of its hearings and it continues to be unacceptable today. We will continue to monitor the Appropriations committee until they prove that they have complied with House rules that require them to webcast their hearings "to the maximum extent practicable."

Photo Credit: the test pattern is from Gak on Flickr.

 

At one point on Wednesday morning 163 people were watching a live webcast of the Legislative Branch Appropriations Subcommittee's budget hearing on the Library of Congress.

That's about 160 more than would have been allowed through the doors if they had tried to go in person.

Room HT-2 in the Capitol, where the hearing was held, is barely large enough to hold members of Congress, witnesses, and their respective staffs, let alone more than a few members of the public. Sunlight's Daniel Schuman was one of only 3 members of the public to gain access to the room at last year's hearing.

The Legislative branch subcommittee held two days of hearings this week. They have already been covered by numerous news outlets and, since they concluded, more than 550 people have accessed the archived videos.

Earlier this week we applauded the Appropriations Committee for moving to broadcast more of their hearings live on the web, calling it "a new precedent...in favor of transparency."

We plan to monitor the Appropriations committee to see if they continue to webcast their hearings and provide this easy way for the American public to access their proceedings.

This morning the House is debating passage of the Legislative Branch Appropriations Bill. Rep. Honda, the ranking member of the Legislative Branch Appropriations Subcommittee and a longtime champion for better access to legislative information, spoke on the importance of improved access as part of his broader comments. Here's that excerpt:

I am privileged to represent Silicon Valley, the center of technological innovation in this country. Since I joined this subcommittee, I have tried to push the House and other agencies to explore technological solutions to issues such as transparency, evacuation management, and data storage.

As you probably know, Federal agencies, including our own in the legislative branch, can be slow to change and adopt new technologies. This is mentioned in the report, which includes language on the issue of bulk data downloads of legislative information, something I requested and secured language about in this bill in fiscal year 2009.

This effort is now also being championed by leadership on both sides of the aisle, as it is a way to increase transparency by allowing the public to easily download and analyze government data.

There are some concerns about cost and the ability to authenticate the data that the language in the report tries to address.

I think, however, that these are relatively simple matters to overcome, as data is already being compiled in a format that can easily be distributed and technology support staff has indicated that only a simple procedure is needed to make the bulk data available.

Furthermore, the GPO already employs an authentication standard for its own accessible bulk data through its FDSys website that we could also utilize.

The House Majority recently announced that it will immediately create a task force, as described in this bill, to expedite a report and implementation of public access to bulk legislative data. While I believe the time to implement this is now, I expect to be included in these efforts as Ranking Member of the Subcommittee and longtime advocate.

It may feel like an ordinary Wednesday, but today is a milestone for legislative transparency. The House's leadership has issued a statement adopting the goal of "provid[ing] bulk access to legislative information to the American people without further delay." They have stated that bulk access "ranks among our top priorities in the 112th Congress" and directed a task force "to begin its important work immediately."

The statement was made by many of the key players on this issue in the House: Speaker Boehner, Majority Leader Cantor, Legislative Branch Appropriations Subcommittee Chairman Crenshaw, and Oversight and Government Reform Committee Chairman Issa. It was prompted in part by a measure in the legislative branch appropriations report that, as initially formulated, may have frozen efforts to move in this direction, followed by a partial fix to the report and ultimately a proposed amendment to the bill. (The amendment was apparently rejected by the House Rules Committee.) Of course, all of the letters to Congress and news coverage helped reinforce this as a higher priority.

The debate over whether there should be bulk access to legislative data is over. Because bulk access is a top priority of the 112th Congress, we expect to see tangible progress in the upcoming months. The remaining questions largely concern how bulk access should be implemented to meet the needs of the public while respecting the legitimate concerns of Congress and its support agencies.

While we are disappointed that the task force will not include members of the public, we hope that the public will be consulted. After all, the American people are the intended end-users. Sunlight and our friends in the transparency community stand ready to be of assistance as the technical, policy, and scope issues are addressed.

We have reached this turning point for a number of reasons. Rep. Honda has pushed to make bulk access happen over the last half-decade. Rep. Boehner made legislative transparency a priority when he was elected speaker. Reps. Cantor and Hoyer co-hosted the Congressional Facebook Hackathon, which declared bulk access to be an important goal. Rep. Lungren and the Committee on House Administration held hearings and issued directives establishing the important transparency portal docs.house.gov as well as hosted the Legislative Data and Transparency Conference. And there's many staff and members of Congress who have labored for years to bring this to fruition.

While this is clearly progress, there's still much more to do. We will be monitoring this issue closely.

The media's magnifying glass is concentrating attention on actions by the House Appropriations Committee that could stall progress on the public's access to legislative information. The Sunlight Foundation and our allies continue to push Congress to stop dragging their feet and join the 21st century by allowing developers access to open legislative data to build the tools to keep citizens informed about what their government is doing.

Please find and call your Representative at 202-224-3121 or write to reinforce the American public's hunger to read and follow legislation. Here are some excerpts from recent media coverage on this important transparency issue:

Roll Call reports on Republican House leadership's strong support for bulk access and quotes Rep. Crenshaw misunderstanding the issue of authentication:

“The Speaker pledged to make the 112th Congress the most open and transparent Congress in history and to make legislative data available online and in bulk,” said Michael Steel, spokesman for Speaker John Boehner (R-Ohio). “He continues to look for the best way to do that.”

“Facilitating public access to bulk legislative data ... has been and will continue to be a priority for this committee,” echoed Salley Wood, spokeswoman for the House Administration Committee. But lawmakers’ hands would be tied until a task force could be convened and report back on its findings, according to the House report language.

“We wanted to create a system where we could have this available but also make sure we protect the authenticity and integrity of all this information,” said Rep. Ander Crenshaw (R-Fla.), chairman of the Appropriations Subcommittee on the Legislative Branch.

The Washington Examiner addresses the committee's confusion over how citizens use and should access government information:

Folks with computers -- notably, professional and citizen journalists -- would be able to take information about massive numbers of bills and analyze them in myriad ways -- if Congress would allow such information to be downloaded from THOMAS in bulk.

It won't. And, according to a new draft report from the House Appropriations Committee, it won't be allowing bulk data downloads from THOMAS anytime soon.

Instead of taking a step towards greater transparency, the committee got hung up on whether people would know if the data they're seeing on the Internet were accurate and really from Congress -- "authentication," they call it.

FierceGovernment notes the lack of a deadline for decision making:

The report retains language decried by transparency opponents that would indefinitely postpone public bulk downloads of legislative information in XML. Good government groups, including the Sunlight Foundation, have pressed for the Library of Congress to release the bulk data used to track legislative developments in the library's THOMAS website, arguing that they could do a better job of presenting information.

TechPresident reports on the frustration among transparency advocates:

Open government advocates are up in arms over what appears to be another attempt by government bureaucrats to stall the move to enable bulk data downloads of legislative information online.

Slashdot opens the issue for conversation to their community:

The House Appropriations Committee is considering a draft report that would forbid the Library of Congress to allow bulk downloads of bills pending before Congress. The Library of Congress currently has an online database called THOMAS (for Thomas Jefferson) that allows people to look up bills pending before Congress. The problem is that THOMAS is somewhat clunky and it is difficult to extract data from it. This draft report would forbid the Library of Congress from modernizing THOMAS until a task force reports back. I am pretty sure that the majority of people on Slashdot agree that being able to better understand how the various bills being considered by Congress interact would be good for this country.

Legal Informatics also has a nice collection of blog posts on this issue.

Follow the latest developments here.

Does information about legislation belong to Congress or to the American people? This basic question is at the heart of a fight over how Congress releases data about what it does. Americans increasingly use the Internet to make sense of the world around them, and open data opens up Congress in a way that's never been possible before.

In the pre-YouTube pre-iPhone pre-Amazon days, Congress built a website -- THOMAS -- to let citizens follow legislation from home. THOMAS was revolutionary ... in 1995. But the Internet continued to develop, becoming more sophisticated and interactive, allowing web developers to easily share the data behind their websites with others. It's why we can book flights on Travelocity, check the weather on our phones, and follow legislation on OpenCongress and GovTrack.

Unlike Travelocity and the National Weather Service, Congress doesn't share the data behind THOMAS with anyone. Instead, web developers must reverse-engineer the website to transmute its pages into usable data, like assembling a puzzle from thousands of ragged pieces without a picture on the box as a guide. This slow, difficult, and time-consuming process isn't perfect, but it's responsible for how most Americans follow what's happening in Congress.

The better approach is for Congress to publish the data behind THOMAS. Government regularly does this elsewhere, and "bulk data" is responsible for clever new uses of information developed by citizens, journalists, and even the government itself.

In upcoming days, the House is likely to pass legislative language that pays lip service to releasing THOMAS data while putting the idea in a deep freeze. This would be a disaster. But it's not too late. Tell your representative that you want Congress to publish legislative data now.

PS. For more information and the latest developments, go here. 

This Thursday at 11am, the House Appropriations Committee will mark-up the legislative branch appropriations bill and an accompanying committee report. The report has unfortunate language that would undermine how legislative information is made available to the public based on misunderstandings of technology and policy.

Many people are calling the members of the committee to complain, with a particular focus on Rep. Andew Crenshaw, who chairs the subcommittee, and Reps. Hal Rogers and Norman Dicks, the Chairman and Ranking Member for the full committee. (Rep. Mike Honda , who is the ranking member of the subcommittee, has been supportive of bulk access to legislative data for years, and should be congratulated for his efforts.) Here are some helpful talking points.

• The current draft of the legislative branch committee report needs to be changed. It imposes harmful new barriers on public access to legislation information that will undermine transparency. It does so by stopping bulk access to some legislative information, such as the Congressional Record, and undermining important efforts like the House's transparency portal docs.house.gov.

• The report will also indefinitely delay any efforts to open up new legislative information to bulk access. It does so by creating a "task force" to study the issue, much like the one created four years ago. There's no date by which the task force must report and no member of the public has been invited to serve. This is not progress, it is death by bureaucracy.

• What the committee should do is require bulk access to THOMAS data within 120 days of the appropriation bill's passage. It should also create an advisory committee to guide the evolution of THOMAS.

• The concerns raised in the committee report about the authenticity of data from THOMAS are a red herring. Legislative information is already publicly available; we are only asking for it to be more accessible -- in bulk. This is an uncontroversial, inexpensive, and common practice across the government.

• The THOMAS website was created in a matter of months when the Speaker of the House decided it was a priority. The House's leadership thinks it's a priority, members of the public think it's a priority, and so do many members of Congress. It's time for the Appropriations Committee to make bulk access to THOMAS a priority as well.

by Daniel Schuman and Eric Mill

House Appropriators may deal a tremendous blow to prospects for improving public access to legislative information. In a draft report expected to accompany the Legislative Branch Appropriations Bill for 2013, scheduled for a full committee vote tomorrow, appropriators misunderstand how data can be "authenticated," and kick responsibility for improving public access to legislative data to a non-public task force with no set reporting date. Unless corrected, this draft report represents a tremendous step backward for transparency, and fails to seriously grapple with the history of efforts to free legislative information for widespread public use.

Legislative Information Should Be Widely Disseminated

The purpose of THOMAS is to bring legislative information to as many people as possible; preservation and authentication is best handled through other long-established methods that THOMAS was never intended to address. The lack of authenticity to THOMAS data does not present a problem for most users. Rather, the largest problem with THOMAS is that the data is not provided so that it can be easily copied, placing a significant burden on citizens who wish to make sophisticated use of the information. The THOMAS website directly provides nearly a million users each month with an "inauthentic" version of information about legislative activities, a practice that will continue unabated under the draft committee report. While THOMAS often links to a GPO document that is "authenticated," its display of bill text, legislative summaries, cosponsor data, and other information is not certified as being correct, and often changes because of the Library's errors in how it publishes the data.

To the extent to which THOMAS information should be authentic, the report does not engage with best practices around authenticity of data on the Internet. Verifying the authenticity of data can be performed securely and reliably with the use of metadata external to the data itself. In fact, this is precisely how GPO's FDSys currently authenticates XML documents of the US government, including its legislation, regulations, and laws. GPO accompanies each document it publishes with a "PREMIS" metadata file that includes information needed to cryptographically verify the authenticity of documents. For example, here's the PREMIS file accompanying HR 6289. Worries about authenticity are a red herring.

Bulk Access is a Separate Question from Authenticity 

Bulk access to THOMAS data is a simpler and less controversial step than this draft report contemplates. The underlying information is already publicly available on the THOMAS website, and third parties already are scraping the data from the site to make it available in bulk. It simply makes sense for the Library to meet the needs of the public directly through providing the data in bulk itself. This merely opens up another avenue to access info that's already being released. It would also eliminate any errors created through the scraping process.

The Draft Report Creates a Secret, Never-Ending Process

The draft report would require the establishment of a task force to examine and report back on a number of issues raised in the report regarding bulk access to legislative data. This is seriously flawed in several major ways.

First, bulk access is about granting the public better access to legislative information. It stands to reason that the public should be included in all discussions. However, the proposed task force does not include any non-governmental participants. A number of individuals and organizations are expert in these matters, and should be full participants.

Second, the draft report imposes no deadline for a report from the task force. The last time Appropriators required a task force on a similar matter, four year ago, it never reached any conclusions or reported back. Without a deadline, the same will happen here.

Third, the task force's report should be provided to the public as well as to the committee at the time it is completed. Draft reports should also be made available for public comment.

Fourth, the report language is terribly overbroad: it prohibits the establishment of bulk data downloads of legislative information prior to the reporting back of the task force. Making use of modern technology to provide information in better ways should be something that is encouraged, not prohibited. Information is already being provided to the public in bulk regarding certain legislative activity. Would this report language stop the GPO from providing bulk access to the Congressional Record, as it does now? Would it prohibit the House of Representatives from providing bulk access through its innovative docs.house.gov portal? If so, that would be a disaster for transparency.

Finally, the idea of a task force to assess these questions ignores that these issues were already addressed by the Library of Congress in a 2008 memo.The memo explained that the XML database containing bill metadata was expected to be able to be released in bulk by May 2008. It also stated that "CRS... will continue to identify and analyze ... the following policy matters for the Committee's consideration," including "data accuracy" and "data permanence and authentication." Where are the results of CRS's analysis? What is the strategic plan for THOMAS referenced in the memo? Where is the study promised that would engage in "an examination of permanence and authentication of legislative data, along with any attendant issues, risks and workload?"

Simply put, the draft committee report's establishment of a task force is another recipe for delay. We saw this four years ago, the last time the Library was pressed to make improvements on this issue. The time is long past for action, and the Appropriations Committee will be judged on whether it makes another plan to make a plan, or whether it establishes real deadlines for progress. THOMAS itself was created in a matter of months when the Speaker of the House decided it was a priority. Bulk access to legislative data will also come about when legislators decide that being transparent is more important than establishing a task force to talk about it.

The House Appropriations Committee just announced it will hold a full committee markup this Thursday (May 31) on the Legislative Branch Appropriations Bill for FY 2013. While the Sunlight Foundation is hopeful this legislation will address a number of our transparency priorities, we will be watching in particular to see whether the longstanding issue of improving public access to legislative information will finally be addressed.

At the subcommittee markup, there was a tantalizing hint that the committee report will contain language to require bulk access to THOMAS data, the importance of which the public interest community has addressed many times before. We are hoping this bill will require the Library of Congress to implement bulk access to THOMAS within 120 days of the legislation's passage, and require the Library to create an advisory committee on THOMAS.

While we were able to observe that the subcommittee adopted this draft legislation without amendment, the accompanying report (where language regarding bulk access was inserted) has not been released to the public. Congressional staff have told me the report will not be released to the public until the full committee markup.

Committee members will get a look at the bill and accompanying report no later than Tuesday, which is three calendar days before the hearing (if you count Thursday as a calendar day, otherwise it should be available today, Friday), in accordance with Committee rule 6(j). The public must wait get a look at the bill no later than Wednesday at 11am, which is when the committee chairman must electronically post a copy of the bill according to committee rule 4(d)(4). The committee's rules makes no mention of when committee reports must be released to the public, although such a report will presumably be ready when the draft bill is released to the public.

 

In the last 18 months, the House of Representatives has made significant strides towards greater openness and transparency in congressional deliberations, but significant work remains. The Legislative Branch Appropriations Bill for 2013, which was marked-up by a subcommittee last week, presents a major vehicle for the House leadership to make good on its promise to implement common-sense transparency measures this session.

While there are many issues that can be addressed a number of different ways, Sunlight will be looking at  the full committee markup to see if the bill:

-- Provides bulk access to THOMAS data

-- Fully funds the Office of Congressional Ethics

-- Requires Publication of  CRS Reports online

-- Publishes the Constitution Annotated online as it's updated in XML

-- Reinstates the Office of Technology Assessment

-- Makes reports to Congress available online

-- Publishes House spending information in an appropriate format for the data

Improve Public Access to THOMAS Data

THOMAS was created by Congress to make legislative information freely available to the public, but the Library of Congress has not kept up with best practices. One such practice -- "bulk access" -- would ease the development of new tools and technologies by publishing THOMAS data files online, promoting accurate and timely information dissemination. Congress has expressed its support for bulk data as have many organizations, but the Library continues to stall despite a 2008 memo describing how easy it would be to implement.

At the recent legislative subcommittee hearing, Rep. Honda mentioned that text has been inserted into the committee's report that would in some way address the bulk data question. The last time this happened, the language was watered down sufficiently so that the Library of Congress successfully evaded its obligations over the last half a decade. We hope the bill will contain these two provisions:

(1) Congress directs the Library of Congress to implement bulk access to THOMAS within 120 days of passage

(2) Congress directs the Library of Congress to immediately create an advisory committee on improving public access to legislative information that is composed of people inside and outside of government.

Fully Fund the Office of Congressional Ethics

The Office of Congressional Ethics is the House of Representatives' independent ethics watchdog. It came into existence in March 2008 after a series of corruption scandals prompted congressional leaders to explore creating a transparent, outside enforcement entity. While OCE is not as robust as originally contemplated, it plays a crucial role in ethics oversight. Last year, the office survived a counterproductive effort by nearly 100 members of Congress to significantly reduce its funding. This year's appropriations bill maintains OCE's funding at $1,548,000, which is the same level as last year. We believe that OCE should be strengthened, but at a minimum, its funding should be sustained at least at this level.

Publish CRS Reports Online

Congressional Research Service reports undergird the public's understanding of Congress, but CRS no longer directly releases the reports to the public. As a consequence, while many reports used by citizens, courts, and government employees are on the internet, they are often out-of-date, and a fair number are available only for a fee or not at all. By comparison, sister agencies like CBO and GAO regularly publish reports online. For more than a decade, organizations and members of Congress have urged that CRS reports be publicly available, and CRS concerns have been refuted by a former counsel to the House of Representatives. The reports are already digitized and available on Congress's intranet; it would take a trivial effort to publish them online.

During the markup of the 2012 Appropriations Bill, Rep. Leonard Lance introduced an amendment that would have required the Clerk of the House and the Secretary of the Senate to maintain a website containing CRS Reports and Appropriation products while protecting confidential advice from CRS. Similar legislation has been introduced by Rep. Quigley. We hope that House Appropriators will move to make these reports more readily available to the public.

Release the Constitution Annotated Online

The Constitution Annotated (or CONAN) is a continuously-updated 100-year-old legal treatise that explains the Constitution as it has been interpreted by Supreme Court. Maintained by CRS and printed by GPO, a hard copy is published (and put online) only once a decade, with printed updates every two years. However, CONAN is updated frequently, with those updates available on Congress' internal website. In November 2010 (18 months ago), the Joint Committee on Printing directed that the continuously-updated version of CONAN be made available online as a searchable PDF, but it still is not. Many organizations have asked that the underlying document be published online in its original (XML) format, which is more user friendly than a PDF, and would take minimal effort to release.

This upcoming year, the Constitution Annotated will be up for its once-a-decade print edition. With at least 4,870 statutorily mandated copies, at an estimated cost of $226, the House and Senate will pay over $1.1 million for a document that will go out of date almost immediately. We suggest that some of these costs may be recouped by asking House offices if they wish to receive a print copy, as a continuously updated web version is already made available to all congressional offices. Regardless, we urge that the web version that is already made available to congressional offices also be made available to the American people in its web friendly format. While publishing the document as a PDF would be a small step forward, the best use of taxpayer dollars to maximize usability would be to publish it in XML, the format in which it is prepared.

Other Provisions

Sunlight support additional measures in the Legislative Branch Appropriations bill. Those provisions include:

The reinstatement of the Office of Technology Assessment, as proposed by Rep. Rush Holt last year. OTA provided Congress http://sunlightfoundation.com/blog/taxonomy/term/Office-of-Technology-Assessment/ with the “means for securing competent, unbiased information concerning the physical, biological, economic, social, and political effects” of technology.

Inclusion of the Access to Congressionally Mandated Reports Act, which would would gather together all reports to Congress from federal agencies in one place. It requires that they be published online by GPO in bulk, in open formats, and in a timely fashion, so that people can easily learn about the work of the federal government. The legislation would not require any additional appropriation, and would bring much needed transparency and coordination. It has already passed the Committee on Oversight and Government reform, was introduced in the Senate, and is awaiting action by the House.

Avoiding decreasing funding levels for the House of Representatives and certain legislative support agencies below the subcommittee proposal. Funding for the House has already diminished by at least 10% over the last two years. This raises the concern that congressional staff may become more susceptible to influence from lobbyists, and that support entities (like GPO, the Clerk, and the Library of Congress) that have transparency roles will be less able to fulfill their missions.

Publishing the House Expenditure Reports in a data-friendly format such as CSV. The quarterly reports contain all spending by the House of Representatives, and are currently published online as a PDF. Starting in 2009, then Speaker-Pelosi began publishing House Expenditure Reports online, which was a significant step forward in making them available, as they had only been published in giant books. Unfortunately, publishing columns of data in a PDF does not allow for the data to be analyzed. Simply put, we're only halfway to House spending transparency. The Sunlight Foundation goes through significant effort to scrape the data from the PDFs and put them into spreadsheets, but this should really be done by the House. It would increase accuracy and timeliness -- and so long as the House releases the information, it should do so in the most useful way possible.