The Senate Homeland Security and Governmental Affairs Committee voted out S. 3335, the Earmark Transparency Act today, but not without objections. The bill would create a single searchable, sortable database of earmark requests, providing one-stop shopping for valuable, detailed information about congressional earmarks.

An effort to move the bill forward last month was delayed when Sen. Levin raised concerns about the cost and technical feasibility of the bill. We understand the need for caution before mandating the Secretary of the Senate and Clerk of the House undertake a new responsibility, but the smart folks over in Sunlight Labs, who create these kinds of databases for a living, assure me that most of the Senators’ concerns can be addressed.

At today’s markup, Senator Levin suggested that it would be difficult to sort data by the various data elements required by the Earmark Transparency Act. Projects we’ve worked on, such as Subsidyscope and Elena’s Inbox, belie any claim that such a requirement is too technologically cumbersome for the Secretary of the Senate and Clerk of the House. Technological “hurdles” should not be used as an excuse to water down the bill. We would be disappointed if, as has been suggested, the earmark database were limited to the five data-points listed in Senate Rule XLIV. There is a great deal more information about earmarks that should be made publicly available and easily accessible.

On the other hand, Senator Levin raised some points that deserve a second look. He is concerned that the bill does not adequately protect proprietary information and that it requires information—such as cost estimates and percentage of federal funds used for a project—that Members simply don’t have access to at the time they make an earmark request.

These seem like issues that could and should be easily remedied—and we would urge all parties resolve them before this bill comes to the floor of the Senate. Time is running out this Congress, and unfortunately, the Earmark Transparency Act probably won’t get a week of floor time to hammer out differences through debate and amendments. (Of course, after negotiation, we would want to see the revised bill online for 72 hours prior to floor consideration.)

A single database of earmark requests is an idea whose time has come. The technology is there and the public deserves to know the who, what, when, where, why and how much behind the earmark requests that are being made by their elected officials. We urge the leaders on this issue to continue working to resolve their differences so that a strong earmark database bill can be voted on this Congress.

Update 2:30 pm by John Wonderlich —

Here’s the text of the letter we sent Monday to the same effect, explaining details on the technical feasibility of the bill:

7/26/2010

The Honorable Senator Carl Levin 269 Russell Senate Office Building Washington, D.C. 20510 cc: Dr. Tom Coburn

Dear Senator Levin:

The Sunlight Foundation is advocating for passage of S. 3335, the Earmark Transparency Act. By centralizing all earmark requests in a single database, the bill vastly improves the way in which information about earmarks is disclosed.

We understand that you strongly support the goal of making earmark information more transparent and accessible to the public, but are concerned about the feasibility of this particular bill. The Sunlight Foundation has a great deal of expertise creating databases that enable the public to sort and search diffuse and complicated government data. We hope to address many of your concerns based on our experience.

We understand that you are concerned that it may not be technically feasible to satisfythe bill’s requirement to create a website that allows the public to search and aggregateearmarks by any of more than twenty data-points listed in the legislation, especially data-points that include lengthy and non-standardized information such as justification for the project and supplemental documents. In fact, it is possible to aggregate calculations by values derived from fields that include attachments or lengthy text descriptions by sorting by phrases contained in those fields. For example, a query for all the earmarks with the phrase “ethanol”, binned by congressman, is certainly achievable. There are several free and open source search engines for a variety of programming languages that support this kind of free text search and aggregation. Our Subsidyscope and Elena’s Inbox projects required searches to be performed on non-standard text fields in a manner similar to what would be required by the Earmark Transparency Act. For those projects, we had a choice of versatile, free software options to perform the advanced query, filter and aggregation operations we required. The software we chose used a standards-based web interface that allowed it to communicate with any programming language.

The Earmark Transparency Act calls for the website to be put in place within six months of enactment, and we understand you are concerned that may not be sufficient time for the Clerk of the House and the Secretary of the Senate to create the website and the earmark database. Based on our own experience, we believe the six-month time frame should be adequate. At Sunlight, we built a searchable database of subsidy related transactions from the data that drives USASpending.gov in less than two months.The Clerk of the House and the Secretary of the Senate may have somewhat less staff availability and expertise than Sunlight, so a slight cushion in terms of timing may bereasonable. Nevertheless, it is important that public availability of this crucial database not be unnecessarily delayed.

Finally, you have raised questions about the cost to create this website. Sunlight believes the costs would be relatively small, encompassing the costs staff needed to develop the database and relatively inexpensive web hosting costs. We would strongly encourage dedicated funding to ensure this project has the resources to be successfully launched and maintained.

In addition to addressing some of the concerns you have raised, we would be remiss if we did not take this opportunity to recommend that a unique identifier for Members of Congress be added to the required data elements in the bill. Bioguide ID is an identifier Congress assigns to each Member and has become the de facto identifier for congressional datasets including OpenSecrets, govtrack and our own Congress API. Including bioguide ID would allow for very easy cross-referencing with other data about Congress, furthering the bill’s goals.

The Earmark Transparency Act is important legislation that would create a powerful new resource for American citizens. We hope we have addressed some of your concerns and would be happy to answer any follow up questions you have.

Sincerely,

Ellen S. Miller