In an astonishing move, the Treasury Department is requiring users to agree to “terms of use” before they can access and download TARP Transaction Reports in spreadsheet format. This requirement undermines the rationale for releasing the data, may implicate federal law, and is simply foolish. It also sets a bad precedent.

The TARP Transaction Reports, which contain detailed information on the government’s Wall Street bailout, are available in PDF format going back to November 2008. Only now are they being published online in a much more useful spreadsheet format, known as XLSX, thanks to President Obama’s Open Government Directive. The Report accompanying the Directive lauds the release of these weekly transaction reports as “improving transparency of federal bank supervisory activities as well as the investment activities of financial institutions.”

The “terms of use” that accompany the TARP reports, reproduced in full at the bottom, require users to:

• Affirm that they have read and understood the site’s privacy policy and disclaimers,
• Acknowledge that the terms of use may be modified at any time, which the user agrees to accept,
• Clearly cite Financialstability.gov on all reuses of data accessed or retrieved from Financialstability.gov,
• Clearly state that neither Financialstability.gov nor the U.S. Government vouches for the data or analyses derived from the data after it has been retrieved from the website.

Users who refuse to click “accept” are not permitted to download the data.

Rationale for releasing the data

The rationale for releasing the data, according to the Open Government Directive, is to “break down barriers to transparency, participation, and collaboration between the federal government and the people it is to serve.” The availability of the data thus far — in PDF format only — has facilitated clever efforts, like those undertaken by SubsidyScope*, to investigate and analyze how TARP money is being used.

Making the information available in a spreadsheet, and not just a PDF, would likely save SubsidyScope hours of data entry, and open up the data to many others who don’t have similar resources to transform the data into a usable format. However, the government’s imposition of these terms of use puts a stumbling block in front of those efforts. If the government can modify the terms of use at any time, they can control how the information can be used. Forcing people to extract data from the PDFs, for most people, effectively makes the data unaccessible. This is antithetical to transparency, participation, and collaboration.

Federal law and government data

I am far from an expert on intellectual property law, but there seems to be a fundamental contradiction here that arises from the intersection of the Open Government Directive, copyright law, and the terms of use.

Federal law prohibits the government from copyrighting “any work of the United States Government.” It is the copyright that gives the owner of information the right to control how a work is published, distributed, and adapted. The government’s terms of use acknowledges that no copyright can be claimed.

Nevertheless, the government asserts that it has the right to control how the data is used, with terms changeable at its whim. It does so by creating a contract of adhesion: the user has no opportunity to negotiate over the terms of the contract. Essentially, the government has created an “end user license agreement” more typically used by software companies.

I do not know if this is legally permissible. The data is in the public domain; can the government retain control? This raises serious questions, and goes against the spirit behind both the Open Government Directive and copyright law’s government information exception.

Also, there are questions of enforceability. For example, suppose user Adam downloads the file and posts it to his website. User Bob then copies the file off of Adam’s website and transforms it. Is Bob bound by the government’s restrictions on Adam? Under the current terms, Adam should require Bob to cite Financialstability.gov, but that’s about it. This is silly, and creates unnecessary confusion.

This restriction on data use is foolish

As noted above, tefore the government made available TARP information in spreadsheet format, it did so in PDF. The PDFs are not subject to terms of use restrictions. The spreadsheets and the PDFs contain the same data, just encapsulated in different formats. Clever folks, like those at SubsidyScope, merely enter the data from the PDFs into their own database. This is not a trivial effort: it takes takes much, much longer to make the data usable, but the consequences are the same. Why should the government treat the same information, just made available in different ways, differently?

My best guess is that the government may be authenticating the PDF files, but not the spreadsheets. The authentication would allow users to know the “provenance” of the information, in a similar fashion to how art dealers verify the authenticity of paintings.

If this is the reason behind the restrictions, it is unnecessary. The government could authenticate the spreadsheet data, just like the PDFs, obviating the need for the terms of service. Or, if it chooses not to authenticate the data in spreadsheet format, a simple warning or note to the user would be sufficient. The restrictions on the use of the data go far beyond that necessary for authentication.

Additionally, requiring users to agree to terms of use that they must click through each time impedes the automated gathering of this information. The whole point of putting TARP information online in database format is to make it easier to share, but the user agreements thwart this.

Setting a precedent

In the coming weeks and months, the government will likely make available a tremendous amount of data to the public in formats that encourage the use, analysis, and transformation of the underlying data. This term of use agreement is unwise from a policy perspective, but hundreds of term of use agreements would be a disaster for open government. The administration should set consistent policies that address the questions of authentication, the needs of the agencies to avoid liability for disclosures (if any liability exists), and most importantly ensures the broadest possible public access to the information. And it should do so in consultation with the public.

Terms of Use December 9, 2009

This terms of use agreement (the “Agreement”) governs your use of the data (the “Data”) available through this FinancialStability.gov Web site (the “Site”). You acknowledge that you have read and understood the Site’s Privacy Policy, available at http://www.financialstability.gov/about/privacypolicy.htm, including without limitation the Disclaimer of Endorsement, the Disclaimer of Liability, and Official Seal, Names and Symbols. In addition, you acknowledge that once the Data has been downloaded from the Site, the United States Government (including the Department of the Treasury) cannot vouch for its quality and timeliness, and the United States Government cannot vouch for any analyses conducted with the Data retrieved from the Site.

The Site may modify this Agreement from time to time, and your continued use of the Data and/or the Site constitutes your acceptance of any and all modifications.

No copyright may be claimed for any work on this Site that was created or maintained by any Federal employee in the course of their duties. Images and text appearing on the Site may be freely copied, however, with respect to the Data you agree:

1. To cite the date that Data was accessed or retrieved from FinancialStability.gov; and

2. To clearly state that “FinancialStability.gov and the United States Government (including the Department of the Treasury) cannot vouch for the data or analyses derived from this data after the data has been retrieved from FinancialStability.gov”.

This Agreement and the Data available through this Site is not intended to, and does not, create any right or benefit, substantive or procedural, enforceable at law or in equity, by a party against the United States Government, its Departments, Agencies, or other entities, its officers, employees, or agents. Nothing in this Agreement alters, or impedes the ability to carry out, the authorities of the United States Government, its Departments, Agencies, or other entities, its officers, employees, or agents to perform their responsibilities under law and consistent with applicable legal authorities, appropriations, and presidential guidance, nor does this Agreement limit the protection afforded any information by other provisions of law.

By clicking on the “I Accept” button below, I acknowledge that I have read, understand, and agree to the above conditions.

* SubsidyScope is an initiative of The Pew Charitable Trusts’ Economic Policy Group, in conjunction with the Sunlight Foundation, its research and technology partner.