Connecting freedom of information to open data: How to build a better

(Image credit: Screenshot of

In January 2017, the next White House will have an extraordinary opportunity to close the gap between open government data and the Freedom of Information Act (FOIA) when it deploys a new portal, as mandated by the reforms that became law this summer.

After Sunlight shared our vision for open government at the White House Open Data Summit in September, officials at the Office of Management and Budget (OMB) delegated with implementing FOIA reforms contacted us for more information about how that new portal should work. Here’s what we told them: Over the next year, the White House should move aggressively to ensure that all FOIA responses be published as open data on — not in separate databases — under a “release to one, release to all” policy.

Over the past year, we’ve explored different ways that FOIA could work better without more legislative action in Congress. Here’s a specific breakdown how to use the reformed law to fix FOIA, from how the remade statute can be leveraged to drive positive change to an affirmative vision for a new

How the FOIA Improvement Act enables change

The White House Office of Management and Budget has been entrusted with building a new by the FOIA Improvement Act of 2016 that will enable the public to submit a FOIA request to any agency.

(m)(1) The Director of the Office of Management and Budget, in consultation with the Attorney General shall ensure the operation of a consolidated online request portal that allows a member of the public to submit a request for records under subsection (a) to any agency from a single website. The portal may include any additional tools the Director of the Office of Management and Budget finds will improve the implementation of this section.

As the text of the law notes, the reform “shall not construed to alter the power of any other agency to create or maintain an independent online portal for the submission of a request for records.” Agencies can still continue to operate the FOIA systems they have or buy new ones.

There is an opportunity in 2017 to build a technical solution to requests, but also to rethink how the federal government handles public access and disclosure to taxpayer-funded information. FOIA and open data should not be separate issues and outcomes. FOIA demand should drive releases, and releases should be open data.

How to build a better

Instead of building out FOIAOnline, which currently enables the public to submit requests to 14 different agencies, the next White House should consult the specialists at 18F who began to build a centralized request portal for the Department of Justice, then begin an iterative, agile development process in the open to create FOIA software that serves multiple stakeholders. (Look at the open beta of to see how this approach is already benefiting the public.)

The new should develop an open data standard for a FOIA request that can be adopted and implemented by the developers of FOIA software around the country. And Congress agrees. The recent FOIA reforms — now law — direct OMB to “establish standards for interoperability between the portal required … and other request processing software used by agencies subject to this section.”

By adopting such an open standard, can act as a platform for inside and outside of government to benefit. To whit:

  • A new should enable the public to learn about the law and how it works, submit and track FOIA requests, see the performance of agencies over time, and search for existing records. Each FOIA request should be assigned a case number, which will then be used to track performance metrics, including time to respond, time to process, involvement of Office of Government Information Services (OGIS), a FOIA resource, response time to respond to OGIS, partial/full response, and related use of exemptions.
  • A new should enable the Office of Government Services at the National Archives to be able to monitor agency and FOIA officer performance and to better arbitrate disputes using open data.
  • A new should enable members of Congress to easily follow up on their constituents’ requests and see at a glance how well agencies are performing. It should enable the Congressional Research Service, U.S. Government Accountability Office and agency inspectors general to do the same, improving the internal checks and balances we know is crucial to accountability.
  • A new should enable agencies to easily track and transfer cases, improving performance. It should enable the Office of Information Policy at the Department of Justice to do more than collect and publish self-reported performance metrics at the end of each year, but monitor and analyze trends to better target training and guidance.
  • A new should provide journalists with better data to hold agencies accountable for FOIA performance and backlogged cases, analyzing patterns of exemptions or delays.
  • It’s even possible that a new could act as a platform, based upon an open application programming interface, that enables vendors of FOIA software to compete to provide the best user experience and user interface for people to submit requests.

Better IT investment, better outcomes

We know that people inside and outside of government will look at this set of requirements and raise the cost of building such a project. In response, we ask that Congress, the White House and the public take a step back and consider what return on investment we are currently getting from spending on FOIA. As we’ve noted before, the U.S. government spent $461 million dollars on FOIA in the 2014 fiscal year, while the backlog grew 67 percent. The U.S. government spent $448,961,678 processing requests in 2015, while the backlog fell. In that context, would allocating $5 million dollars to building a state of the art platform for creating and sending a request for information to any federal agency be unworthy?

We anticipate that proactive, periodic disclosure of structured open data that’s subject to frequent FOIA requests will, in fact, reduce costs and the backlog caseload at many agencies, freeing up FOIA processors to work on the more complicated public interest requests submitted by journalists that deliver the accountability the public deserves.

Instead of a Janus-like approach where open data is published on portals — which some investigative journalists regard as “propaganda,” and FOIA responses often arrive in envelopes, burned onto CDs and DVDs, or, for a minority of federal agencies, are published in oft-neglected FOIA e-reading rooms — let’s do something better. We all know the status quo isn’t good enough.

Let’s fundamentally shift a broken public access and information disclosure system to something worth of American democracy in the 21st century.