Siblings or silos: How do open data and FOIA work together?


Last year, 28 U.S. cities published open data policies bringing the total to nearly 120 cities nationwide. This growth spurt has signaled a sweeping move from reactive responses to public records requests to systematically and proactively publishing open data. For most of the last 50 years, Freedom of Information Laws (FOI) have ensured that residents have access to information on government activities as a fundamental, democratic right. But now that many of these cities are “setting the default to open,” city staff are left guessing about how to balance these two varying but equally crucial channels for public access to information.  

From a research perspective, the relationship between FOI laws and open data policies begs the question: are the public information channels of open data and FOI law competitors or complements? Though both are channels for public transparency, they serve different functions: open data provides access to raw data or broadly applicable public information; public records meet specific, often time-sensitive needs, scoped and defined by individual data users. Understanding the relationship between open data and public records requests could help city staff better allocate internal resources in meeting these varying needs. City staff need to understand exactly when and why residents are still struggling to engage with public information and apply resources to improve the way open data and FOI laws work together.

To that end, I’m working with the Sunlight Open Cities team to conduct new research that aims to answer three key questions about the relationship between open data and FOI:

  1. Does adopting an open data policy affect the volume of public record requests that cities receive?

If open data can be an effective tool to reduce public records requests through proactive disclosure, the potential cost savings are significant – the US Federal Government spent $448,961,678 processing Freedom of Information Act (FOIA) requests in 2018 alone and Yakima, Washington (population 93,986) spends $500,000 annually responding to public records requests.

However, limited evidence exists on this topic and that evidence is mixed: the Yale Law Journal article above notes that since passing the US Open Data Policy, Federal FOIA requests have increased by over 16%, while Reinvent Albany estimates in a 2014 report that uploading data to public reading rooms or the city’s open data portal could reduce New York City FOIA requests by 20%, saving the city $3.5 million annually. Socrata and the City of Chicago found a 50% reduction in FOIA requests since launching its open data portal.

As the first multi-city study of the relationship between open data and FOI, our research aims to cut through the mixed signals and provide rigorous evidence on this question. This will help cities focus limited open government resources where they will have the greatest impact.  

  1. Does the community of records requesters change over time?  

Open data is a public good which has the potential to provide significant value to a wide group of stakeholders – including advocates and journalists, non-profit organizations, local businesses, researchers, government agencies, and concerned citizens.  

At Sunlight Foundation, we have had the privilege to work with a diverse group of open data stakeholders in our partner cities to use data to address the problems that matter most to them – such as identifying ways to better serve people experiencing homelessness, face challenges of flooding and displacement, and support equitable and complete neighborhoods. If our research finds that cities adopting open data policies see a diversification of the individuals submitting public records requests, that would indicate that open data policies are effective tools to create a “big tent” of government data users. If this diversification does not occur, that would raise important questions for cities about whether certain communities are not being reached, or if other tools are effectively serving the needs of these communities.

  1. Does the type of public records requested change following passage of an open data policy?

After a city decides to adopt an open data policy, they must then determine how to prioritize the use of their limited resources for proactive data disclosure. Cities should use previous public records requests as a roadmap to understand what data is demanded by their citizens to inform these prioritization decisions. As cities release data demanded by citizens, it should follow that the type of records requested will shift to information that has yet to be released. If the type of data people are requesting doesn’t change, cities may need to better prioritize highly-requested data for publication as open data or to invest in raising citizen awareness of available data resources.

The findings of this research will directly inform our work and the work of our What Works Cities partners as we support cities across the US to adopt open data policies and effectively engage citizens through open data.

To answer these questions, we have begun collecting public records request data from over 40 different US cities — some which have adopted open data policies and others which have not. We will use statistical methods that control for fact that cities that choose to adopt open data policies may differ from other cities in key ways to isolate the effect of open data policy adoption on public records requests.

If you’re interested in reviewing or advising on the methodology for this project, please review our draft pre-analysis plan and provide feedback on our research design. You can also follow along with this project on our GitHub. Please send your thoughts, ideas, and questions to or comment below.