Today we’re excited to launch the new U.S. City Open Data Census on a new and improved platform, and with an open invitation to new cities and new organizations to submit information about what datasets cities make public across the country.

Our technical partner on this project, Open Knowledge International, made several changes to the platform that supports the U.S. City Open Data Census (and dozens of similar projects around the world). We took that opportunity to also revisit the datasets included in the Census’ assessment. Here’s a quick synopsis of each of those changes.

Dataset changes

As part of this process we revisited what datasets are included in the Census’ assessment and made seven changes. Specifically, four new datasets are now included in the Census:

• Emergency Calls: Crime reports are already a dataset in the Census; this is a dataset that is a listing of calls or dispatches for police, fire, and emergency medical services.
• Employee Salaries: This dataset shows the annual salary (including overtime) for all employees of the city. Making this information public is a way for taxpayers to know what services they are paying for and hold city staff accountable for the work the are paid to do. We’ve also found publication of employee salary data to be a likely barometer for a city’s overall commitment to transparency.
• Police Use-of-Force: Policing data has made enormous strides since we last updated the Census, in part as a response to the Black Lives Matter movement. Publishing use-of-force data is a real step police departments can take to understand and address bias in policing.
• Traffic Crashes: Across the country, the Vision Zero movement is putting a spotlight on street safety for all users. Data on where and when crashes are occurring is crucial for both measuring street safety and figuring out where interventions are needed.

That means the full list of datasets included on the Census is: Budget; Business Listings; Code Violations; Construction Permits; Crime Reports; Emergency Calls; Employee Salaries; Lobbyist Activity; Parcels; Police Use-of-Force; Procurement Contracts; Property Assessment; Property Transfers; Public Facilities; Restaurant Inspections; Service Requests; Spending; Traffic Crashes; Website Analytics; and Zoning.

The new datasets represent some of the most important information public agencies are measuring today. We are excited to include them in the Census and, by extension, on the list of datasets that forward-thinking city halls should be considering. Read more about what is included in each of these datasets in our datasets explainer.

We are also removing three datasets from the Census’ assessment, solely for jurisdictional reasons. Asset Disclosure and Campaign Finance Contributions were previously included in the City Open Data Census, but these datasets are more often maintained by state governments and not available at the local level. Similarly, Transit Information was formerly included in the Census, but this information is rarely held by the municipal government. Not every municipality has transit, and even if they do, it is often a regional agency responsible for this data. We have removed all three of these datasets.

We want to take a moment here to state that these datasets are extremely important to government transparency, and our decision to remove them from the City Open Data Census is in no way a discredit to their relevance or gravity. Rather, we recognize that city agencies often do not have control over these datasets, and their inclusion in the Census was not a very effective way to incentivize their release. We would be very interested to talk with other organizations about how to track and evaluate states’ publication of municipal ethics data, as well as advocacy organizations interested in hosting a census dedicated to transit agency information.

Changes to the submission process

The new Census makes several changes to how users submit information about open datasets.

• More detailed questions: The submission form now asks more detailed questions about the availability of datasets, and gives more opportunities to make notes.
• Required dataset attributes: Before, the dataset descriptions would say what the dataset was supposed to include, and submitters either would or wouldn’t take that to heart. Now, when making a submission, you are specifically asked about a list of attributes that that specific dataset must include, and have to check a checkbox for each. This should help make sure that datasets really do have all the information that they’re supposed to.
• Required update frequency: The old Census asked, “Is the data provided on a timely and up to date basis?” But there was no guidance on how often counts as timely for a specific dataset. The new platform allows us to specify a specific required update frequency. For example, we have specified that crime-report data needs to be updated daily. The Census now asks, “Data should be updated every day: Is the data up-to-date?” (In most cases, “repetitive” open data should be updated automatically rather than needing manual uploads each time.) This new feature helps us be more strict about what counts as timely, and make sure that cities get recognition for regular updates.
• No more “Unsure” option: The Census asks a number of yes-or-no questions for each dataset. The old platform also had an “Unsure” answer. The new one does not.

One of the most common complaints we’ve heard from volunteers over the years is that submissions often take too long to be reviewed and approved. We saw this as an opportunity to improve how the site works for our volunteers, and with this in mind we’ve switched to an “accept-by-default” submission model, meaning new submissions will appear on the site right away.

While this change does create the potential for inaccurate submissions appearing on the site, Sunlight staff plan to monitor this risk. Additionally, any submission can now be reviewed and replaced by anyone — the Wikipedia approach — so if you notice an inaccurate submission in any city, go ahead and replace it. By making this process more open, we hope to make submissions more streamlined and incentivize more submissions from more people.

Our hope is that all these changes make the U.S. City Open Data Census more accurate and robust. If you have questions about the changes detailed here were are always open to questions and feedback: email us any time at opencities@sunlightfoundation.com.

If your city makes data open and available online, add it to the new U.S. City Open Data Census! And if your city doesn’t make its data open, now is a great time to ask your elected leaders to get on the map – and the Census.