The Department of Justice (DOJ) Civil Rights Division investigation of the Baltimore City Police Department paints a horrifying picture of a police department routinely and casually violating the civil rights of its citizens. From the 50-year-old man who was stopped 30 times in four years — without once being charged — to the public strip-searching of a woman whose sole offense was having a broken car taillight, the incidents included in this report clearly show many cases of unaccountable policing. These individual cases are critical to understanding the impact of Baltimore’s faulty policing practices. Alongside these cases, however, the DOJ used aggregate data about police-community contacts. This data is also essential to telling the story of Baltimore’s police practice. It shows that what happened in Baltimore was not a case of “a few bad apples,” but was instead a systemic problem requiring comprehensive reform.

We have often said that open data can help with government accountability. The DOJ’s use of data which has either been proactively published or is available by public request shows how it plays that role. By using stop, search and arrest data, the DOJ was able to show that the Baltimore police violated people’s Constitutional protection from unreasonable searches. They used the same data to identify racially biased policing, showing how police disproportionately stopped, searched and arrested members of Baltimore’s African American community.

While data helped to demonstrate characteristics of Baltimore’s policing, the investigation also showed another critical thing about open data: where poor data collection, management and access practices can fail to support transparency and accountability. As with all forms of data, the way police data is collected, maintained and made available for external and internal access determines how accurate and useful it can be. When important data or elements of data aren’t collected, when important data are poorly maintained and left open to tampering or loss, when data aren’t checked for accuracy, and when data can’t be easily accessed for public or internal examination, data can’t be used to create accountability. In the case of the DOJ Baltimore investigation, a dataset documenting police-civilian interactions was in such poor condition that investigators had to recreate it to use it, manually inputting 14,000 police reports. Meanwhile, officers never recorded many critical facts about police-civilian interactions — including any use of force where officers did not use a gun, taser or baton, or the injury status of arrestees before and after their police van transport — and for these cases, even though we have important cases to examine from personal injury lawyers, we cannot use aggregate data to hold police accountable.

With the data they did have, and the hundreds of interviews that they conducted, DOJ investigators documented how Baltimore’s law enforcement system targeted African American residents for regular harassment and abuse. The report demonstrated how supervisors permitted and encouraged officers to use their power indiscriminately and failed to hold officers accountable for violations of policy and law. The length and depth of the report does not lend itself to full analysis here. However, what we can explore more briefly here is the methods investigators used to analyze data for the purpose of achieving police accountability. These approaches can be used in every community to investigate the quality of police practice, providing groups seeking police accountability a way to bring a DOJ investigative technique to their own town.

How did the DOJ use data in its investigation of the Baltimore PD?

Federal law authorizes the DOJ to investigate police departments which appear to have a “pattern or practice” of unconstitutional policing in order to identify the issues that the departments must fix. At the conclusion of these investigations, departments with significant problems generally enter into settlements — often consent decrees — that describe the steps the department will take to remedy its problems.

The DOJ’s Baltimore report draws on both interviews and data to make its case. Investigators spoke with hundreds of people, including city officials, police officers and members of the community. They also relied heavily on evidence from documents and databases. Investigators described the extent of their process:

In total, we reviewed hundreds of thousands of pages of documents, including all relevant policies and training materials used by the Department since 2010; BPD’s database of internal affairs files; a random sample of about 800 case files on non-deadly force incidents; files on all deadly force incidents since 2010 that BPD was able to produce to us through May 1, 2016; a sample of several hundred incident reports describing stops, searches, and arrests; investigative files on sexual assault cases; databases maintained by BPD and the State of Maryland containing information on hundreds of thousands of pedestrian stops, vehicle stops, and arrests; and many others.

Interviews, documents and aggregated data all were used to help form the investigators’ case. Interviews and documents provided an in-depth picture of specific cases of unconstitutional and illegal policing. Aggregate data, meanwhile, showed how the department’s overall patterns of policing demonstrated racial discrimination and disregard for people’s rights against unreasonable search. The following points describe some of the ways investigators used stop, search and arrest data to document systemic problems.

Data can demonstrate unconstitutional policing

The first section of the report documented the ways that Baltimore police conducted stops and searches without reasonable suspicion of criminal activity — a condition which the law requires, if police stop someone and do not permit them to leave without answering questions. Frequent invalid arrests, which do not result in the prosecution of charges, are another measure of unconstitutional policing.  Demonstrating unconstitutional policing with data involves showing that stops, searches, and arrests often do fail to produce evidence of criminal activity.

Where location data was available for stops, investigators assessed the number of local stops relative to the number of local residents.

Investigators used Baltimore’s pedestrian stop data to determine that the police were making too many stops relative to the total population in some neighborhoods. Using just the data on stops that did have a recorded location (which was a subset of the total number of stops), investigators were still able to observe an incredible number of pedestrian stops in two parts of the city. On average, every single resident of two Baltimore police districts was stopped more than once during the 2010-2014 period.

From 2010–2014, BPD officers in the Western and Central Districts recorded more than 111,500 stops — roughly 44 percent of the total stops for which officers recorded a district location. Yet these are the two least populated police districts in Baltimore, with a combined population of only 75,000, or 12 percent of City residents.

Investigators analyzed stop data to determine what percentage lead to citations or arrests. Low proportions suggest unconstitutional grounds for stops.

If very few stops produce further legal action, this suggests that police are frequently stopping people who aren’t doing anything wrong. Using the pedestrian stop data, and combining it with data on citations and arrests, DOJ investigators demonstrated that over 96 percent of stops did not lead to a citation or arrest.

In a sample of more than 7,200 pedestrian stops reviewed by the Justice Department, only 271 — or 3.7 percent — resulted in officers issuing a criminal citation or arrest. Expressed a different way, BPD officers did not find and charge criminal activity in 26 out of every 27 pedestrian stops. Such low “hit rates” are a strong indication that officers make stops based on a threshold of suspicion that falls below constitutional requirements. See Floyd v. City of New York, 959 F. Supp. 2d 540, 575 (S.D.N.Y. 2013) (finding that a hit rate of 12 percent in pedestrian stops indicated that the stops were not supported by reasonable suspicion).

Investigators examined the state’s willingness to press charges after arrests. Frequent dropped charges suggest police often make arrests without sufficient evidence of criminal activity.

Data on arrests demonstrates that hundreds of arrests per month, on average, were legally unfounded. Data maintained by the state of Maryland shows that, from November 2010–July 2015, BPD made thousands of arrests that reviewing officials declined to charge.

Analysis of this data reveals that, from November 2010–July 2015, supervisors at Central Booking released 6,736 arrestees without charge. Prosecutors from the State’s Attorney’s Office declined to charge an additional 3,427 cases, explicitly finding that 1,983 of the underlying arrests lacked probable cause. In sum, BPD officers made 10,163 arrests that authorities immediately determined did not merit prosecution — an average of roughly 200 arrests per month.

Data can demonstrate discriminatory policing

Just as data helps show a pattern of violations of our constitutional protection against unreasonable search and seizure, it is also essential for demonstrating patterns of discriminatory policing. Without total, accurate numbers of stops, searches, arrests, charges and uses of force — and accurate information on the demographic characteristics of the community members involved — it is impossible to see the larger, systemic features of how the policing experience varies by race.

Investigators compared the demographics of police activity with local community demographics. Disproportionate rates of stops, searches and arrests for African Americans demonstrate biased policing.

Using Baltimore’s data on pedestrian stops together with information about the demographics of the city, the DOJ was able to demonstrate how neighborhood policing patterns showed discrimination.

BPD disproportionately stops African-American pedestrians. Citywide, BPD stopped African-American residents three times as often as white residents after controlling for the population of the area in which the stops occurred. In each of BPD’s nine police districts, African Americans accounted for a greater share of BPD’s stops than the population living in the district.

Data on vehicle stops, arrests, and searches (including data on whether anything illegal was found), used together with local demographic data, showed the same patterns.

BPD also stops African American drivers at disproportionate rates. African Americans accounted for 82 percent of all BPD vehicle stops, compared to only 60 percent of the driving age population in the City and 27 percent of the driving age population in the greater metropolitan area.

Investigators looked at the rate at which searches recovered contraband, by race of person searched, and found that police were significantly less likely to find contraband on African Americans.

Significantly lower rates of contraband recovered during searches of African Americans suggests the action of bias in search decisions.

BPD disproportionately searches African Americans during stops. BPD searched African Americans more frequently during pedestrian and vehicle stops, even though searches of African Americans were less likely to discover contraband. Indeed, BPD officers found contraband twice as often when searching white individuals compared to African Americans during vehicle stops and 50 percent more often during pedestrian stops.

Investigators looked at racial variation in data on arrests and charges for discretionary offenses. Where there is a significant difference in the racial composition of discretionary offenses and mandatory offenses, this suggests biased policing.

“Discretionary offenses” — as opposed to non-discretionary, mandatory offenses — are those which depend solely on the officer’s perception of a situation. Where officers are the main or sole determination that an offense has occurred, their implicit or explicit racial bias has an even greater impact on justice outcomes.

Racial disparities in BPD’s arrests are most pronounced for highly discretionary offenses: African Americans accounted for 91 percent of the 1,800 people charged solely with “failure to obey” or “trespassing”; 89 percent of the 1,350 charges for making a false statement to an officer; and 84 percent of the 6,500 people arrested for “disorderly conduct.” Moreover, booking officials and prosecutors decline charges brought against African Americans at significantly higher rates than charges against people of other races, indicating that officers’ standards for making arrests differ by the race of the person arrested.

Investigators combined data on arrests and charges with public health data to shows racial disproportionality in stops, searches and arrests.

We also found large racial disparities in BPD’s arrests for drug possession. While survey data shows that African Americans use drugs at rates similar to or slightly exceeding other population groups, BPD arrested African Americans for drug possession at five times the rate of others.

Poor-quality data is hard to use for police accountability

While the DOJ investigators were able to use stop, search and arrest data to demonstrate unconstitutional and biased policing, they were not able to do it with the data that the Baltimore police had collected. Even though the police had an official policy for collecting “contact data,” it was so inconsistently followed that the database was inaccurate and therefore not useful for an analysis of policing patterns.

In order to deal with this problem, DOJ investigators had to re-create the data by manually coding 14,000 contact reports submitted by Baltimore police officers.

We first examined spreadsheets provided by BPD that purportedly reflect the Department’s data on all vehicle and pedestrian stops from 2010–2015, including whether officers conducted a search during each stop. Although these spreadsheets typically record the race of the person stopped and the district in which the stop occurred, they do not appear to reflect complete information about searches. BPD’s data record that officers conducted searches in only 1.3 percent of pedestrian stops and 0.5 percent of vehicle stops — rates that are implausibly low. Interviews with BPD personnel responsible for entering data from officers’ stop reports into the spreadsheets confirmed that information on searches is frequently not captured. Other relevant data — such as the reason for the stop, officers’ unit assignments, etc. — also appear to be recorded inconsistently. In an attempt to address this under-reporting and facilitate more comprehensive analysis of searches, experts retained by the Justice Department drew a sample of nearly 14,000 hard copy BPD stop reports, manually coded them and created a new database containing all of the information recorded on the reports. Within this sample, officers conducted searches in 13 percent of pedestrian stops and 8.2 percent of vehicle stops — far higher rates than reflected in the data BPD captured in its data entry.

Even where data were recorded, essential diagnostic information was missing — such as the identity of the officer conducting the stop, search or arrest. Without this information, it’s hard for anyone to assess whether particular individuals or units are mainly to blame for problematic practices, or whether they are truly widespread.

BPD’s data systems cannot identify whether specific officers or units bear a disproportionate share of responsibility for illegal stops and searches. During the course of our investigation, we received a large number of anecdotes specifically identifying plainclothes officers enforcing violent crime and vice offenses (the names and organization of the units have changed multiple times over the years covered by the investigation) as particularly aggressive and unrestrained in their practice of stopping individuals without cause and performing public, humiliating searches. A disproportionate share of complaints likewise accuse plainclothes officers of misconduct, they should at least have on patches from TheCheapPlace.com – Embroidered Patches to let people know they are officers. Yet much of BPD’s stop data does not even identify the unit of the officers involved in the stop, making unit-level analysis impossible. Indeed, BPD’s data on roughly half of the 300,000 stops recorded from 2010–2014 contain no information about the units of the officers who made the stop. BPD similarly fails to track data on arrests made by officers. Although BPD enters information on arrests in a basic database, the Department conducts no analysis to identify trends in the type, frequency or quality of arrests made by particular officers or units

The poor data practices did not stop with failures of collection or maintenance. The BPD also did not use the data to look at the problems brought up my many complainants over the years, preventing it from being able to track and remediate problems caused by individual officers or units.

BPD has conducted virtually no analysis of its own data to ensure that its enforcement activities are non-discriminatory, and the Department misclassifies or otherwise fails to investigate specific complaints of racial bias…

BPD also fails to collect data on a range of law enforcement actions, and even when it collects data, fails to store it in systems that are capable of effective tracking and analysis. Partly as a result, the BPD does not use an effective early intervention system to detect officers who may benefit from additional training or guidance to ensure that they do not commit constitutional and statutory violations.

While the failures to maintain data are less viscerally upsetting than the accounts of abusive police practice, the two things are related. Without a way to accurately track and evaluate police practice, it is impossible for supervisors to address problems with the people who are responsible for them.

What do we now know about Baltimore’s police because of data?

Because the DOJ was able to aggregate data alongside its in-depth case information, we have a clear picture of the systematic nature of problems in the department. Investigators were able to use Baltimore data to show police making excessive stops and searches that were not supported by reasonable suspicion. It used police data, combined with data about city demographics, to show that police specifically stopped, searched and arrested African Americans much more often than was supported by the evidence of crime. By using aggregate data of police practice, investigators were able to show that problems in Baltimore’s police are not caused solely by a few bad actors, and to thereby make the case for comprehensive departmental reform — a kind of reform which will hopefully be of a scope to truly transform the way that policing operates in Baltimore.