In a sweeping investigation, a team of Reuters reporters, editors, and data analysts raised serious questions about Taser’s manufacturer’s claim that no one has died directly from device’s shock. Reviewing the results of hundreds of autopsies, filing hundreds of public records requests around the country, and other painstaking open-source research made the investigation possible.

The team also uncovered a sweeping operation by Axon Enterprise Inc., the new name of Taser’s manufacturer, to make sure that vendor-paid experts were often the first to provide analysis and expertise when Taser-related deaths were investigated.

Hundreds of public records requests around the country, plus other painstaking open-source research, made the investigation possible, as Reuters national affairs correspondent Peter Eisler explained.

“It just kept getting bigger and bigger”

It all started with an idea by National Affairs Editor Jason Szep for a much smaller story looking into the litigation burden of cases where Tasers had been used.

“It began sort of gradually, almost organically. Tim Reid, one of the reporters who did the project, reported the story and then said it was a bigger issue,” said Eisler. “It just kept getting bigger and bigger. We said, wouldn’t it be wild if we could find 50 of these lawsuits? And then it was 75, and then 100, and we just kept going to see how high it could get.”

The original story that sparked the investigation never ran, with the team opting instead to see how far they could take it.

The team decided to go back and build their own database of deaths involving Tasers, focusing on incidents where the Taser seemed to be a possible contributor to the death. Similar databases had been put together before, but with different criteria.

“The last news organizations that did some really good reporting on this were USA Today and a Phoenix paper in the early 2000s. They got a relatively small sampling and went for it,” Eisler said. “That was very very early, there were not a lot of deaths.”

Since then, Amnesty International and Fatal Encounters have also maintained databases tracking this data, but Eisler said that he thought there was an opportunity to take the research further.

“Nobody had ever looked at the litigation burden before,” he said. “We knew that was completely untilled ground. When Amnesty looked at cases they relied almost exclusively on news accounts.” And in some cities, an officer killing a suspect might not even make the news, meaning that there was no media record to cite.

Eisler explained the process:

We combed through Nexis, Factiva, Google and other news sources, using the broadest search parameters we could, and identified any fatal incident in which a Taser may have been used. We also filed requests with police departments for incident reports and investigation records, often with an eye towards confirming or building on information we found in news stories. For example, we might have a news story that mentioned a death, but we wouldn’t have the person’s identity or it would be unclear whether a Taser was involved, so we’d use police records to flesh out the facts and determine whether the case met the criteria we’d set for cases we were including in our database. Once a case was confirmed to meet our criteria, we’d move it to a master spreadsheet and it would be assigned to a reporter for further investigation.

We used a similar process to sort through legal databases, such as Westlaw, Lexis and Pacer. Again, we’d use the broadest possible search parameters to identify any lawsuit involving Tasers (in addition to using “Taser” as a key word, we also used various iterations of the more generic phrases used to describe these weapons, such as “electronic control device”). This process returned thousands of court records, and we went through every one of them to see if it qualified as a Taser-related wrongful death suit (we decided early in the project to focus only on fatalities and opted not to tally injury cases, because the universe simply would have been too large to manage). Reviewing the lawsuits was an enormously time-consuming and painstaking process.

Our search for lawsuits served two functions: not only was it the basis for our assessment of the litigation burden associated with fatal police incidents involving Tasers, it also turned up scores of deaths that got no coverage from local news media. So if we’d used only news reports as a way to track down fatal incidents involving Tasers, we would have missed a lot of them. This was particularly true in more rural areas or in big cities where a fatal police incident, in itself, might not get covered in local news accounts.

That process was incredibly time and labor intensive. Eisler said that the idea for the series first came about around 18 months ago, and the team was working primarily on the project for about a year, for some people almost exclusively.

But the results were eye opening: While Reuters said that its criteria for inclusion in the database was much stricter than Amnesty’s, they still ended up with hundreds of more Taser-linked deaths.

“Their most recent number was 700, and they did great work, but we thought we could bring more resources to bear, and be more comprehensive and more informative,” he said. Ultimately their Taser database catalogued 1,005 deaths.

Building the database

Tackling a project of that scale brought a lot of its own challenges. Ultimately, Reuters database categorized each death against dozens of different criteria, some of which haven’t yet been made public.

“Everything from zip codes for the map, the case number, the docket number, news stories, other information that we garnered as we went along,” said Eisler. “Anytime we saw a reference to ‘excited delirium” and whether it was made by the prosecution or defense.”

That last field, for example, helped tell the story about how “excited delirium,” which is not a universally-recognized medical condition, would ultimately be pinned as the culprit for many Taser-related deaths, thanks in large part to “a thicket of intersecting relationships among police, coroners and a wide network of scientists the company taps.”

Building a comprehensive database occasionally meant double back and adding fields they had not thought of from the start.

“The big project where you’re building a really large data set over time is figuring out what you want to get out of that data that you’re collecting, so you collect all of the data for each case,” said Eisler “We fundamentally understood the process of doing that, but there were things that halfway through we thought, we really should be tracking this. We re-looked at cases we thought we had put to bed.”

In this case, there was not a lot of fancy technology at play.

“We did it all in spreadsheets. We did it all in Excel. A giant Excel spreadsheet. It’s a lot of data, on a thousand plus cases,” he said, crediting Grant Smith and Michael Pell for amazing work helping analyze the trove of information. One key piece of advice for making that go smoothly? “If you have multiple people gathering or augmenting data, having a mechanism built into the spreadsheet or process to identify who did what is very important.”

Being able to check in with people about what a change or addition meant kept everyone on the same page.

Also: look before you leap.

“Before you go down that road, be sure you have a pretty good idea on whether the data you want to get is even attainable,” said Eisler. “A pre-reporting effort can be very helpful in that. Let’s give ourselves a month or two months and scramble the jets and see if we can get what we want to get, or if we want to trim it back or cut it entirely, or expand it.”

But while the amount of work that went into the database was enormous, the payoff is still continuing.

“As we continue to drill down into the data, we continue to find more interesting stories,” said Eisler, adding that the team planned more stories in the coming months, some based on data points that Reuters has not yet publicly released.

For data that has been released, however, Reuters’ visualizations allow readers to sort and filter by a number of criteria, including region, race of the deceased, and whether there were secondary factors, such as a fall afterwards. That means other organizations can build on the work Reuters started, localizing it or digging deeper into particular cases.

“Our hope is that other news organizations will find patterns and stories themselves. We won’t be able to tell every story in every city that has a preponderance of these sorts of deaths,” he said. “We’ve seen some news organizations just go ahead and do it.”

Public records project advice

Eisler also had advice for those embarking on ambitious public records projects.

“We requested the better part of a thousand autopsies alone. Every case we found in a lawsuit and every death we found that resulted in a lawsuit, we end up seeking police records in those cases,” he said. In some states, where a central office handled autopsies, they could get 30 with one request. In other places, they had to go agency-by-agency.

One thing they found helpful was outlining the scope of the project.

“We would make it very clear in our records request that this was something we were asking of a lot of different entities or agencies contemporary to them. We would say this is one of a thousand cases, so they were a needle in a haystack,” Eisler said. “We collected Taser policies from nearly three dozen of the largest police departments in the country. We made it clear we were collecting lots of them and comparing and contrasting them. We certainly found it to be an advantage to let folks know that we asking everyone for the same records.”

But even then, they sometimes had to call it quits with some departments — usually the smaller ones.

“The fact is that we could not litigate scores or perhaps hundreds of cases for one autopsy each, or even more importantly for settlements,” he said. “There were lot of jurisdictions for settlements, and we would say we want to see what they paid out and they would just say no.”

Eisler said that as someone who had been taught to take every public records fight to the end as a matter of principal, it was a frustrating but necessary strategy.

With a scope that broad, it can be tough to decide when it’s really “over,” particularly as some records fights can go on endlessly and as people continue to be killed in Taser-involved incidents. But for the Reuters team, the end came when they felt that they had made their database as comprehensive as it could be.

“It was when we had convinced ourself that we had exhausted the universe of cases that were out there. We felt like we had captured all the cases that we could, perhaps minus one or two,” he said. “There are almost certainly cases that we didn’t find, but we felt like we found all the ones we were going to. When we thought that the data set was really complete.”

Read the full Shock Tactics investigation on Reuters, and sign up for our newsletter to get more FOIA stories and advice.

And please, send over your favorite FOIA stories via email, on Twitter, or on Facebook, and maybe we’ll include them in the next roundup.

Image under public domain through U.S. Marine Corps, by Sgt. Brian Marion