Joshua Brustein of Bloomberg just published a fascinating article on CivicScape, the new entry into the predictive policing market. CivicScape offers open source data-driven predictive policing services for police departments to improve the efficiency of their patrols and crime reduction strategies. CivicScape joins a growing list of predictive policing services (PredPol, HunchLab, IBM, Motorola, Hitachi, Risk Terrain Modeling) seeking to merge predictive analytics with policing tactics.
The newsworthy difference is that CivicScape decided to embrace uber-transparency, releasing its code, strategy, and even concerns with the whole predictive policing data project for everyone to see. In contrast to the usually proprietary or secret world of policing technology, CivicScape opened itself to honest critique by offering its generic code on GitHub.
As someone who has spent the last few years highlighting predictive policing’s problem of transparency and racial bias, and even wrote a book about it (“The Rise of Big Data Policing: Surveillance, Race, and the Future of Law Enforcement”), this move is welcome news. It is not a complete solution, and as detailed in the Bloomberg article CivicScape has some questions yet to answer, but, it is a start.
This post, however, addresses another aspect of the move toward uber-transparency: how it minimizes future litigation risk. In this blog series on what VCs and other technology investors should be thinking about when it comes to new policing technology, I wanted to focus on why releasing one’s code is a potentially smart business move.
In general, what police technology companies largely fail to account for is how their data, proprietary algorithms, and risk assumptions will be dissected by litigants seeking to delegitimize the technology. Trust me, as a former defense lawyer, if your predictive policing algorithm resulted in the police stopping my client, I am duty-bound to expose your technology as flawed. This might mean demanding access to the underlying proprietary algorithm, hiring experts to point out why assumptions in the data are wrong, and generally confusing the situation such that the judge cannot rely on the technology. But, my best argument is to suggest that the secret nature of the technology is itself a reason to reject it.
For companies like CivicScape, the decision to embrace transparency allows them to escape the traditional limitations of other more secretive technologies. A focus on transparency provides legal, symbolic, and practical benefits, all of which minimize litigation risk.
As an initial matter, the embrace of transparency provides a legal response to claims that a technology should be distrusted because it is secret. In criminal court, secrecy is frowned upon as it cuts against due process and fairness principles. In addition, the inability to determine whether the technology is accurate makes it much harder for a judge to bless the use of the technology in court. If, for example, a predictive high crime algorithm is used to justify a Fourth Amendment stop, the answer to the judge that the “technology is secret” is not going to go over very well. But, if instead one can point to an openly available technology, which is testable and understandable (with some technical expertise), the technology might overcome this litigation challenge.
Symbolically, CivicScape’s release of its code signals a confidence that it recognizes that the technology will be contested, and it is ready for this challenge. Because the criminal justice system is adversarial, this is an important, although rare admission. It takes some forward-looking confidence to look to the day when your new technology will be bought, used, and then litigated. For start-ups, so much of the early stage effort is just trying to make something out of nothing, raising money, and just surviving. Litigating in court at some point far in the future seems like a distant luxury. But, by seeing how challenges will be raised about the technology, addressing them openly, and trying to forestall criticism, companies like CivicScape stake out a strong claim to minimizing a litigation risk that some companies do not even see.
Finally, as a practical matter, companies that embrace transparency provide adopting jurisdictions a stronger defense from legal attack. While not always recognized, the technology sold to police must be defended by the government (not the company) in court. It is prosecutors and police department lawyers that end up arguing in court after the technology is challenged. But, the reality is that these lawyers are not technologists and have just as little knowledge as anyone else in the courtroom. By releasing the data and model, however, companies like CivicScape kind of reverse the burden of proving the accuracy or reliability of the data. While as a technical, legal matter, prosecutors retain that burden, they can also simply point out that despite having open access to the model, the defense has not adequately challenged it. A government lawyer can point to the open nature of the information and show that this information is available and testable.
The risk, of course, is that police many decide not to release the data in the future. The Bloomberg article details some of the general police resistance to open data. In addition, there is the risk that by providing the data in an open way it will be challenged. But, if I were choosing between a company confident enough to open their data and one that did not, I might go with the statement of confidence.
The above arguments do not purport to defend CivicScape. I have no idea if the technology works, if the code makes sense, whether they will continue their transparency efforts, or anything else. But, from a business strategy perspective, the open approach minimizes certain litigation risks that could befall other companies.
Of course, CivicScape is not alone in thinking about transparency. Other predictive policing services like Risk Terrain Modeling (RTM) have opened themselves up to scrutiny through academic papers (and a fascinating National Geographic special) and remain non-commerical, Hunchlab uses open source modeling code (and allowed a news reporter from the Marshall Project to embed with an officer using the service), and the industry leader, PredPol, has published a peer-reviewed study. But, the current calculation by CivicScape is to turn a push for transparency and a concern about racial bias into a commercial sales pitch. If it works it will be a statement that forward thinking companies will win out in the policing tech space. If it doesn’t it will be a warning about the value of open access.
Policing Risk is an ongoing project to examine the litigation risk facing new law enforcement technology. As more start-up companies enter the criminal justice space and gain venture funding difficult questions will arise about how innovative products can navigate the reality of a tradition-bound criminal justice system.