Judicial approval for predictive coding: canvassing opinion | Practical Law

Judicial approval for predictive coding: canvassing opinion | Practical Law

Greg Wildisen of Epiq Systems considers the views of various parties, including the judiciary, on the use of technology in document review.

Judicial approval for predictive coding: canvassing opinion

Practical Law UK Articles 2-521-5179 (Approx. 4 pages)

Judicial approval for predictive coding: canvassing opinion

by Practical Law
Published on 26 Sep 2012United Kingdom
Greg Wildisen of Epiq Systems considers the views of various parties, including the judiciary, on the use of technology in document review.
In the UK, Practice Direction 31B was introduced into the Civil Procedure Rules in order to encourage parties to manage electronic documents efficiently to minimise the cost incurred, and to use a broad range of evolving technologies to assist in this process. Predictive coding, or computer assisted review, is one such technology.
Put simply, predictive coding is a process by which software is trained by a senior lawyer (the "expert") who is familiar with the issues of a matter. The expert reviews a representative sample of the documents and registers his decision on the relevance or otherwise of each document reviewed on the system. In this way, the expert "trains" the technology to recognise responsive patterns.
Predictive coding has become a source of lively debate in 2012, as a number of cases involving disputes about whether it should be used in particular circumstances have come before judges in the US.

The panel debate

Epiq Systems hosted a panel debate on the question of whether, and if so when, it was appropriate to use technology in legal disclosure (see box "The panel").
The panel was united in the view that the enormous increase in information, fuelled by the prolific use of email, mobile devices and social media, meant that the traditional model of document review was increasingly ineffective from a fiscal and practical perspective. All panellists and the audience agreed that technology was the only means by which lawyers could hope to cut through vast amounts of information within reasonable timescales and at a palatable cost. Lawyers and their clients are well used to using keywords as a means of searching information, but an important question is whether (and if so how) parties should begin to use next generation technologies, such as predictive coding, to achieve further efficiencies.

Using technology

The panel recognised that lawyers continued to be reluctant to adopt technology-assisted review. This reluctance is fuelled by various drivers, including:
Lack of judicial approval. In the experience of Judge Andrew Peck, lawyers are creatures of precedent, waiting for cases to establish judicial approval before taking the plunge to use new methods. Luckily, precedents are forthcoming. There have been two cases in the US in 2012 in which parties have received judicial approval for the use of predictive coding (Da Silva Moore et al v Publicis and MSL Group (No 11 Civ 1279, ) and Global Aerospace v Landow (L.P., No. CL 61040)) and Judge Peck predicts that an increase in such cases will prompt lawyers to use the technology more widely.
Master Steven Whitaker was of the view that lawyers in the UK are less likely to be waiting for precedent and more likely to be reluctant to use predictive coding because of a lack of understanding of how the technology actually works.
Fear of cost. Master Whitaker's experience is that lawyers currently try to use techniques that avoid the use of technology providers because of the fear of the cost of using more complex technologies. However, in his view, what they fail to take into account is that these costs are part of the end process which will ultimately reduce the cost of the matter as a whole.
Clearly, if the parties are able to locate the most relevant documents more quickly they will, for example, be able to consider whether settlement is appropriate at an earlier stage. Master Whitaker concluded that "Predictive coding offers a pragmatic and proportionate means of reducing the number of documents that human eyes are going to look at", and noted that it may ultimately be a better way of deciding what not to look at.
Robert Lewis raised the interesting point that while many companies are not yet using predictive coding, the functionality to do so is likely to be available in the disclosure systems that they currently use. Certainly at Barclays, predictive coding functionality is to be included in the next upgrade of its existing review system tool and the organisation intends to use it once the upgrade has been tested and reviewers have been trained.
Barrister Shantanu Majumdar pointed to lawyers' general levels of ignorance and fear of technology, which has been exacerbated by a number of recent cases in which parties got things wrong during disclosure and were penalised in costs (for example, West African Gas Pipeline Co Limited v Willbros Global Holdings Inc (2012) EWHC 396; see News brief "Electronic disclosure under scrutiny: what now for offshore document review?").
The black box. Lawyers are comfortable using all sorts of technology without understanding how it works: everyone is very happy to use Google searches or Microsoft Office applications without understanding the underlying algorithms or coding. However, for some reason they tend to fear using predictive coding because they do not have a detailed understanding of how it works.
Master Whitaker confirmed that he is interested only in the techniques adopted by the parties in training the software and how it is used; if good quality information is used to train the software, and effective quality control is deployed, good results will come out of the exercise. He continued that when considering predictive coding, litigants should "trust that it does what it says on the tin" rather than requiring the investigation of complex algorithms.
Parties need to be careful about setting accuracy levels and must understand that machines will not give perfection, but human review does not do this either. Master Whitaker concluded that there must be an acceptable margin for error if one is adopting a pragmatic and proportionate process. Judge Peck agreed that, so long as the quality control statistics showed that the parties were producing the right documents and that they had undertaken the appropriate sampling of non-relevant documents, this would be sufficient.

Agreed procedures

In the event that the parties overcome their reluctance to use the software, it is also important that the parties are able to be confident that the court will approve of the manner in which the software has been deployed.
Master Whitaker stressed the importance of the parties' agreement of a protocol establishing how the software is to be trained, the level of input which opponents will have, and how disputes should be resolved. Both judges agreed that in order for predictive coding to be effective, the parties will need to agree that both parties will have an opportunity to consider the seed set (that is, the set of documents used to train the software which may consist of both relevant and irrelevant documents) to ensure that the process of training the software is transparent and open. In this way, it will be difficult for opponents to undermine the methodology adopted by a party at a later date.
This is by no means the end of the discussion and lawyers are likely to continue to approach new technologies with caution. The debate was, however, a useful sounding board for those lawyers considering taking the leap.
Greg Wildisen is International Managing Director at Epiq Systems.

The panel

Epiq Systems assembled a panel representing the judiciary in the UK and US, the bar and commerce to debate the issue of predictive coding.
The panellists were Steven Whitaker, Senior Master and Queen's Remembrancer, Magistrate Judge Andrew Peck, Southern District of New York (who ruled on Monique Da Silva Moore v Publicis Groupe and MSL Group), Robert Lewis MBE, Global Director of CFI/eDiscovery at Barclays Bank plc, and Shantanu Majumdar, a barrister from Radcliffe Chambers.