Enter to open, tab to navigate, enter to select
Predictive coding: an essential consideration?
Practical Law UK Articles 0-630-1983 (Approx. 4 pages)
Predictive coding: an essential consideration?
by Oliver Glynn-Jones, Robin Ganguly and Nick Pryor, Berwin Leighton Paisner LLP
| Published on 30 Jun 2016 |
The High Court's order in May 2016 permitting predictive coding is a significant step as by allowing one party to use predictive coding against the wishes of the other party, the High Court has provided a strong view that the technology is more than just a viable alternative to traditional review methodologies.
The High Court's order in May 2016 permitting predictive coding, in an unfair prejudice petition under section 994 of the Companies Act 2006 between BCA Trading Limited and Mr Brown, is a significant step (David Brown v BCA Trading (unreported)). By allowing BCA Trading to use predictive coding against Mr Brown's wishes, the High Court has provided a strong view that the technology is more than just a viable alternative to traditional review methodologies. The arguments in favour of the technology, which is also beginning to be used in US litigation, were so manifest and compelling that there was no credible basis for any other approach.
In February 2016, the High Court delivered its first reported judgment approving the use of predictive coding in the UK (Pyrrho Investments Limited and another v MWB Property Limited and others [2016] EWHC 256 (Ch); see Briefing "E-disclosure: lift off for predictive coding technology?"). This judicial affirmation of the document review technology was hugely significant. However, it is one thing for the court to approve an innovative approach to disclosure agreed between the parties, it is quite another thing for the court to permit a party to use predictive coding in the face of party opposition.
The BCA Trading decision
Mr Brown sought a buyout of his minority shareholding in BCA Trading. The allegations in the petition are strongly contested and wide-ranging, thereby putting a significant disclosure burden on BCA Trading, which holds the significant majority of potentially relevant documents. Furthermore, the quantum of the case is deeply uncertain; Mr Brown considers the claim to be worth tens of millions of pounds, while BCA Trading considers the shares to be essentially worthless. This brings into sharp focus the need for the parties to ensure that the proceedings, and disclosure in particular, are run in an efficient and proportionate manner.
Mr Brown wished to adopt a traditional approach to document review, whereby the inboxes of an agreed list of custodians would be crudely filtered using keywords and similar parameters. All remaining documents would then be reviewed in a linear fashion by a paralegal. BCA Trading argued that the costs of this approach would be excessive, and that superior results could be achieved more proportionately by using predictive coding.
BCA Trading referred the court to the ten factors set out in Pyrrho in favour of using predictive coding. Of those, one was not applicable (namely, that the parties have agreed on the use of the software, and also how to use it, subject only to the approval of the court) and one was neutral (namely, that there is nothing in the Civil Procedure Rules or Practice Directions to prohibit the use of predictive coding). All of the other factors that weighed in favour of using predictive coding in Pyrrho also applied in this case. On this basis, the court had little hesitation in ordering that BCA Trading's solicitors, Berwin Leighton Paisner, could use predictive coding.
Confronting inaccurate perceptions
A common refrain from those averse to predictive coding is a fear of losing control of the disclosure process. This is misconceived. Predictive coding is not a "black box" technology, delegating case-critical decisions about what should be disclosed or withheld to a computer. It is machine learning technology that is directed and controlled by human decision making (see Know how articles "E-disclosure: the state of the art" and "Judicial approval for predictive coding: canvassing opinion").
At the outset, a senior lawyer well versed in the case reviews a relatively small "seed set" of documents. These decisions are then analysed by the predictive coding engine and used to generate a further sample output for review. Through a process of iterative refinement, the algorithm can reach a level of review accuracy that can be applied across the entire data set, categorising documents by relevance or by issue in a manner that is far more efficient and scalable than a traditional document review.
The cost advantages of this approach are beyond question (see Briefing "Using technology in litigation: costs savings and competitive advantages"). In BCA Trading, Berwin Leighton Paisner estimated that predictive coding would cost less than a third of what it would take to use a linear paralegal review approach. A significant factor in being able to achieve this level of cost savings was the fact that Berwin Leighton Paisner could run the predictive coding work in-house, rather than needing to pay an external technology provider. However, even an outsourced model could deliver significant efficiencies over traditional review models.
The advantages of the predictive coding model go much further than merely saving costs. It stands to reason that an extrapolation of the senior lawyer's decisions will be more reliable than the decisions made by a disparate body of paralegals making subjective, independent assessments under time pressure. This is borne out by every objective comparison; predictive coding is simply more consistent and more accurate in its decision making.
So the argument for predictive coding on quality and costs grounds is beyond reproach. As for the biggest remaining objection, namely, loss of control, the fact is that predictive coding requires no surrender of control, but actually provides the lawyers with much greater control of the process (see box "The power of predictive coding").
Embracing legal technology
The decision in BCA Trading affirms the authors' view that law firms should embrace the opportunities provided by technology, rather than holding to traditional, familiar processes and that emerging technologies can improve the quality and efficiency of legal work. This approach is right for law firms but, more importantly, it is right for their clients.
Oliver Glynn-Jones is a partner, Robin Ganguly is a senior associate, and Nick Pryor is a principal knowledge development lawyer, at Berwin Leighton Paisner LLP.
The power of predictive coding
A great example of how predictive coding empowers the legal team can be seen in the way the engine codes the documents. It does not provide a crude, binary assessment of whether each document is relevant, but instead graduates the documents according to its confidence in its own coding. This enables the solicitor to determine what would be a proportionate percentage of the full data set to subject to human review in order to verify and correct the results, to sweep for privileged materials and to eventually disclose. This cost/benefit analysis, and assessment of proportionality by reference to the overriding objective in the Civil Procedure Rules, can be precisely calibrated using the system's recall and precision figures.
For example, a solicitor might initially propose that the system should review the top 10,000 documents identified by the system, from an original data set of one million. The solicitor can then consult the system's recall and precision metrics for that 10,000. Recall refers to the system's confidence in the percentage of all relevant documents contained within that 10,000; that is, how many relevant documents it thinks that it has missed. Precision refers to the system's confidence in its own accuracy; that is, how many of the 10,000 have been correctly flagged as relevant, and how many are false positives.
Armed with that knowledge, the solicitor, ideally in consultation with the other party to the dispute, and potentially even with reference to the court, can determine whether 10,000 is the appropriate threshold. Reviewing 20,000 documents might increase the recall from 80% to 85%, but the returns would be diminishing since more false-positives would leak through, and of course it would double the cost of the review. Document review is always an exercise in balancing completeness against proportionality but predictive coding enables the lawyers to make much more informed and precise choices about where that balance lies, and to keep within increasingly strict costs budgets.