The time has truly arrived for parties and their legal teams, courts and tribunals, to engage with (or perhaps discover) Technology Assisted Review (‘TAR’). Often referred to as ‘predictive coding’, TAR looks set to become a core tool in the data review and disclosure and document production processes, in medium to large scale litigation and dispute resolution.
TAR involves a human reviewer “training” a computer system to identify and classify relevant documents within large volumes of data. Typically, batches of documents (which may number in the thousands) from a data set (which may include millions of documents) are reviewed by senior members of a legal team and classified for relevance and other pertinent categories (for example, whether or not they are privileged).
The computer system then extrapolates out the parameters set by the human reviewer’s classification of documents and applies those parameters to the remaining documents. These results are then themselves further tested by human review (often several times). The test results are fed back to the computer system, for it to develop and refine its appraisal of the relevance of the documents. Ultimately, the testing process ends when it reveals the disparity between human and computer classification of documents falls within a satisfactory margin of error.
Following the recent decisions in Pyrrho Investments v MWB Property and Brown v BCA Trading, TAR has been approved by the English courts. Both cases are well worth a careful read.
In Pyrrho, the disclosure exercise required the restoration of back-up tapes (even after a de-duplication exercise), and over 3 million electronic documents fell to be searched and reviewed. The evidence before Master Matthews was that adopting a TAR process meant that, in real terms, a senior reviewer would have to review a total of only around 24,000 documents. Remarkably, the cost of a TAR exercise was said to be between £182,000 and £470,000, compared to a manual review which was predicted to be at least several million pounds. The Master held that TAR could amount to a reasonable search for documents and was permitted under Civil Procedure Rules.
The court observed, with reference to authorities from other jurisdictions, that firstly, there is no evidence to show that the use of TAR software leads to less accurate disclosure; indeed there is some evidence (referred to in the US federal case of Moore v Publicis Groupe and the Irish case of Irish Bank Resolution Corporation Ltd v Quinn) that it is at least as accurate, if not more accurate, than a manual review of documents. Secondly, there will be greater consistency in using TAR to apply the approach of a single senior lawyer, than in using dozens, perhaps hundreds, of lower-grade fee-earners, each seeking independently to apply the relevant criteria in relation to individual documents.
In BCA Trading, a much smaller dataset (around 500,000 documents) was involved and the costs of disclosure were much smaller, around £132,000 for TAR compared to around £300,000 for more traditional methods. Nevertheless, the court held that TAR was the proportionate way to deal with disclosure.
TAR is not a silver bullet. Some electronic files, such as images, audio files, photographs and Excel spreadsheets are not suitable for predictive coding and will usually require a separate parallel manual review exercise. Similarly, hard-copy documents which are scanned and subject to optical character recognition (‘OCR’) may not be suitable for TAR due to inevitable errors in the scanning process.
As noted by the Master in Pyrrho, TAR will usually be justified in circumstances where the size of the claim and the volume of the document set means that the cost of TAR can be achieved at a lower cost than manual review. In lower value cases, the front-loading of cost, and the costs of processing and hosting the electronic documents required to carry out TAR, may not be proportionate. Recent research by Consilio suggests that the document set should be at least 50,000 documents in size to make TAR appropriate.
TAR is a process that should be borne in mind when a document review and/or disclosure/production process is to be undertaken; if only to be discounted as inappropriate to the task in hand. When it is suitable and used appropriately, it is likely to result in substantial savings in both time and costs.
