In US civil lawsuits, courts will permit a producing party to use technology-assisted review (TAR), software that learns from lawyers' coding to sort documents, according to a March 2, 2015 opinion by Magistrate Judge Andrew Peck in Rio Tinto PLC v. Vale S.A. The same opinion says TAR should not be held to a higher standard than keyword searches or manual review. It approved a protocol that checks results with statistical sampling and left open how much training material a party must share.
Key takeaways
- TAR trains software on lawyers' coding of a small set of documents, then ranks or sorts the rest by likely responsiveness.
- The Rio Tinto protocol checks the work with random samples sized for a margin of error of 2 percent at a 95 percent confidence level.
- Courts the opinion cites refused requests to force a producing party to use TAR, and the opinion treats the producing party as best placed to choose its method.
- Whether the training set must be disclosed was called an open question in 2015, with decisions split when parties had not agreed.
- The opinion is a 2015 snapshot of one court's view and does not address newer generative AI tools.
What the court approved
The case was a dispute between a mining company and a competitor in the Southern District of New York. The parties agreed on a TAR protocol, and the judge wrote a full opinion, not just a signature on the stipulation, because of interest in TAR cases among e-discovery practitioners. He stressed that the approval followed the parties' agreement and was not a court order, and that it did not endorse any vendor or tool.
The opinion traces the law back to Da Silva Moore v. Publicis Groupe, decided on February 24, 2012, which the author describes as the first decision approving computer-assisted review. By 2015 he found the case law had settled enough that a producing party who wanted TAR would be allowed to use it. The Tax Court had reached a similar view in Dynamo Holdings, decided September 17, 2014, saying that courts generally leave the choice of review process to the parties. The full text is available as a published copy of 306 F.R.D. 125.
How the protocol works, step by step
The stipulation attached to the opinion defines each stage. The table summarizes it; the stages are simplified, and the document governs the details.
| Stage | What happens | What the protocol requires the responding party to disclose |
|---|---|---|
| Control set | A statistically valid random sample is reviewed to estimate how many documents are responsive | Sample size, coding counts, and all non-privileged documents reviewed |
| Seed set | Reviewers code an initial batch used to train the software | Size, selection method, and the non-privileged documents with their coding |
| Training rounds | Further batches are coded and the software is retrained | Counts, plus estimated recall and precision with error margins |
| Uncategorized documents | Files the software cannot read or rank are reviewed in full | The number of such documents |
| Validation set | A sample of documents ranked as non-responsive is reviewed | Sample size, responsive documents found, and implied recall |
The protocol also requires written notice before review starts, naming the software, the person overseeing the process, and the document universe. The responding party must make that technical expert reasonably available for questions.
Reading recall and precision
The stipulation defines precision as the fraction of documents flagged as likely responsive that really are responsive. Recall is the fraction of all responsive documents that the process flagged. A process can score high on one and low on the other. A statistically valid sample, in the protocol's words, permits extrapolation with a margin of error of plus or minus 2 percent at the 95 percent confidence level.
A footnote to the opinion gives the sample-size arithmetic. With a population of 1,000,000 documents, a 95 percent confidence level, an assumed richness of 0.5, and a 2 percent margin of error, the required sample is 2,395 documents. Richness means the share of responsive documents; 0.5 is used when it cannot be estimated. These are the opinion's own example numbers (illustrative, not a recommendation for any case).
Who chooses the method
The opinion ties its view to a Sedona Conference principle that responding parties are best situated to judge the methods and technologies for producing their own electronic information. It notes that when requesting parties asked courts to force TAR on a producing party, the courts refused, citing In re Biomet and Kleen Products. It adds that in those cases the producing parties had already spent over USD 1 million on keyword-based review, so it was unclear what a court would do if the issue came up before any money was spent.
Two federal rules frame the practical stakes. Rule 26(b)(1) limits discovery to what is proportional to the needs of the case. Rule 26(g) requires an attorney's signature on discovery responses to certify, after reasonable inquiry, that they are complete and neither unreasonable nor unduly burdensome. The text of Rule 26 is public.
Transparency: the question left open
The judge called seed-set transparency unresolved. In Da Silva Moore the producing party volunteered to share its seed set, and a bench ruling in a federal housing finance case required it. A different court found no authority to require sharing and suggested the party reconsider. The opinion says that where parties do not agree, decisions are split.
It adds two points. With continuous active learning, in which review and training run together, the contents of the seed set matter much less. And a requesting party can test the work by other means: a statistical estimate of recall at the end, a look for gaps in the production, and quality control review of samples from documents coded non-responsive. The opinion cannot be read as a rule for every court, and later decisions were not reviewed for this page.
Limits of this page
This page describes one reported opinion and the protocol attached to it, as read from the published text. It does not cover Canadian discovery practice, state courts, or generative AI review tools, and it is general information, not legal advice. Anyone facing a real discovery obligation should rely on counsel and the current rules of the court involved.
FAQ
Is technology-assisted review the same as predictive coding?
The opinion treats the terms predictive coding, computer-assisted review, and technology-assisted review as names for the same approach, and says TAR had become the preferred term by 2015.
Can a court make a party use TAR?
The opinion cites decisions in which courts refused to force it on a producing party, while noting the open question if the issue arose before any review spending.
Does TAR replace human reviewers?
No. In the protocol, lawyers code the control, seed, and training sets, and humans review the validation sample and any documents the software cannot evaluate.
Did the court say how much recall is enough?
No. The protocol requires disclosure of estimated recall and precision but sets no threshold, and the parties' vendors described repeating rounds until the metrics reached an acceptable level.
Related reading
- An AI review assistant and the review process
- Human reasoning inside AI systems
- Advancements and challenges in AI technology
- A plain-language review of AI basics
More in AI Tools and Industry.
Last verified: October 2026
Sources
- Rio Tinto PLC v. Vale S.A., 306 F.R.D. 125 (S.D.N.Y. March 2, 2015), Opinion and Order with attached stipulation and order, Magistrate Judge Andrew J. Peck
- Federal Rules of Civil Procedure, Rule 26, Legal Information Institute, Cornell Law School