Insights & News

E-Discovery Alert
Predicting the Future of Document Review
Themes in the Computer-Assisted Review Cases

December 19, 2014
Client Alert
Predictive coding or “computer-assisted review” (CAR) – essentially, the use of sample sets from a collection of electronically stored information (ESI) to train a computer algorithm to identify and categorize additional ESI based on the judgments and preferences reflected in the user’s coding of the sample documents – continues to be one of the hottest topics in electronic discovery. While the use of CAR to identify documents for production in discovery is still relatively new and remains subject to characterization by opponents as “unproven technology,” it has been nearly three years since Magistrate Judge Peck announced in Moore v. Publics Groupe (S.D.N.Y. Feb. 14, 2012, adopted sub nom. Apr. 26, 2012) that the wait for its judicial acceptance was over. The number of cases remains relatively small, and the details of these decisions differ. Nonetheless, when this developing body of cases is viewed as a whole, some clear themes emerge.

Compendium of Case Law
An online search identifies approximately 35 decisions referencing CAR in the discovery context, roughly two-thirds of which address the technique substantively. This of course does not include additional case management orders, negotiated protocols or other documents that reflect the potential or actual use of some form of CAR to expedite or enhance document review and production, but are not readily found through a standard public search.

Of the known cases, all have involved the proposed or actual use of CAR by one or more of the litigants – or in one instance, a nonparty – to identify documents for production to a requesting party. Courts considering the topic generally have embraced CAR’s merits, noting that it appears to provide an effective method to identify relevant documents, expedite review and production, and reduce discovery costs, particularly in cases with large volumes of ESI. The specific issues the courts have addressed, and the case outcomes, however, have varied. Some cases have considered whether CAR may or must be employed by a producing party or the scope of its use in a given case. Others have addressed specific aspects of its application and the extent to which the party receiving the production should have input – or at least insight – regarding this protocol.

Common Themes
In Moore, widely viewed as the first instance in which a court expressly approved a party’s use of CAR to identify ESI for production, plaintiffs had preliminarily agreed to defendant’s use of the technology, provided that safeguards were built into the protocol and it was established as reliable. The dispute was whether those standards had been met in the protocol as adopted. The court effectively found they had, as the protocol included both standards for measuring the process’s reliability and plaintiffs’ active participation. Post-Moore, courts began considering whether to permit or require CAR when one party supported it over the other party’s objections, either for the supporting party’s own production (e.g., Global Aerospace v. Landow Aviation (Va. Cir. Apr. 23, 2012)) or for its adversary’s production (e.g., Kleen Prod. v. Packaging Corp. of Am. (Sept. 18, 2012)). Courts also began, on their own initiative, to order its use or show cause why CAR was “not the way to go” (e.g., EORHB v. HOA Holdings (Del. Ch. Oct. 15, 2012)) or at least to encourage it (Nat’l. Day Laborer Org. Network v. U.S. I.C.E. (S.D.N.Y. July 13, 2012)). Additionally, parties began to seek courts’ views as to whether CAR could be employed on top of traditional keyword searches (e.g., In re Biomet (N.D. Ind. Apr. 18, 2013, and Aug. 21, 2013)) or when it was not contemplated by the case management order or discovery protocol (e.g., Bridgestone Am. v. Int’l Bus. Mach. Corp. (M.D. Tenn. July 22, 2014); Progressive Cas. Ins. v. Delaney (D. Nev. July 18, 2014)). One of the most recent cases addressed CAR in the context of privilege review and a Rule 502(d) clawback agreement (Good v. Am. Water Works (S.D. W. Va. Oct. 29, 2014)).

Collectively, the CAR cases reflect the impact of certain factors on the decisions reached by the courts and several common themes, and set the stage for a few predictions:
  1. Satisfaction of Discovery Obligations. Does the approach at issue – whether some form of CAR, keyword searching, linear review or a combination – efficiently and effectively identify responsive documents for production and fulfill the producing party’s discovery obligations under the specific circumstances of the case? Does it comply with the applicable rules and other broadly accepted guidelines?

  2. Proportionality and Timing of Request. Will the proposed review approach result in at least as complete a production of relevant information as another approach at a cost proportionate to the value of the case? Do the anticipated benefits of the approach at issue outweigh the estimated added burden and expense? At what point in the discovery process did the party seeking the use of CAR raise the issue? Does the timing adversely affect the proportionality analysis? For example, has the producing party already expended effort that effectively will be wasted? Did the parties agree at the outset to a very specific protocol that one party now wants to change in a way that risks delay or prejudice?

  3. Cooperation and Reasonable Transparency. Has the party using CAR involved the receiving party in the process or at least disclosed to that party critical elements of its approach? While courts generally have been consistent in urging cooperation among parties regarding ESI discovery generally, and in the CAR context specifically, there are active questions about transparency and the extent, if any, to which a producing party using CAR must – or at least should be encouraged to – involve an opposing party in the process, especially the identification of the sample documents used to train the tool. Does CAR – as one court has suggested – require an “unprecedented degree of transparency and cooperation” that is fundamentally different from earlier search, review and production methodologies?

  4. Discovery Status and Focus on Results Over Process. Is a party’s objection to an opponent’s use – or nonuse – of CAR premature because the producing party’s production has not been shown to be clearly deficient or is still ongoing? Should courts be asked to decide how parties may conduct their document review or instead simply whether that process – however structured – resulted in a reasonably timely and complete production of responsive, non-privileged information?

  5. Case-Specific Determination of Methodology Within an Evolving Framework of Best Practices. There is no “one size fits all” model or black line rule for when CAR may or should be used. General guidelines have emerged with respect to the types of case and the volume and nature of data for which CAR has been found to yield the most effective results. Ultimately, however, whether its use is appropriate in a given matter is a judgment call that depends on several case-specific considerations. As has occurred over the past several years, CAR technology will continue to mature and the types and volumes of ESI potentially at issue will further develop. In addition, more and more litigants will build experience and expertise both in assessing when, how, and on what types and volumes of ESI CAR can and cannot be used effectively and in responding to the related expectations and demands of courts, clients and opposing parties. As that occurs, best practices with respect to these issues and the associated tools, workflows and team members – and the specificity with which this information should or must be memorialized in a formal order or otherwise shared with opposing counsel or the court – will continue to evolve.

  6. Each Player’s Role in the Process. The CAR cases bring to the forefront the question in discovery of “who should decide what?” CAR is one of the many e-discovery areas in which federal, state and administrative judges have become increasingly informed, sophisticated and impatient with litigants who are not. At the same time, litigants and their experts may be better positioned to address and cooperatively resolve certain technical issues relating to CAR implementation, including, for example, the selection of one computer training methodology over another or the specific metrics to use to confirm effectiveness of the process. In addition, as the court recently explained in Dynamo Holdings Ltd. P’ship v. Comm’r of Internal Revenue, 143 T.C. No. 9, *3 (T.C. Sept. 17, 2014), “although it is a proper role of the Court to supervise the discovery process and intervene when it is abused by the parties, the Court is not normally in the business of dictating to parties the process that they should use when responding to discovery.” As the use of CAR and related technologies expands and litigants continue to grapple with efficient and effective methods for identifying and producing, from the massive quantities and varieties of ESI data often potentially at issue, the subset of information that is non-privileged, responsive to an opposing party’s requests and relevant to the issues in dispute, courts undoubtedly also will be asked to consider new, not-yet-contemplated questions and disputes that are likely to arise.
The posting of information on this website, or the receipt of information by viewers of this website, is not intended to – and does not – create an attorney-client relationship. This website is not intended to provide legal advice, and visitors to this website should refrain from acting on information posted here without seeking specific legal advice from individually qualified counsel.

Related Services

back to top