E-Discovery Resource Guide
Employ a Reasonable Method, Know Its Limits
Executive Counsel July/August 2009
Search is fundamental to discovery. Before discovery went digital, search typically involved lawyers or paralegals gathering information manually, primarily through interviews of custodians and witnesses, about where relevant paper documents were likely to be found. Usually lawyers would leave it to the custodian to provide the relevant documents. Sometimes lawyers would search through files provided by the custodian, or go to the location where the relevant documents were likely to be found, and then rummage around in the papers.
When the discoverability of electronically stored information (“ESI”) was established, the fact that it could be searched electronically brought hope that the search would become easier and less expensive. But it turns out that while electronic search has reduced the need for humans to cull laboriously through documents to find those that are relevant, the sheer volume of ESI, and its many sources and formats, presents significant new challenges even with state-of-the-art search technology.
The use of keywords, a technique familiar to lawyers from their experience with computerized legal research services, constitute for many legal professionals the definition of electronic search. Today it is common for lawyers to negotiate or litigate over which keywords are to be applied as search terms to a corpus of potentially relevant information.
However for a number of reasons, including the infinite quirks and variations of language as it is actually employed, using keywords as a measure of potential relevance is likely to be over-inclusive in some respects and under-inclusive in others. It may be impossible to know how serious this problem is in any particular case, but the available research suggests that it is substantial.
The inescapable truth about keyword searches is that (a) document reviewers spend a lot of time, and therefore clients spend a lot of money, reviewing documents that are not even close to being relevant to their case; and (b) a substantial number of documents that are relevant are never even reviewed.
This status quo is not acceptable, not to clients who want to avoid wasting money, not to their adversaries who cannot accept the idea that they are not getting all of the smoking guns, and not to judges, who want cases to be decided on the merits.
Accordingly, tremendous re-sources are being focused on improving the electronic search. In effect, we are in an arms race. While search technology is advancing rapidly, the extent of ESI, in terms of both variety and volume, continues to increase. Moreover, problems with applying electronic search tools to ESI are beginning to appear on the radar of lawyers and the courts.
Several recent judicial opinions have explicitly recognized these issues and called into question the adequacy of one or another party’s ESI search protocols. The message to lawyers is that they need to pay close attention to the way they search for relevant and/or privileged ESI. These opinions recognize that simply using a reasonable set of keywords only scratches the surface of the problem of ESI search.
The imperative for lawyers to find a better way has become even more pronounced with the passage of Federal Rule of Evidence 502, which among other things requires that reasonable measures be taken to prevent the disclosure of privileged materials to avoid a waiver of privilege by inadvertent disclosure.
Not surprisingly, technology vendors have reacted to these opinions and Rule 502 like sharks to blood in the water. In the scramble to peddle solutions to lawyers struggling to get their arms around the practical implications of increased scrutiny of search techniques, many vendors have made fantastic claims about what their products can do. Clients who want to reduce costs look to technology as a way to narrow the universe of documents that leave the confines of the corporation, and thus tame expenses for ESI processing, hosting and review. But, caveat emptor: There are things lawyers and clients need to keep in mind in looking for salvation in search technology.
The reality is that many so-called solutions were created for an entirely different purpose, loosely defined as “enterprise content management” (ECM). Vendors of “enterprise search” and email archiving products are trying to cash in on the electronic discovery bonanza by adapting products they created to facilitate business operations to the litigation context. Many of these vendors do not understand how different litigation is from the day-to-day operation of a business.
In litigation, an adversary is scrutinizing your every move and is ready to pounce on any failure to search for or produce a relevant document. Missing an important document can be sanctionable and, as a practical matter, dispositive for the case. Therefore system limitations that might be acceptable in the business setting may be unacceptable in litigation.
At a minimum, lawyers need to be aware of those limitations. They need to understand, for example, that the technology they are employing may not be able to search some file types or report that certain files, like those that were password protected, could not be searched. Litigation may put these limitations in an ugly light if you are not aware of them in advance.
There is no magic bullet in the form of a product for the problem of performing defensible searches. It is important not only to recognize the limitations of any particular search technology in the litigation environment, but also to recognize that search in the discovery context is not just about applying technology. It is about applying a process or methodology that is reasonable under the circumstances.
At least for now, there is no one way to search that constitutes the only single valid solution to legal compliance, even though the outlines of what defensible searches might look like are starting to emerge. But by combining a reasonable process with widely available search technology, you can defend your hunt for relevant documents – even when it is accompanied by the full disclosure that no one can guarantee that every single relevant bit of ESI has been produced.
What will the new world of electronic search look like? There are many possibilities. One, which uses current search technology, begins as do all discovery searches, by identifying potentially relevant sources of ESI. And, as it always has, the identification of sources will involve gathering information by interviewing IT personnel, witnesses and information custodians, and by reviewing existing descriptions of the IT environment.
When the sources that need to be checked have been identified, it can then be determined whether an electronic search can be applied and if so what kind of search it should be. Where a keyword search is the starting point, the results can be reviewed and the keyword search refined for greater accuracy.
A number of search tools on the market can use a given document to locate other documents that are similar but may not contain the same keywords. These tools can then be used to refine a keyword search further, to suggest other potential exploration directions, as well as to identify dead ends. Depending on the document universe in question, statistical sampling might then be employed to evaluate the accuracy of the search process.
Whether something like this will become a de facto standard is not certain, but it seems clear that lawyers will have to get used to defending their discovery search protocols and technology. This is consistent with the goals of the 2006 amendments to the Federal Rules of Civil Procedure, insofar as it will provide greater transparency to the discovery process.
There is some concern that recent opinions addressing search will lead to an explosion in motion practice over disputed methods, possibly requiring hearings and testimony by expert witnesses, and thereby raising the cost of discovery even more. This possibility suggests that cooperation and compromise between adversaries to arrive at mutually acceptable search protocols is more important than ever.
The balanced view is that while no search is likely to be perfect, effective ways of identifying relevant documents are available. We should all welcome the possibility that more cases are decided on their merits, and not on who does a better job of hiding the ball when it comes to ESI.
Adam I. Cohen (e-mail) is a Senior Managing Director with FTI Technology. He is the co-author of “Electronic Discovery: Law and Practice” (Aspen 2004, updated annually) and “ESI Handbook: Sources, Technology and Process” (Aspen 2008). He is a member of the Executive Counsel Editorial Advisory Board.