Data Deduplication

The ability to distribute multiple copies of documents, communicate with multiple people at once, and automatically capture copies of all of the above—these are some of the greatest conveniences ever offered by the advancement of technology. But when it comes to electronic discovery, all that repetition can represent a significant portion of data to be reviewed, potentially lengthening the discovery process.

The remedy is deduplication or “de-duping,” running each file’s unique “fingerprint” or identifier code through algorithms that cull the duplicates and near duplicates. Thus, repeated information is removed and the volume of data to be processed is greatly reduced. On the razor’s edge of application technology, deduplication methods are continually getting faster and more efficient. Now, for example, a set of emails can be processed to find duplicates not only from within one host system but across many, more easily boiling down intra-enterprise email chains than ever before.