Increased Data Storage Method

About

TECHNOLOGY NEED As the world moves to digital storage for archival purposes, there is an increasing demand for systems that help to increase storage efficiency in the cloud environment. Data compression and deduplication are two common ways of doing this. Existing data deduplication systems cannot identify redundant data between the compressed, uncompressed, and differently compressed versions of folders with exactly the same content. This is because compressed contents will have dissimilar string patterns from their uncompressed counterparts. Also, when the same contents are compressed by different compression algorithms dissimilar string patterns are observed. As different users share the same digital contents with different compression methods this type of hidden data redundancy increases. To have a more efficient cloud storage system, it is necessary to detect and remove the redundancy that stems from compressed data packages.  INVENTION DESCRIPTION/SOLUTION Researchers at UTA have developed an approach to extract unique metadata, such as the “checksum” and “file length” information from compressed data packages. The “checksum” and “file length” are used as file content signatures to help detect and remove file level data redundancy. The method proposed is called the X-Ray Dedup. X-Ray Dedup is shown by evaluation to can break the boundaries of compressed packages and significantly reducing compressed packages’ size requirements, thus further optimizing storage space in cloud environments.  APPLICATIONS Cloud storage optimization. Deduplication of compressed contents. KEY BENEFITS Identifies the redundant data between compressed, uncompressed and different compressed versions of similar/identical files in a cloud storage environment.