Single-instance storage

Single-instance storage (SIS) is a system's ability to take multiple copies of content and replace them by a single shared copy. It is a means to eliminate data duplication and to increase efficiency. SIS is frequently implemented in file systems, e-mail server software, data backup, and other storage-related computer software. Single-instance storage is a simple variant of data deduplication. While data deduplication may work at a segment or sub-block level, single-instance storage works at the whole-file level and eliminates redundant copies of entire files or e-mail messages.^[1]

Concept

In the case of an e-mail server, single-instance storage would mean that a single copy of a message is held within its database while individual mailboxes access the content through a reference pointer. However, there is a common misconception that the primary benefit of single-instance storage in mail servers is a reduction in disk space requirements. The truth is that its primary benefit is to greatly enhance delivery efficiency of messages sent to large distribution lists. In a mail server scenario disk space savings from single-instance storage are transient and drop off very quickly over time.^{[citation needed]}

When used in conjunction with backup software, single-instance storage can reduce the quantity of archive media required since it avoids storing duplicate copies of the same file. Often identical files are installed on multiple computers, for example operating system files. With single-instance storage, only one copy of a file is written to the backup media therefore reducing space. This becomes more important when the storage is offsite and on cloud storage such as Amazon S3. In such cases, it has been reported that deduplication can help reduce the costs of storage, costs of bandwidth and backup windows by up to 10:1.^[2]

Novell GroupWise was built on single-instance storage, which accounts for its large capacity.

ISO CD/DVD image files can be optimized to use SIS to reduce the size of a CD/DVD compilation (if there are enough duplicated files) to make it fit into smaller media.

SIS is related to system wide file duplication search and multiple file instance detection tools such as the P2P application BearShare (5.n Versions and below) but differs in that SIS reduces storage utilization automatically and creates and retains symbolic linkages, whereas Bearshare allows for manual deletion of duplicates and associated user-level file system, Windows Explorer type of icon links.

Microsoft

SIS was introduced with the Remote Installation Services feature of Windows 2000 Server. A typical server might hold ten or more unique installation configurations (perhaps with different device drivers or software suites) but perhaps only 20% of the data may be unique between configurations.^[3] Microsoft states that "SIS works by searching a hard disk volume to identify duplicate files. When SIS finds identical files, it saves one copy of the file to a central repository, called the SIS Common Store, and replaces other copies with pointers to the stored versions."^[4] Files are compared solely by their hash functions; files with different names or dates can be consolidated so long as the data itself is identical.^[3] Windows Server 2003 Standard Edition has SIS capabilities but is limited to OEM OS system installs.^{[citation needed]}

The file-based Windows Imaging Format introduced in Windows Vista also supported single-instance storage. Single-instance storage was a feature of Microsoft Exchange Server since version 4.0 and is also present in Microsoft's Windows Home Server. It is deduplicating attachments only in Exchange 2007 and was dropped completely in Microsoft Exchange Server 2010.^[5] Microsoft announced Windows Storage Server 2008 (WSS2008)^[6] with Single Instance Storage on June 1, 2009, and states this feature is not available on Windows Server 2008.^[6]

The feature is officially deprecated since Windows Server 2012, when a new, more powerful chunk-based data deduplication mechanism was introduced. It allows files with similar content to be deduplicated as long as they have stretches of identical data. This mechanism is more powerful than SIS.^[7] Since Windows Server 2019, the feature is fully supported on ReFS.^[8]

See also

• Capacity optimization
• Data deduplication
• Peer-to-peer file sharing
• WinFS

References

1. Explaining deduplication rates and single-instance storage to clients. George Crump, Storage Switzerland
2. Deduplication + Amazon S3 will save you time and money. White Paper: Published June 2008
3. Douceur, John (JD); Goebel, David; Corbin, Scott; Bolosky, Bill (August 2000). "Single Instance Storage in Windows 2000" (PDF). Microsoft Research. Microsoft Research and Balder Technology Group.
4. Single Instance Storage in Microsoft Windows Storage Server 2003 R2 Archived 2007-01-04 at the Wayback Machine: Technical White Paper: Published May 2006
5. [1] The Exchange Team Blog, Microsoft Corp.
6. Windows Storage Server 2008 at Microsoft
7. FileCAB-Team (10 April 2019). "Introduction to Data Deduplication in Windows Server 2012". Microsoft Tech Community.
8. "Data Deduplication interoperability". docs.microsoft.com. 29 March 2022.