De-duplication Of .fsrm file in Server 2012

[Natstar]

Hello,

Not too sure if anyone else is playing around with this. We are pulling off our Long Term recovery goals in DPM 2012R2 to another Windows Server 2012 box using Firestreamer. I've tuned on de-duplication and am achieving about a 22% saving. I see from a previous post that someone was having very low dedup ratios using a Data Domain device and that this was due to Data Domain being unaware of the file format of Firestrermer's fsrm files. My questions are; is anyone else using dedup in 2012 and what savings are you getting? I know that Windows dedup works very differently to may other dedup technologies, but is this a similar situation with Data Domain i.e. does Windows dedup have to have knowledge of fsrm files to increase the dedup saving?

Looking forward to hearing any more information!

Many thanks,
Geoff

[jsf]

Deduplication splits files into data blocks and maintains only one copy of the same block across multiple files. The problem is, when tape data blocks are converted into a file, the tape blocks are prepended with a small header, so that (tape_block + header)=file_block. Because the headers are likely to be different even if tape blocks contain the same data, the resulting file blocks are also different even if the tape data in those file blocks is the same. Without the knowledge of the header, it's difficult to identify identical blocks of tape data within file blocks, hence the low dedup rate.

We will see if we can do anything about it in future versions of Firestreamer.

[Natstar]

John,

As always, a full response! Many thanks for giving us details of why dedup ratios can be low with .fsrm files.

I'm sure that you can appreciate the exciting world that is deduplication in Windows Server 2012: for us it is turning into a vital feature that we are using more and more on our servers to try and reduce our storage requirements (we are a college and have had our budgets cut yet again). To be able to dedup .fsrm files more effectively would be a huge bonus for us. If anything can be done in future versions of Firestreamer to improve this I think that many Firestreamer users will see the huge advantages of being able to dedup their VTLs.

Many thanks,
Geoff

[jsf]

Geoff, thank you for your feedback.

[lavaliant]

Has there been any progress in supporting Windows 2012/2012R2 deduplication for Firestreamer to increase deduplication ratios? Currently, I'm not getting any space savings with Windows 2012 deduplication. Thanks...

[jsf]

We have not released the version that specifically targets deduplication yet, sorry. However, we may have a beta version ready within the next several weeks. Would you be willing to test it? If yes, please contact us privately at our email address. Thank you.

[da]

Is there a release date of version that supporting deduplication in Windows 2012 R2? Thanks.

[jsf]

There is no definite release date yet, sorry.

[jsf]

We've reconsidered the problem as follows.

Like a physical tape library, Firestreamer is not supposed to deduplicate data written to tapes as that defeats the purpose of tape backups. For example, if Firestreamer or the underlying storage deduplicated two similar tapes into a single copy of data on a physical disk, it would mean that instead of two copies of data you had only one. If something happened to that single copy, you would lose your data despite mistakenly believing that you had two independent tapes. In case of tape backups, deduplication should be done by the backup software according to the retention policy and the above considerations.

We still plan to release a new version of Firestreamer which will hopefully enhance deduplication rates for virtual tapes. However, we no longer consider deduplication a must-have feature, hence the delay. I apologize for the inconvenience.