Capacity Optimization for Hitachi File and Content Services
by Michael Hay on Jul 12, 2009
Okay so there have been discussions on HDP, RAID, HUR, etc. already with respect to how Hitachi optimizes capacity on our platforms. However what about file? Well I’ve on many occasions I’ve discussed capacity optimization technologies in the entire Hitachi File and Content Services portfolio. For the sake of a rather short post, I’ll be using a bullet style with some short definitions for each capability.
- Index stubbing – Application(s): HDDS, Description: patent pending approach to ensuring that the data placed in the full content index is minimized across federated indexes
- Legal/Compliance safe Single Instancing – Application(s): HCAP, Description: two phase process using high order cryptographic hashes (note up to SHA-512 can be supported) to first identify single instancing candidates and then a second phase of a binary comparison is done and only if both phases match is the object single instanced. This approach is hash collision proof.
- Compression – Application(s): HCAP, Description: checks each object in the archive that’s eligible for compression. If the object isn’t already compressed, it compresses it. If compressing the object doesn’t reduce its size (for example, because it’s already in a compressed format), the compression policy marks it as incompressible and doesn’t try to compress it again in future runs. If an object that was not eligible for compression becomes eligible, the compression policy compresses it on its next run. Similarly, if a compressed object loses its eligibility for compression, the compression policy decompresses it on its next run.
- Object Level Migration – Application(s): HNAS and HDD-MS, Description: movement of objects from one system to another to reduce capacity on a primary storage device or repository. Has the benefit of also reducing backup windows, and when combined with other products’ capacity optimization features (HCAP for example) an associated capacity reduction is provided. HDD-MS migrates objects from Sharepoint to HCAP and HNAS migrates content from HNAS to HCAP, HENP, NetApp, Ocarina, etc.
- WAN Capacity Optimized Replication – Application(s): HNAS, HENP, HCAP, Description: ensures that various strategies are employed to minimize the impact to a wide area network for replication tasks. Either sends block level differences of the file objects (HNAS IBR, HENP RUS) or sends only changed objects (HCAP Replication, HNAS IDR) and may include compression over the TCP/IP connection (HCAP Replication).
- HDP friendly NAS Systems – Application(s): HENP, HNAS, Description: a NAS device that supports a file system which maximizes capacity savings when used with block storage thinly provisioned LUNs (such as HDP on USP-V/VM and AMS).
- Low Overhead NAS Systems – Application(s): HNAS, Description: a NAS device that supports a file system which minimizes capacity overhead when additional features are employed or under general usage. For example HNAS has a fixed file system overhead of 2GB while NetApp has a flexible file system overhead that can be as high as 38%, that is in some configurations the AWFL err WAFL file system usable capacity can be as low as 62%. The remaining 38% is used for housekeeping, metadata, snapshots, core dumps, etc.
So as you can see Mr./Ms. Reader, Hitachi has a lot to offer in this space of capacity optimization. Specifically, due to things like capacity efficient file systems, we are allowing our customers to use more storage out of the box instead of requiring something like ASIS just to get as close as possible back to zero — hey let’s not forget NetApp tried to buy Data Domain which is a repudiation of their ASIS strategy in my opinion. When you look at the body of the capacity efficiencies Hitachi can offer we’ve been doing a lot here for a long time. I think we have figured it was about time that we got the word out again.
P.S. On the flip side of capacity reductions is Controlled Instancing that my colleague Ken Wood will soon be penning a post or two about.
P.P.S. Incidentally the image is a “compressed Ferrari” which I thought people would get kick out of. You can find the originating site of the compressed car here. Also the included screenshot from HCAP is from a live system with SiS going strong.