Object-based storage devices, or object storage for short, have been around for a number of years. But it’s become a hot topic with the near-endless capacity needs of cloud storage and the “big data” requirement for increasingly large, shared storage infrastructures that can be accessed and searched like a single system.
As the need for more shared storage capacity has grown, the limitations of file systems have become a problem. The hierarchical nature of file systems creates a large amount of data about the files themselves and the way they’re accessed. This data about data, or “metadata,” is required to search for and manipulate files in the file system. As file systems grow, the amount of metadata and the number of metadata operations required also grows, eventually becoming a limitation on file system size and a performance bottleneck. And, traditional file systems require a centralized metadata controller, which doesn’t support the distribution of data storage, fast becoming a requirement for cloud implementations.
An object-based storage device seeks to address the limitations of large file storage systems by manipulating data in “containers” called “objects,” each with metadata defining the container’s data and an identifier called an “object ID.” Replacing the traditional hierarchical, tree file structure with a flat index of object ID numbers greatly reduces the metadata generated as the storage system grows and allows this metadata store to be distributed. The result is an extensible, highly scalable storage architecture that fits big data and cloud applications very well.
There are a number of companies that have object storage offerings, both hardware- and software-based. Data Direct Networks is one we wrote about recently, as are HDS, Dell (OEM is Caringo) and Amplidata, to name a few. For VARs this is the time to get familiar with object storage technologies, if you’ve not already done so. Most primary enterprise vendors will have object-based solutions, and many newer offerings are available that can give you a great introduction into accounts where you’re not currently getting the disk business.
Follow me on Twitter: EricSSwiss