Automating the Data Store

Smart software can automatically allocate data to the most cost-effective form of storage. But will it work for you?

Computerworld |

Michael Glenn was wasting storage, and he knew it.

A document-scanning project had created a single, 1.4TB LUN of old court records. Glenn, senior IT manager for the Ohio Court of Claims, knew that only 6% of the files had been accessed in the past year and that the rest shouldn't be on expensive Fibre Channel disk.

His challenge was determining which 94% he could move at any given time to slower, less expensive Serial ATA drives. It turned out he already had the software he needed -- Dynamic Storage Technology (DST), part of his Novell Open Enterprise Server 2 implementation -- to create and automatically execute file movement policies based on when files were last accessed.

After spending a week tweaking the configurations last spring, Glenn says, "I just let it alone. It's been working great," freeing up at least a dozen Fibre Channel drives. By reducing the number of active files, he also cut his daily backup time from 14 hours to 47 minutes.

Installation was simple, and configuration required just migrating the old LUN to the SATA drives, renaming that LUN, creating a smaller LUN on Fibre Channel to replace it, and designating the new LUN as the primary volume and the old LUN as the shadow. "Then I started setting up the migration rules," Glenn says. There was no extra cost for DST, he adds, but he estimates he saved about $140,000 through reduced demand for disk drives and power.

Glenn is one of the early beneficiaries of a new technology called automated data tiering, which automates not just the movement of data, but also the task of monitoring how data is being used and determining which data should be on which type of storage. Such automated tiering isn't yet in the mainstream because few vendors offer the technology and it hasn't been proved to work in very high-end, transaction-intensive environments. Also, it's typically used only within a single vendor's arrays or file system or supports only a limited number of storage protocols or topologies. But for organizations with simpler needs, the automated tiering tools available today are more than good enough.

How Tiering Became Automated

"Tiering" means moving data among various types of storage media as demand for it rises or falls. Moving older or less frequently accessed data to slower, less expensive storage such as SATA drives or even tape can reduce hardware costs, while putting the most frequently accessed or most important data on faster, more expensive Fibre Channel drives or even solid-state drives (SSD) boosts performance. Finally, automating the entire process prevents it from getting bogged down in the data classification and policy-setting that hampered earlier "tiering" efforts such as information life-cycle management (ILM).

Storage administrators have long been able to move data between tiers, but they had to manually initiate the process, or at least classify their data and create tiering policies upfront. While some policy creation is still required, the latest crop of automation products is designed to reduce or eliminate the need for staffers to monitor storage systems and find the specific files, volumes or blocks that need retiering and manually move them.

IT managers must first look at which criteria the software can consider (such as how often data is accessed) and whether it can evaluate and move individual blocks or files rather than just larger volumes or LUNs. Since as little as 10% of the blocks in a volume may be active enough to justify a move to faster, more expensive storage, you'll save money if you can move just those, especially if you're moving to expensive SSDs.

Other factors to consider include how quickly the software can detect and react to changes in data usage, and whether administrators can override the automated tiering if it interferes with application performance. Administrators can also use it to predict when certain data (such as accounting files for the quarterly close) will be needed, so the tiering software can update it ahead of time. Finally, administrators need to decide how comfortable they are ceding control to an automated tool.