Saturday, June 6, 2009

What Is Our Current Data Storage Capacity?

Storage capacity, current data volume, and rate of growth are the three minimum pieces of information required to determine when more storage capacity is needed. Organizations should keep close tabs on these three metrics. Many people speak of storage capacity in terms of how much disk drive capacity they have. Hard disks are not the only media data is stored on. Hard drives are commonly used for primary online storage and tape or other removable media (e.g. optical disk) for backup. Industry offers compelling technology and solutions to store data online or near-line on tape and optical based media but hard drives are most prevalent. When conducting enterprise capacity planning, planners should examine all tiers of storage. Ultimately, data should be categorized by business value and stored in a tier of storage that provides the appropriate level of performance and protection.

Considering that data storage costs comprise as much as 15 percent of IT operational spending and up to 20 percent of IT capital spending[i], storage capacity and data volumes are worthy of management attention. Enterprise IT usage policies and storage quotas are key controls for managing the growth of data. Strategic planning and development of an enterprise data life cycle policy and associated procedures can help organizations eliminate unneeded data and reclaim data storage space.

Management of storage capacity requires the ability to routinely monitor it. Manual monitoring of storage capacity is impractical in large environments. A storage resource management tool can provide automated monitoring and control of many storage resources. For more information on storage resource management tools, see my paper “How Much Data Do We Have?”

When examining hard drive media capacity, IT management needs to understand there is a significant difference between raw capacity and usable capacity. This is an extremely important fact to keep in mind when dealing with storage equipment vendor sales personnel. The Storage Networking Industry Association (SNIA) defines usable capacity of a disk as “…the total formatted capacity of the disk.” Formatted capacity does not include raw capacity reserved for metadata, disk size equalization, or check data. Usable capacity is the number to focus on when shopping for additional disk based storage.

The amount of usable capacity available from a given raw capacity varies depending upon how the disk or array of disks is configured. RAID (Redundant Array of Independent Disks) configuration has a significant impact on the ratio of raw space to usable capacity. For example an array of disks configured for RAID 1, in which all data is mirrored, will use two units of raw capacity for every unit of usable storage. Therefore an array with 50 TBs of raw capacity configured for RAID 1 will yield less approximately 25 TBs of usable capacity.

To read my entire white paper on this topic, go to http://www.storagestrategies.com/data_storage_strategies_whitepapershome.html

[i] Corporate Executive Board

Tuesday, March 24, 2009

John Moore, freelance writer for Federal Computer Weekly (FCW), quoted me in his article, "New Tech Trends Force Government to Rethink Storage Strategies."

http://fcw.com/Articles/2009/03/09/Reinventing-storage.aspx

Thursday, January 22, 2009

How Fast Is Our Data Volume Growing?

While there is certainty that more and more data will be stored over time, the question is how much data and how fast? IDC reports that enterprise data stores will grow an average of 60 percent annually. This number will fluctuate depending on the enterprise. Using a rough guess for annual capacity planning may be effective but is also very wasteful. IDC also reports that across the industry disk utilization rates range from 28-35%. In order to avoid wasting up to 65% of corporate investment on data storage capacity, some detailed trend analysis is required.

  • It is very important to understand the growth rate by type of data. Examination of volume growth trends by type can help storage managers identify possible system problems and anomalous user behavior.
  • Data storage managers should collect data and regularly report to management on changes in the volume of data stored
  • Data storage managers should keep a weather eye on opportunities to reduce the data volume through elimination of unnecessary or duplicate data and archiving.
  • IT management can help abate the unstrained growth of data through user education and policy.

Despite the fact that the relative cost of computer data storage media per unit volume has fallen 63 percent since 1998[i], the overwhelming data volume growth is causing storage costs to grow rapidly. Industry has responded to the market’s need for more intelligent storage of data. A tiered storage model and data deplication solutions have entered the main stream and can help economize on data storage investments.

To read my entire white paper on this topic, go to www.storagestrategies.com


[i] Bureau of Labor and Statistics Producer Price Index for computer storage media. http://data.bls.gov

Friday, January 2, 2009

How Much Data Do We Have?

Capturing the total amount of data stored can be challenging depending on what applications are in use and how users behave. Centralized application data and corporately hosted personal or shared directory data are generally easy to locate. However, if users are permitted to, or are in the habit of storing data locally, identifying and accounting for all user data can be tremendously challenging.

In order for management to answer this question, it will need to employ some sort of monitoring tool to detect and report on data stored on all disk arrays, servers, and workstations. A number of tools that can collect and report this data exist. A snap shot of meta data from all existing files provides a rudimentary core of information to analyze. However, in order to effectively manage data storage over time regular detailed reports are necessary.

To read my white paper, "How Much Data Do We Have?", which includes a list of leading SRM solutions, go to our web site at: www.storagestrategies.com