What Is Our Current Data Storage Capacity?

Storage capacity, current data volume, and rate of growth are the three minimum pieces of information required to determine when more storage capacity is needed. Organizations should keep close tabs on these three metrics. Many people speak of storage capacity in terms of how much disk drive capacity they have. Hard disks are not the only media data is stored on. Hard drives are commonly used for primary online storage and tape or other removable media (e.g. optical disk) for backup. Industry offers compelling technology and solutions to store data online or near-line on tape and optical based media but hard drives are most prevalent. When conducting enterprise capacity planning, planners should examine all tiers of storage. Ultimately, data should be categorized by business value and stored in a tier of storage that provides the appropriate level of performance and protection.

Considering that data storage costs comprise as much as 15 percent of IT operational spending and up to 20 percent of IT capital spending[i], storage capacity and data volumes are worthy of management attention. Enterprise IT usage policies and storage quotas are key controls for managing the growth of data. Strategic planning and development of an enterprise data life cycle policy and associated procedures can help organizations eliminate unneeded data and reclaim data storage space.

Management of storage capacity requires the ability to routinely monitor it. Manual monitoring of storage capacity is impractical in large environments. A storage resource management tool can provide automated monitoring and control of many storage resources. For more information on storage resource management tools, see my paper “How Much Data Do We Have?”

When examining hard drive media capacity, IT management needs to understand there is a significant difference between raw capacity and usable capacity. This is an extremely important fact to keep in mind when dealing with storage equipment vendor sales personnel. The Storage Networking Industry Association (SNIA) defines usable capacity of a disk as “…the total formatted capacity of the disk.” Formatted capacity does not include raw capacity reserved for metadata, disk size equalization, or check data. Usable capacity is the number to focus on when shopping for additional disk based storage.

The amount of usable capacity available from a given raw capacity varies depending upon how the disk or array of disks is configured. RAID (Redundant Array of Independent Disks) configuration has a significant impact on the ratio of raw space to usable capacity. For example an array of disks configured for RAID 1, in which all data is mirrored, will use two units of raw capacity for every unit of usable storage. Therefore an array with 50 TBs of raw capacity configured for RAID 1 will yield less approximately 25 TBs of usable capacity.

To read my entire white paper on this topic, go to http://www.storagestrategies.com/data_storage_strategies_whitepapershome.html

[i] Corporate Executive Board

John Moore, freelance writer for Federal Computer Weekly (FCW), quoted me in his article, "New Tech Trends Force Government to Rethink Storage Strategies."

http://fcw.com/Articles/2009/03/09/Reinventing-storage.aspx

How Fast Is Our Data Volume Growing?

While there is certainty that more and more data will be stored over time, the question is how much data and how fast? IDC reports that enterprise data stores will grow an average of 60 percent annually. This number will fluctuate depending on the enterprise. Using a rough guess for annual capacity planning may be effective but is also very wasteful. IDC also reports that across the industry disk utilization rates range from 28-35%. In order to avoid wasting up to 65% of corporate investment on data storage capacity, some detailed trend analysis is required.

• It is very important to understand the growth rate by type of data. Examination of volume growth trends by type can help storage managers identify possible system problems and anomalous user behavior.
• Data storage managers should collect data and regularly report to management on changes in the volume of data stored
• Data storage managers should keep a weather eye on opportunities to reduce the data volume through elimination of unnecessary or duplicate data and archiving.
• IT management can help abate the unstrained growth of data through user education and policy.

Despite the fact that the relative cost of computer data storage media per unit volume has fallen 63 percent since 1998[i], the overwhelming data volume growth is causing storage costs to grow rapidly. Industry has responded to the market’s need for more intelligent storage of data. A tiered storage model and data deplication solutions have entered the main stream and can help economize on data storage investments.

To read my entire white paper on this topic, go to www.storagestrategies.com

[i] Bureau of Labor and Statistics Producer Price Index for computer storage media. http://data.bls.gov

How Much Data Do We Have?

Capturing the total amount of data stored can be challenging depending on what applications are in use and how users behave. Centralized application data and corporately hosted personal or shared directory data are generally easy to locate. However, if users are permitted to, or are in the habit of storing data locally, identifying and accounting for all user data can be tremendously challenging.

In order for management to answer this question, it will need to employ some sort of monitoring tool to detect and report on data stored on all disk arrays, servers, and workstations. A number of tools that can collect and report this data exist. A snap shot of meta data from all existing files provides a rudimentary core of information to analyze. However, in order to effectively manage data storage over time regular detailed reports are necessary.

To read my white paper, "How Much Data Do We Have?", which includes a list of leading SRM solutions, go to our web site at: www.storagestrategies.com

Interview with Dave Wennergren, Deputy CIO of DoD

I had the good fortune to meet with Dave Wennergren, the Deputy CIO of the Department of Defense recently. Dave also serves as the Vice Chair of the Federal CIO Council. I met him when he was working as Deputy CIO for the Department of the Navy, when I was working on the Navy Marine Corps Intranet (NMCI) program. Dave is an energetic and enthusiastic person the DoD is lucky to have as a career IT executive. He’s a strong advocate of government adopting industry best practices and leveraging new capabilities to improve mission effectiveness. His long held reputation as an agent of change will serve him well with the new administration. Here are a few pearls of wisdom and insight from Dave:

• “The change pace must be dramatically faster or we will risk becoming dramatically irrelevant.”
• He’s a fan of managed services and would like to see the government embrace more of it.
• Information sharing is a must. We can’t build walls. We have to figure out to how to share securely in a Web 2.0 environment.
• He’s an advocate of measuring the effectiveness of information sharing and solutions that include measures to gauge effectiveness.
  

It’s refreshing to speak with an IT executive eager to leverage new capabilities to improve mission effectiveness but pragmatic about measuring that progress in an objective fashion. It was clear Dave has adopted several tools Karen Evans successfully employed as the OMB CIO. We’re fortunate to have him as a source of continuity as the administration changes.

Interview with Karen Evans, Outgoing CIO of OMB

I recently had the pleasure of interviewing Ms. Karen Evans the CIO of the Office of Management and Budget (OMB). For those a little rusty on your Civics, OMB supports execution of the President’s management agenda and oversees the departments and agencies in the Executive branch of the Federal Government. Big job. Ms. Evans has been responsible for implementing a management framework to manage IT as a portfolio and strategic investment. She was quick to point out that although IT is a strategic asset, it is also just an enabler of the mission. Her comments were very candid and caveated by the fact that she’s got 30 days left in her job. She was unwilling to comment on any forward looking projections regarding the next Administration but offered that the Transition Team seemed to respect the basic management framework OMB has used to review IT strategic plans, progress toward those plans, and portfolio investments.

Parting shots she had to offer:

• The Government needs to continue to work on structuring contracts to ensure vendors are incented to behave in ways that benefit the customer.
• Industry should look for niches where there’s a need and the Government’s struggling on developing a solution, develop a solution, and relieve the Government of the burden of development.
• Industry must demonstrate the ability to deliver solutions and services while meeting Government requirements for transparency (FOIA), data quality, security, and privacy.
• The most significant challenge going forward is managing risk while delivering on mission and ensuring the above requirements are satisfied.

I have to admit, going in; I was expecting to hear a little more talk about IT from U.S. Government’s senior CIO. But, in retrospect, it makes sense the person responsible for overseeing the planning and execution of over $60B of Federal spending on IT would be more focused on wise management of those resources rather than the sizzle.

Welcome to the Storage Strategies blog. This blog is intended to help IT leaders learn about important topics regarding data storage management they need to understand to successfully manage their organization’s information assets. Everyone knows that data volumes are growing rapidly. IDC reports that enterprise data stores will grow an average of 60 percent annually. Considering the inevitable growth of information assets and the probable reduction of IT capital budgets over the next year (resultant from the global economic crisis), IT managers must actively scrutinize both the data their users are creating and the resources they plan to spend on hosting and protecting that data.

To start off here are ten questions regarding data management every IT manager must be able to address to be successful. Can you answer these questions?

1. How much data do we have?
2. How fast is our data volume growing?
3. What is our current data storage capacity?
4. What are our most critical applications/data?
5. Is our data backed up regularly and are data restoration processes and procedures tested regularly?
6. Are data backups and data archives stored at a secure off site location?
7. How long will it take to restore access to data if our primary data stores are corrupted or destroyed, and how much data are we willing to lose to recover?
8. Do we have a documented data retention policy and do local operating procedures ensure it is implemented?
9. Do we have policies and procedures for provisioning additional data storage to users?
10. Are our data storage personnel appropriately trained and certified?
    

If you can’t answer these questions because you don’t know or can’t give an affirmative answer, subscribe this blog and stay tuned for more.

To read my white paper, “Ten Things IT Management Needs to Know About Their Data Storage”, go to this link to down load it: www.storagestrategies.com