As drive sizes in storage arrays increase, should we keep putting eggs into larger baskets?

Hard drive sizes have increased again.  It is just part of the evolution of technology.  When it comes to using them in storage arrays however, bigger is not always better – or at least may not be the best course of action.

We now have the ability to put 900GB SAS and 3TB Nearline SAS drives into our storage designs.  How does that change things?  There are definitely some positives.  Each time drive sizes move up, their individual performance and reliability hold steady or move slightly upward – so over time the traditional hard drive as a whole continues to improve.  It also makes it easier to put more useable TB into a smaller sized array.   It pushes the cost/GB down.

On the other side of the fence, larger hard drives allow for some potential pitfalls:

• They make it easier to short cut on IOPS - What used to be a shelf of 300’s can now be stored on just five 900’s.   In some cases this is a good thing, but the more important metric as it relates to storage design is that it provides one-third of the IOPS for the same amount of space.  That significantly increases your margin of error if you don’t correctly calculate the workload going on to those drives.
• Rebuild times get longer – No one really expects a drive to fail, but it happens.  With RAID-5 stripes being the norm for most of our customers, once you lose a drive your performance is noticeably impacted until your hot spare is rebuilt.  The larger the drive, the longer it takes to rebuild, the longer performance is impacted, and most importantly – the longer your data is at risk to another drive failure.

So how do we find the right balance between taking advantage of the new found capacity while keeping our performance and integrity in check?  Here are a couple of suggestions.

• Use 3TB drives only in archive use cases – these drives are like the online version of an oil tanker.  They can hold a lot, but if things ever go bad it will be a guaranteed disaster.  So steer clear of using them for production data; though they will be a cost effective choice as an archive tier for older data that has to stay online.  Use RAID 6 to give yourself that extra parity drive and level of protection.
• Use the 900GB SAS drives (vs. smaller SAS drives) where you have a lower IOPS to capacity requirement – maybe smaller virtualized shops.  This size drive may also play well in DR environments where you want to have your replicated data ready to roll, but cost is a factor.  Loading up your DR array with SATA/NL-SAS to save costs is tempting, but when the day comes when you need to go live with your DR site - this drive will handle multiple workloads better.
• Use Flash – Extending your system cache with Flash drives will allow you to better handle the performance spikes when using larger/slower drives.  Not all I/O is cache friendly but more often than not, extended cache makes a positive impact across the entire array and reduces I/O load on the spindles by handling repetitive requests.  It can be selectively turned off for LUNs that do not benefit from it.

Overall, a solid storage design will be based on a combination of real I/O data, vendor best practices, and common sense.  Use the new drives if things line up, but don’t be afraid to be conservative if your gut tells you to keep your eggs in slightly smaller baskets.