Spring World 2018

Conference & Exhibit

Attend The #1 BC/DR Event!

Winter Journal

Volume 30, Issue 4

Full Contents Now Available!

Tuesday, 30 June 2015 00:00

Do SANs Need Defragmentation?

Written by  Brian Morin

I was recently forwarded an e-mail from an IT director who asked, “There is such a quandary about disk defragmentation in the virtualization world. One says defrag and another says never. Who is right?”

Before I turn this into the end-all, be-all article on SAN fragmentation – and explain how the real problem isn’t what most people suppose – let’s first address why there’s fear, uncertainty, and doubt; why there are varying opinions about fragmentation and SAN in the IT community.

There are virtual machine performance best practice guides that cite increasing virtual machine memory as the No. 1 recommendation to improve performance, while the No. 2 recommendation is to defragment the file system on all guests. Moreover, one of the leading storage analyst firms just released a best practice video on steps to increase MS-SQL performance before simply adding more flash, and its No. 1 recommendation was to first solve performance robbing fragmentation.

This is where it gets confusing.

When you move to the other side of the aisle, storage vendors are replete with recommendations to NOT defragment, saying that fragmentation is not an issue on their systems.

So who’s right?

The answer is that they are both right, but neither are sharing the full picture.

Yes, fragmentation robs SAN storage performance and wastes much of any new investment in flash or spindles to mask the problem. And, yes, SAN systems do a good job of combatting fragmentation at the physical layer, and a traditional defragmentation process will do more harm than good on a production SAN.

So if modern SAN storage systems are doing a good job of mitigating the ill-effects of fragmentation at the physical layer, then what’s the problem?

The problem has to do with fragmentation within the Windows file system that exists outside the SAN storage device where the SAN has no control. Remember what is unique about Windows in a SAN environment – it is extracted from the physical layer. How blocks are physically managed is the SAN’s job. Windows doesn’t see the SAN system at all nor does it have any idea as to the kind of storage media being used. The OS references the logical disk layer for any LUN or volume that presents itself. As a result, Windows controls the logical disk and the SAN merely receives the data then manages how the blocks are physically stored.

Well, here’s what even some of the most deep-in-the-weeds storage techies and engineers don’t fully understand – a fragmented logical disk. The reason this goes unnoticed by the storage vendors is because the problem exists outside the world they live in. Fragmentation is inherent to the fabric of Windows. Since Windows takes a one-size-fits-all approach, Windows will split a file into multiple pieces before writing each piece to a different address within the logical disk software layer. The SAN has no control over this behavior. Since each piece of a file that has its own dedicated address at the logical disk layer, that means every piece of a file needs its own dedicated I/O operation to process as a read or write.

Therefore, if Windows sees a file existing as 20 separate pieces at the logical disk layer, it will execute 20 separate I/O commands to process the whole file as a read or write. It doesn’t matter how well a SAN is organizing data on the backend or even if it appears fragmentation free at the physical layer if it has to execute 20 separate I/O operations to process any unit of data.

Morin-photo

The fragmentation problem in a SAN environment is not a physical disk platter problem at all, but rather an I/O overhead problem from the file system that generates a lot of unnecessarily small, fractured I/O which steals throughput, forces systems to work much harder than necessary to process any given workload, and makes organizations much more IOPS dependent than they need to be. This I/O overhead issue is irrespective of storage media as it dampens SAN SSD performance equally as much as it would impact mechanical disks within the SAN.

If only Windows was aware of file sizes and could choose proper file allocations from the beginning, a fragmented logical disk could be eliminated. This would enable Windows to see files existing in a more contiguous or sequential fashion and merely require just one or two I/O operations to process the whole file.

Some might say the way to clean up this problem is by running a traditional defragmentation process every now and then to restore a healthy relationship between I/O and data. However, as mentioned, running a traditional defrag process on a production SAN is worse than the problem itself since defragmentation will result in change block activity at the physical layer that will skew thin provisioning, trigger features like replication, and disrupt how the SAN has chosen to manage its own blocks. This is why the SAN storage vendors recommend against defragmentation. And this is also why some administrators employ the laborious process of migrating data off a volume, taking it offline, and run defragmentation before bringing it back.

It’s for this very reason there are alternatives to address this issue regarding inline fragmentation prevention technology that address this from a software I/O reduction approach to increase performance so systems can reclaim throughput and process more data in less time with increased I/O density.

Most admins are simply unaware of how much performance has actually degraded due to I/O inefficiency from a fragmented logical disk. My experience finds that the typical customer experiences performance dampening in the range of 25 percent on their most I/O intensive applications. In more severe cases, it’s much more than that when single files have been fragmented into hundreds or even thousands of pieces within the logical disk. In fact, for some organizations, it’s not just an issue of sluggish performance but reliability as they have to regularly reboot servers and have issues with certain data sets.

Virtualized customers should also be aware of the additional performance penalty from the “I/O blender” effect which exacerbates performance problems even further as I/O is not just unnecessarily small and fractured, but also mixed and randomized with other VM workloads at the point of the hypervisor. By preventing fragmentation, the I/O requirement for any given workload is reduced which further reduces the amount of I/O per GB that gets randomized at the point of the hypervisor.

As you can see, fragmentation in a SAN environment is a real problem and it’s just not a physical disk platter problem on the SAN itself. And just when administrators thought they had little to no viable options other than throwing more hardware at the problem there are I/O optimization software alternatives that are easier and more economical path to improved performance while protecting existing CapEx investments.

Morin-BrianBrian Morin is senior vice president of global marketing for Condusiv Technologies, responsible for the corporate marketing vision by driving demand and awareness worldwide. Efforts over the last year led to growing adoption of V-locity®, which has quickly amassed more than 1,000 new customers looking to accelerate their virtual environment with a 100 percent software approach. Prior to Condusiv, Morin served in leadership positions at Nexsan that touched all aspects of marketing, from communications to demand generation, as well as product marketing and go-to-market strategies with the channel. Morin notably steered rebranding efforts and built the demand generation model from scratch as an early marketing automation adopter. Growth led to the successful acquisition by Imation. With more than 15 years of marketing expertise, Morin has spent recent years on the forefront of revenue marketing models that leverage automation for data-driven determinations. His earlier background has roots on the agency side as creative director, helping companies build brands and transition to online engagement.