Migrating to SAN Environment Needs Defragmentation

SANs typically employ a clustered/SAN file system to pool disk arrays into a virtualized storage volume. This is not NTFS, but rather proprietary software, provided by a SAN hardware or software vendor such as EMC (Celerra), NetApp, LSI (StoreAge SVM), VMware (VMFS), etc. VMFS, for example, uses between 1MB to 8MB storage blocks. This file system essentially ‘runs on top of NTFS’, it does not replace it. Keeping in mind that every file system is a ‘virtual’ disk, Stacking one virtual component over another (i.e. one file system on top of another) is very doable and increasingly more common.

What the vendor of a SAN file system does at ‘that’ file system is irrelevant to what a decent third party defragmenter does. Claims that "you do not need to defragment" may be misunderstood and incorrectly implied to mean ‘NTFS’. It is very possible that you do not need to defragment the ‘SAN file system’. The experts on that file system and the source from which you should get setup tips, best practices, and SAN I/O optimization methodologies is that manufacturer.

As for NTFS, it still fragments and causes the Windows OS to ‘split’ I/O requests for files sent into the SAN, creating a performance penalty. You can measure this using Window’s built-in PerfMon tool and watch the split I/O counter. You can also use the Average Queued Disk I/O, given you account for the number of physical spindles. Given that SANs are ONLY ever block-level storage, they do NOT know what I/Os relate to what files. Therefore they cannot intelligently spread the fragments of a file across multiple disks. A whole mass of separate I/Os writes/reads for fragmented files (which will most certainly be interspersed with other simultaneous data writes/reads) will be non-optimally spread across the disks in the SAN storage pool (i.e. write more fragments of a given file to one disk rather than evenly spreading the data across all the disks).

SAN file system vendors may offer optimization strategies to, over time, move data around the disks as it learns typical data requests (such as from that fragmented file incorrectly laid out on the disks) are not properly load-balanced across SAN spindles. Generally speaking, the above holds true for disk striping as well (RAID). SAN designers or developers agree that NTFS fragmentation IS an issue and advanced defragmentation is important (‘basic’ defragmenters can actually cause problems).

Also when the SAN starts getting performance hits caused by excessive I/O’s resulting due to the fragmentation within NTFS, a common way in which companies improve SAN performance, particularly I/O performance, is to simply add more disks. In general we are masking the problem by adding more disks/spindles to spread the excessive I/O’s, but the actual problem still remains (excessive I/O’s would still be generated as the NTFS still fragments) and you will end up purchasing additional hardware. While SAN manufacturers would certainly not discourage this, it might not be the optimal solution. Handling the fragmentation within the NTFS is the only way to get rid of the actual problem.

Diskeeper data performance software is an automatic defragmenter and is ‘proactive’ on the basis that it prevents fragmentation from occurring before it happens. Diskeeper prevents up to 85% fragmentation before it occurs, which leaves little fragmentation to handle afterwards.

Windows IT Pro analyst David Chernicoff published a white paper that covers the issues file fragmentation causes on SAN storage, entitled Maximize the Performance of your Windows SAN Infrastructure.