Automated optimization of disks
As files are created, modified, and moved, fragmentation can occur both within the volume's allocation table and on the volume itself. This happens because files are written to clusters on disk as they are used. The file system uses the first clusters available when writing new data, so as you modify files, different parts of files can end up in different areas of the disk. If you delete a file, an area of the disk is made available, but it might not be big enough to store the next file that is created and, as a result, part of a new file might be written to this newly freed area and part of it might be written somewhere else on the disk.
Although the file system doesn't care whether the file data is on contiguous clusters or spread out across the disk, the fact that data is in different areas of the disk can slow down read/ write operations. This means it will take longer than usual to open and save files. It also makes recovering files more difficult in case of serious disk error. Windows Server 2012 R2 provides the Optimize Drives utility, a tool for defragmenting volumes. Unlike Check Disk, which cannot check and repair the operating system volume while it is in use, Optimize Drives can, in most cases, perform online defragmentation of any volume, including the operating system volume.
Preventing fragmentation of disks
Windows Server 2012 R2 analyzes fragmentation and optimizes volumes as part of automated maintenance. Windows does this by using the Optimize Drives utility. Although automated maintenance triggers the disk analysis, the process of calling and managing Optimize Drives is handled by a separate task. In Task Scheduler, you find the ScheduledDefrag task in the scheduler library under Microsoft\Windows\Defrag, and you can get detailed run information on the task's History tab.
Automatic analysis and optimization of disks can occur while the disks are online so long as the computer is on AC power and the operating system is running but otherwise idle. By default, disk optimization is a weekly task rather than a daily task-and there's a good reason for this. Normally, you need to optimize a server's disks only periodically, and optimization once a week is sufficient in most cases. That said, the more frequently data is updated on drives, the more often disks should be optimized.
Windows automatically performs cyclic pickup defragmentation. With this feature, when a scheduled defragmentation pass is stopped and rerun, the computer automatically picks up the next unfinished volume in line to be defragmented. Although nonsystem disks can be rapidly analyzed and optimized, it can take significantly longer to optimize system disks online.
You can control the approximate start time for the analysis and optimization of disks by changing the automated maintenance start time. Windows Server also notifies you if three consecutive runs are missed. All internal drives and certain external drives are optimized automatically as part of the regular schedule, as are new drives you connect to the server.
With manual optimization, Optimize Drives performs an online analysis of volumes and then reports the percentage of fragmentation. If defragmentation is needed, you can then elect to perform online defragmentation. System and boot volumes can be defragmented online as well, and Optimize Drives can be used with FAT, FAT32, exFAT, NTFS, and ReFS volumes.
You can configure and manage automated defragmentation by following these steps:
- In Computer Management, select the Storage node and then the Disk Management node. Press and hold or right-click a drive and then tap or click Properties.
- On the Tools tab, tap or click Optimize. In the Optimize Drives dialog box, note the last run-time and status of each volume. The status shows the percentage of fragmentation. A volume that needs optimization is listed as Needs Optimization. Otherwise, the volume status is listed as OK.
- Under Scheduled Optimization, note the scheduled optimization settings, which indicate whether automated optimization is enabled and provide the run frequency. If you want to change how optimization works, tap or click Change Settings. This opens the dialog box shown. To cancel automated defragmentation, clear the Run On A Schedule check box. To enable automated defragmentation, select Run On A Schedule.
- The default run frequency is set as shown. In the Frequency list, you can choose Daily, Weekly, or Monthly as the run schedule. If you don't want to be notified about missed runs, clear the Notify Me check box.
- If you want to manage which disks are defragmented, tap or click Choose and then select the volumes to defragment. By default, all disks installed within or connected to the computer are defragmented, and any new disks are defragmented automatically as well. Select the check boxes for disks that should be defragmented automatically and clear the check boxes for disks that should not be defragmented automatically. Tap or click OK to save your settings.
- Tap or click OK and then tap or click Close.
Fixing fragmentation by using Optimize Drives
By using Optimize Drives, you can check for and correct volume fragmentation problems on FAT, FAT32, and NTFS volumes. The areas checked for fragmentation include the volume, files, folders, page file (if one exists on the volume), and the MFT. You also can defragment volumes with cluster sizes greater than 4 KBs.
You can run the graphical version of Optimize Drives by using either File Explorer or Computer Management. In File Explorer, press and hold or right-click the volume and choose Properties. On the Tools tab of the Properties dialog box, tap or click Optimize to open the Optimize Drives dialog box. In Computer Management, select the Storage node and then the Disk Management node. Press and hold or right-click a drive and then select Properties. On the Tools tab, tap or click Optimize.
In the Optimize Drives dialog box, select a disk and then tap or click Analyze. Optimize Drives then analyzes the disk to determine whether it needs to be defragmented. If so, it recommends that you defragment at this point. If a disk needs to be defragmented, select the disk and then tap or click Optimize. Depending on the size of the disk, defragmentation can take several hours. You can tap or click Stop at any time to stop defragmentation.
NOTE:
Optimize Drives needs 10 to 15 percent free space to defragment a disk completely. Optimize Drives uses this space as a sorting area for file fragments. If a volume has less free space, Optimize Drives will only partially defragment it. By default, Optimize Drives performs partial defragmentation by attempting to consolidate only fragments smaller than 64 MBs.
Optimize Drives can also be run at the command line by using Defrag (Defrag.exe). You can run Optimize Drives in analysis mode at the command line by typing defrag /a followed by the drive designator. For example, if you want to analyze the fragmentation of the D drive, you type defrag /a d:. To analyze and then defragment a volume if defragmentation is necessary, type defrag followed by the drive designator, such as defrag d:. No parameters are necessary (because the /d parameter is implied, which performs a traditional defrag).
You defrag multiple drives by providing the designator of each drive you want to defrag, such as:
defrag /a c: d: g:
This specifies that you want to defrag the C, D, and G drives. By default, Defrag runs with low priority and defrags each volume in turn. Here, that would mean the utility would defrag the C drive, then the D drive, and finally the G drive.
Defrag has several syntaxes. The syntax for analyzing volumes without defragmentation is:
defrag volume(s) /a [/h] [/m] | [/u] [/v]
Here, when you use /h, /m, or both, you cannot use /u, /v, or both. The /h parameter enables you to run the defrag with normal priority, which gives the defrag task the same priority as most other processes and should speed up the defrag. The /m parameter enables Windows to defrag multiple volumes at the same time, in parallel rather than in a series. The /u parameter displays the defrag progress, and the /v parameter displays verbose output, which includes fragmentation statistics. Following this, you could defrag the C, D, and G drives in parallel at normal priority by typing:
defrag c: d: g: /a /h /m
Analyze (/a) is one of several independent tasks that you can perform. Other independent tasks you can perform include traditional defrag (/d), optimization (/o), and free-space consolidation (/x). The syntax for all three variations is the same except for the primary task being performed. As an example, if defrag specifies that drives need to be optimized to reduce fragmentation, you use /o to do this. The syntax for optimizing fragmented drives is:
defrag volume(s) /o [/h] [/m] | [/u] [/v]
The optimization process focuses on reduced file fragmentation. Over time, free space on a drive also can become fragmented. You can use the /x parameter to consolidate the freespace fragments, and the syntax is:
defrag volume(s) /x [/h] [/m] | [/u] [/v]
NOTE:
Thinly provisioned virtual disks (Storage Spaces, dynamic virtual hard disks [VHDs], storage area network [SAN] virtual disks) are the only types of disks whose free space is consolidated when optimized.
Rather than trying to remember all these syntaxes separately, I recommend focusing on the primary tasks you can perform: analysis (/a), traditional defrag (/d), optimization (/o), and free-space consolidation (/x). Then consider the additional options you might want to use, including either higher priority (/h) and multitasking (/m) or progress updates (/u) and verbose output (/v).
Optimizing virtual disks
You optimize the internal structures of virtual disks by using the standard optimization tasks, including analysis (/a), traditional defrag (/d), optimization (/o), and free-space consolidation (/x). That doesn't necessarily optimize the external structures from which virtual disks are constructed, however, and fragmentation in these external structures can affect virtual disk performance as much as fragmentation of internal structures.
The external structures from which virtual disks are constructed are referred to as slabs. A slab is simply an allocation unit or, put another way, a contiguous block of allocated space on the underlying physical disk. The performance of a virtual disk can be affected when there are too many small slabs, too many slabs spread out across the underlying physical disk, or both. You can attempt to consolidate slabs by using the /k parameter. The syntax is:
defrag volume(s) /k [/h] [/m] | [/u] [/v]
When you use the /k parameter, the /l parameter is implied. The /l parameter generates trim and unmap hints for sectors that were previously allocated to the virtual disk but the virtual disk no longer uses. The underlying physical disk uses the trim and unmap hints to recover unused space.
You can retrim a virtual disk without performing a slab consolidation. The syntax is:
defrag volume(s) /l [/h] [/m] | [/u] [/v]
That said, when you optimize standard virtual disks (.vhd), Defrag performs slab consolidation automatically as well but won't retrim automatically. When you optimize differential virtual disks or thinly provisioned virtual disks (Storage Spaces, dynamic VHDs, SAN virtual disks), Defrag doesn't perform slab consolidation automatically but does retrim automatically.
Table below summarizes available Defrag options and parameters and their uses. Note the /C and /E parameters. You use the /C parameter when you want to defrag all available volumes (and don't want to specify the volumes individually by drive designator). You use the /E parameter when you want to defrag all available volumes except those specified after the /E parameter. For example, if you wanted to defrag all available volumes except the D and G drives, you enter:
defrag /d /e d: g:
Command-line parameters for Defrag
Option/ Description
Parameter
Volume(s) Sets the volume or volumes to work with.
/A Performs an analysis only of the specified volume or volumes.
/C Used instead of a drive letter; tells Defrag to optimize all
disks.
/D Performs an analysis of the specified volume or volumes,
followed by optimization if required.
/E Tells Defrag to optimize all disks except those specified
after the parameter.
/G Optimizes storage tiers on the specified volumes.
/H Runs Defrag with higher priority, meaning normal priority
instead of low priority.
/K Performs slab consolidation and retrim of virtual disks.
This applies to virtual disks only.
/L Performs retrim of virtual disks. This applies to virtual
disks only.
/M Multitasks the optimization by running Defrag in parallel
on each volume specified.
/O Performs optimization of the specified volume or volumes if
fragmented.
/T Tracks in-progress tasks on the specified volume or volumes.
/U Provides progress updates on the screen.
/V Displays verbose output containing fragmentation statistics.
/X Performs free-space consolidation on the specified volume
or volumes.
Understanding the fragmentation analysis
You can perform fragmentation analysis at the command line by using the /a and /v parameters. The command-line report shows the summary of fragmentation. The summary looks like this:
Invoking analysis on (C:)... Post Defragmentation Report: Volume Information: Volume size = 183.99 GB Cluster size = 4 KB Used space = 146.14 GB Free space = 37.85 GB Fragmentation: Total fragmented space = 7% Average fragments per file = 1.09 Movable files and folders = 716750 Unmovable files and folders = 20 Files: Fragmented files = 8265 Total file fragments = 61643 Folders: Total folders = 32654 Fragmented folders = 2274 Total folder fragments = 9690 Free space: Free space count = 2470 Average free space size = 15.64 MB Largest free space size = 7.24 GB Master File Table (MFT): MFT size = 921.25 MB MFT record count = 943359 MFT usage = 100% Total MFT fragments = 7
Note: File fragments larger than 64MB are not included in the fragmentation statistics. You do not need to defragment this volume.
The summary of the volume's configuration and space usage reports on the following areas:
- Overall fragmentation:
Gives an overview of fragmentation showing the percentage of used space that is fragmented, the average number of fragments per file, the total number of files on the volume that are movable, and the total number of unmovable files. Ideally, you want the percentage of fragmentation to be 10 percent or less and the number of fragments per file to be as close to 1.00 as possible. - File fragmentation:
Gives an overview of file-level fragmentation showing how many files are fragmented and the total number of excess fragments. - Folder fragmentation:
Gives an overview of folder-level fragmentation showing the total number of folders on the volume, how many folders are fragmented, and the total number of excess fragments. - Free space fragmentation:
Gives an overview of fragmentation on a volume's unused space showing how much free space is available on the volume, the number of extents on which free space is located, the average amount of free space per extent, and the largest free-space extent. - Master file table (MFT) fragmentation:
For NTFS volumes only, gives an overview of fragmentation in the MFT ing the current size of the MFT, the number of records it contains, the percentage of the MFT in use, and the total number of fragments in the MFT. In this example, the MFT has some fragmentation, but the real concern is that it is at 100 percent of its maximum size. Because of this, the MFT could become more fragmented over time-there is still 37.85 GBs of free space on the volume, and if it needs to grow, it will grow into the free space.
If you run Defrag again, using /o, Optimize Drives sets about cleaning up the drive to give optimal space usage. This won't clear up all fragmentation, but it will help so that disk space is used more efficiently. On a volume like the one shown with very little fragmentation, you won't really see performance improvements after defragmentation. However, if the fragmentation percentage were higher, performance improvements could be considerable.
With virtual disks, slab consolidation and retrim are used to optimize external structures. You can perform these tasks at the command line by using the /k and /v parameters. The command-line report shows the summary of optimization. The summary looks like this:
Invoking slab consolidation on New Volume (I:)... Retrim: 100% complete. Slab consolidation: 100% complete. The operation completed successfully. Post Defragmentation Report: Volume Information: Volume size = 126.99 GB Cluster size = 4 KB Used space = 102.06 GB Free space = 24.93 GB Allocation Units: Slab count = 4063 Slab size = 32 MB Slab alignment = 31.00 MB In-use slabs = 3266 Slab Consolidation: Space efficiency = 92% Potential purgable slabs = 48 Slabs pinned unmovable = 704 Successfully purged slabs = 36 Recovered space = 1152 MB Retrim: Backed allocations = 68 Allocations trimmed = 18 Total space trimmed = 539.82 MB
The summary reports on the following areas:
- Volume Information:
Gives an overview of the volume showing its maximum volume size, cluster size, used space, and free space. Because you are optimizing external structures, free space in the volume doesn't affect whether you can consolidate slabs and retrim. - Allocation Units:
Gives an overview of the volume's allocation units showing the total number of slabs allocated to the volume, the size of the slabs, the slab alignment offset, and the number of in-use slabs. The difference between the total slab count and the in-use slab count represents the number of available slabs. - Slab Consolidation:
Gives an overview of the slab consolidation showing how efficiently space is being used, the potentially purgable slabs remaining, the number of unmovable slabs, the number of successfully purged slabs, and the total space recovered. Ideally, after consolidating slabs, a high level of space efficiency and relatively few potentially purgable slabs will remain. - Retrim:
Gives an overview of the underlying physical disk space recovered by trimming sectors that were previously allocated to the virtual disk but the virtual disk no longer uses. It shows the number of allocations trimmed and the total space recovered.