Defragmenting Linux filesystems (Questions and Answers (by Jesse Smith))

fblais · Aug 10, 2021

Quoted from Distrowatch:

Getting-the-pieces-jumbled-together asks:
I was wondering if you could do a quick recap of the status of defrag in different filesystems, for SSDs. More specifically, I cannot figure out if it makes any sense to mount Btrfs root and home subvolumes with "-o autodefrag".

DistroWatch answers:
For people who may not be familiar with the concept of defragmenting a filesystem, first I'd like to provide a little background information. Typically filesystems try to save new files as one long string of data, keeping the pieces of the file all in one place. However, over time files can change in size or the drive can fill up, reducing the amount of free contiguous space. When this happens pieces of files may end up written to different parts of the disk. On a modern SSD storage devices this fragmentation usually doesn't have a big impact, but on spinning hard drives, where the disk needs to physically adjust to scan the area they read from, fragmented files can slow down the process of reading data from the hard drive.

Basically, a fragmented filesystem means the pieces of multiple files are scattered around the hard drive instead of being neatly organized into contiguous chunks. This disorganization can slow down file access.

Defragmenting a filesystem (also known as defragging) means gathering up the pieces of scattered files and reorganizing them. A defragmented filesystem has neatly ordered blocks of files and reading them no longer requires reading from lots of different parts of the disk.

Most modern Linux filesystems do a good job of organizing their data so that fragmentation rarely happens to the point where it becomes a problem. Typically, assuming 20% or more of your hard drive space is free, Linux filesystems will shift data around to make sure the disk organization does not become too fragmented. This is further helped by using SSDs, as the storage devices handle reads from random locations on the drive easier than classic spinning disks.

Now, let's talk specifically about Btrfs, an advanced filesystem with copy-on-write (CoW) data, snapshots, and an option which allows for automatic defragmentation (autodefrag). The Btrfs section of the Linux kernel wiki mentions fragmentation as a possible problem, particularly when using databases and virtual machines: Files with a lot of random writes can become heavily fragmented (10,000+ extents) causing thrashing on HDDs and excessive multi-second spikes of CPU load on systems with an SSD or large amount a RAM. On servers and workstations this affects databases and virtual machine images. The wiki goes on to mention fragmentation may happen when running applications which heavily make use of databases, such as web browsers and e-mail clients. The result can be heavier CPU usage and poorer performance. The recommended solution, according to the wiki, is to periodically run the defragmentation command manually or enable the autodefrag option: "Workarounds include manually defragmenting your home directory using 'btrfs fi defragment'. Auto-defragment (mount option autodefrag) should solve this problem in 3.0."

At this point it sounds as though enabling autodefrag on your Btrfs volumes makes sense. If there is no fragmentation then there is no harm done, but if your filesystem is processing a lot of random writes the autodefrag flag should fix the problem. Before you rush ahead to turn on the automatic defragmentation option, consider Oracle's Btrfs documentation which offers this advice: You can set up automatic defragmentation by specifying the autodefrag option when you mount the file system. However, automatic defragmentation is not recommended for large databases or for images of virtual machines.

Defragmenting a file or a subvolume that has a copy-on-write copy results breaks [sic] the link between the file and its copy. For example, if you defragment a subvolume that has a snapshot, the disk usage by the subvolume and its snapshot will increase because the snapshot is no longer a copy-on-write image of the subvolume. In other words, there is a price to pay for less fragmentation and that is larger amounts of disk space being consumed. The Btrfs manual page echoes this concern: "This may cause considerable increase of space usage depending on the broken up reflinks"

My suggestion would be to leave autodefrag turned off by default. Chances are that, in most situations, you won't need it. However, if you do find your filesystem access slowing down you can perform a one-time defrag from the command line by running "btrfs filesystem defragment". The btrfs-filesysetm manual page has a section dedicated to defragmenting Btrfs and its subvolumes. I recommend looking at the various options, particularly those related to recursively defragmenting a directory.

* * * * *

Additional answers can be found in our Questions and Answers archive.
Click to expand...

EASTER · Aug 10, 2021

Thank You. I've personally harped on this very fact for years, (particularly Windows) and been met with skepticism of wasting my time. However with a spindle HDD keeping the disk platter organized and more often is literally prolonged every (Windows) system I had and still have running optimum for years simply by performing this maintenance routine in particular. Although this topic indeed addresses Linux most specifically.

Basically, a fragmented filesystem means the pieces of multiple files are scattered around the hard drive instead of being neatly organized into contiguous chunks. This disorganization can slow down file access.

Defragmenting a filesystem (also known as defragging) means gathering up the pieces of scattered files and reorganizing them. A defragmented filesystem has neatly ordered blocks of files and reading them no longer requires reading from lots of different parts of the disk.

Most modern Linux filesystems do a good job of organizing their data so that fragmentation rarely happens to the point where it becomes a problem. Typically, assuming 20% or more of your hard drive space is free, Linux filesystems will shift data around to make sure the disk organization does not become too fragmented. This is further helped by using SSDs, as the storage devices handle reads from random locations on the drive easier than classic spinning disks.
Click to expand...

Log in or Sign up

Defragmenting Linux filesystems (Questions and Answers (by Jesse Smith))

fblais Registered Member

EASTER Registered Member

Log in or Sign up

Defragmenting Linux filesystems (Questions and Answers (by Jesse Smith))

fblais Registered Member

EASTER Registered Member

Useful Searches