4.2 Filesystems
The last link between the kernel and user space for disks is typically the filesystem; this is what you’re accustomed to interacting with when you run commands like ls and cd. As previously mentioned, the filesystem is a form of database; it supplies the structure to transform a simple block device into the sophisticated hierarchy of files and subdirectories that users can understand.
At one time, all filesystems resided on disks and other physical media that were intended exclusively for data storage. However, the tree-like directory structure and I/O interface of filesystems are quite versatile, so filesystems now perform a variety of tasks, such as the system interfaces that you see in /sys and /proc. Filesystems are traditionally implemented in the kernel, but p. 81 the innovation of 9P from Plan 9 (https://en.wikipedia.org/wiki/9P_(protocol)) has inspired the development of user-space filesystems. The File System in User Space (FUSE) feature allows user-space filesystems in Linux.
The Virtual File System (VFS) abstraction layer completes the filesystem implementation. Much as the SCSI subsystem standardizes communication between different device types and kernel control commands, VFS ensures that all filesystem implementations support a standard interface so that user-space applications access files and directories in the same manner. VFS support has enabled Linux to support an extraordinarily large number of filesystems.
4.2.1 Filesystem Types
Linux filesystem support includes native designs optimized for Linux; foreign types, such as the Windows FAT family; universal filesystems, like ISO 9660; and many others. The following list includes the most common types of filesystems for data storage. The type names as recognized by Linux are in parentheses next to the filesystem names.
-
The Fourth Extended filesystem (ext4) is the current iteration of a line of filesystems native to Linux. The Second Extended filesystem (ext2) was a longtime default for Linux systems inspired by traditional Unix filesystems, such as the Unix File System (UFS) and the Fast File System (FFS). The Third Extended filesystem (ext3) added a journal feature (a small cache outside the normal filesystem data structure) to enhance data integrity and hasten booting. The ext4 filesystem is an incremental improvement and supports larger files than ext2 or ext3 as well as a greater number of subdirectories.
There’s a certain amount of backward compatibility in the extended filesystem series. For example, you can mount ext2 and ext3 filesystems as each other, and you can mount ext2 and ext3 filesystems as ext4, but you cannot mount ext4 as ext2 or ext3.
-
Btrfs, or B-tree filesystem (btrfs), is a newer filesystem native to Linux designed to scale beyond the capabilities of ext4.
-
FAT filesystems (msdos, vfat, exfat) pertain to Microsoft systems. The simple msdos type supports the very primitive monocase variety in MS-DOS systems. Most removable flash media, such as SD cards and USB drives, contain vfat (up to 4GB) or exfat (4GB and up) partitions by default. Windows systems can use either a FAT-based filesystem or the more advanced NT File System (ntfs).
-
XFS is a high-performance filesystem used by default by some distributions, such as Red Hat Enterprise Linux 7.0 and beyond.
-
HFS+ (hfsplus) is an Apple standard used on most Macintosh systems.
-
ISO 9660 (iso9660) is a CD-ROM standard. Most CD-ROMs use some variety of the ISO 9660 standard.
The Extended filesystem series has long been perfectly acceptable to most users, and the fact that it has remained the de facto standard for so long is a testament to its utility, but also to its adaptability. The Linux development community has a tendency to completely replace components that don’t meet current needs, but every time the Extended filesystem has come up short, someone has upgraded it in response. Nonetheless, many advances have been made in filesystem technology that even ext4 cannot utilize due to the backward-compatibility requirement. These advances are primarily in scalability enhancements pertaining to very large numbers of files, large files, and similar scenarios.
At the time of this writing, Btrfs is the default for one major Linux distribution. If this proves a success, it’s likely that Btrfs will be poised to replace the Extended series.
4.2.2 Creating a Filesystem
If you’re preparing a new storage device, once you’re finished with the partitioning process described in Section 4.1, you’re ready to create a filesystem. As with partitioning, you’ll do this in user space because a user-space process can directly access and manipulate a block device.
The mkfs utility can create many kinds of filesystems. For example, you can create an ext4 partition on /dev/sdf2 with this command:
# mkfs -t ext4 /dev/sdf2
The mkfs program automatically determines the number of blocks in a device and sets some reasonable defaults. Unless you really know what you’re doing and feel like reading the documentation in detail, don’t change them.
When you create a filesystem, mkfs prints diagnostic output as it works, including output pertaining to the superblock. The superblock is a key component at the top level of the filesystem database, and it’s so important that mkfs creates a number of backups in case the original is destroyed. Consider recording a few of the superblock backup numbers when mkfs runs, in case you need to recover the superblock in the event of a disk failure (see Section 4.2.11).
Filesystem creation is a task that you should perform only after adding a new disk or repartitioning an old one. You should create a filesystem just once for each new partition that has no preexisting data (or that has data you want to remove). Creating a new filesystem on top of an existing filesystem will effectively destroy the old data.
It turns out that mkfs is only a frontend for a series of filesystem creation programs, mkfs.fs, where fs is a filesystem type. So when you run mkfs -t ext4, mkfs in turn runs mkfs.ext4.
And there’s even more indirection. Inspect the mkfs.* files behind the commands, and you’ll see the following:
$ ls -l /sbin/mkfs.*
-rwxr-xr-x 1 root root 17896 Mar 29 21:49 /sbin/mkfs.bfs
-rwxr-xr-x 1 root root 30280 Mar 29 21:49 /sbin/mkfs.cramfs
lrwxrwxrwx 1 root root 6 Mar 30 13:25 /sbin/mkfs.ext2 -> mke2fs
lrwxrwxrwx 1 root root 6 Mar 30 13:25 /sbin/mkfs.ext3 -> mke2fs
lrwxrwxrwx 1 root root 6 Mar 30 13:25 /sbin/mkfs.ext4 -> mke2fs
lrwxrwxrwx 1 root root 6 Mar 30 13:25 /sbin/mkfs.ext4dev -> mke2fs
-rwxr-xr-x 1 root root 26200 Mar 29 21:49 /sbin/mkfs.minix
lrwxrwxrwx 1 root root 7 Dec 19 2011 /sbin/mkfs.msdos -> mkdosfs
lrwxrwxrwx 1 root root 6 Mar 5 2012 /sbin/mkfs.ntfs -> mkntfs
lrwxrwxrwx 1 root root 7 Dec 19 2011 /sbin/mkfs.vfat -> mkdosfs
As you can see, mkfs.ext4 is just a symbolic link to mke2fs. This is important to remember if you run across a system without a specific mkfs command or when you’re looking up the documentation for a particular filesystem. Each filesystem’s creation utility has its own manual page, like mke2fs(8). This shouldn’t be a problem on most systems, because accessing the mkfs.ext4(8) manual page should redirect you to the mke2fs(8) manual page, but keep it in mind.
4.2.3 Mounting a Filesystem
On Unix, the process of attaching a filesystem to a running system is called mounting. When the system boots, the kernel reads some configuration data and mounts root (/) based on the configuration data.
In order to mount a filesystem, you must know the following:
- The filesystem’s device, location, or identifier (such as a disk partition—where the actual filesystem data resides). Some special-purpose filesystems, such as proc and sysfs, don’t have locations.
- The filesystem type.
- The mount point—the place in the current system’s directory hierarchy where the filesystem will be attached. The mount point is always a normal directory. For instance, you could use /music as a mount point for a filesystem containing music. The mount point need not be directly below /; it can be anywhere on the system.
The common terminology for mounting a filesystem is “mount a device on a mount point.” To learn the current filesystem status of your system, you run mount. The output (which can be quite lengthy) should look like this:
$ mount
/dev/sda1 on / type ext4 (rw,errors=remount-ro)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
fusectl on /sys/fs/fuse/connections type fusectl (rw)
debugfs on /sys/kernel/debug type debugfs (rw)
securityfs on /sys/kernel/security type securityfs (rw)
udev on /dev type devtmpfs (rw,mode=0755)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
--snip--
Each line corresponds to one currently mounted filesystem, with items in this order:
- The device, such as /dev/sda3. Notice that some of these aren’t real devices (
proc, for example) but are stand-ins for real device names because these special-purpose filesystems do not need devices. - The word
on. - The mount point.
- The word
type. - The filesystem type, usually in the form of a short identifier.
- Mount options (in parentheses). See Section 4.2.6 for more details.
To mount a filesystem manually, use the mount command as follows with the filesystem type, device, and desired mount point:
# mount -t type device mountpoint
For example, to mount the Fourth Extended filesystem found on the device /dev/sdf2 on /home/extra, use this command:
# mount -t ext4 /dev/sdf2 /home/extra
You normally don’t need to supply the -t type option because mount usually figures it out for you. However, sometimes it’s necessary to distinguish between two similar types, such as the various FAT-style filesystems.
To unmount (detach) a filesystem, use the umount command as follows:
# umount mountpoint
You can also unmount a filesystem with its device instead of its mount point.
Almost all Linux systems include a temporary mount point, /mnt, which is typically used for testing. Feel free to use it when experimenting with your system, but if you intend to mount a filesystem for extended use, find or make another spot.
4.2.4 Filesystem UUID
The method of mounting filesystems discussed in the preceding section depends on device names. However, device names can change because they depend on the order in which the kernel finds the devices. To solve this problem, you can identify and mount filesystems by their universally unique identifier (UUID), an industry standard for unique “serial numbers” to identify objects in a computer system. Filesystem creation programs like mke2fs generate a UUID when initializing the filesystem data structure.
To view a list of devices and the corresponding filesystems and UUIDs on your system, use the blkid (block ID) program:
# blkid
/dev/sdf2: UUID="b600fe63-d2e9-461c-a5cd-d3b373a5e1d2" TYPE="ext4"
/dev/sda1: UUID="17f12d53-c3d7-4ab3-943e-a0a72366c9fa" TYPE="ext4"
PARTUUID="c9a5ebb0-01"
/dev/sda5: UUID="b600fe63-d2e9-461c-a5cd-d3b373a5e1d2" TYPE="swap"
PARTUUID="c9a5ebb0-05"
/dev/sde1: UUID="4859-EFEA" TYPE="vfat"
In this example, blkid found four partitions with data: two with ext4 filesystems, one with a swap space signature (see Section 4.3), and one with a FAT-based filesystem. The Linux native partitions all have standard UUIDs, but the FAT partition doesn’t. You can reference the FAT partition with its FAT volume serial number (in this case, 4859-EFEA).
To mount a filesystem by its UUID, use the UUID mount option. For example, to mount the first filesystem from the preceding list on /home/extra, enter:
# mount UUID=b600fe63-d2e9-461c-a5cd-d3b373a5e1d2 /home/extra
Typically you won’t manually mount filesystems by UUID like this, because you normally know the device, and it’s much easier to mount a device by its name than by its crazy UUID. Still, it’s important to understand UUIDs. For one thing, they’re the preferred way to mount non-LVM filesystems in /etc/fstab automatically at boot time (see Section 4.2.8). In addition, many distributions use the UUID as a mount point when you insert removable media. In the preceding example, the FAT filesystem is on a flash media card. An Ubuntu system with someone logged in will mount this partition at /media/user/4859-EFEA upon insertion. The udevd daemon described in Chapter 3 handles the initial event for the device insertion.
You can change the UUID of a filesystem if necessary (for example, if you copied the complete filesystem from somewhere else and now need to distinguish it from the original). See the tune2fs(8) manual page for how to do this on an ext2/ext3/ext4 filesystem.
4.2.5 Disk Buffering, Caching, and Filesystems
Linux, like other Unix variants, buffers writes to the disk. This means the kernel usually doesn’t immediately write changes to filesystems when processes request changes. Instead, it stores those changes in RAM until the kernel determines a good time to actually write them to the disk. This buffering system is transparent to the user and provides a very significant performance gain.
When you unmount a filesystem with umount, the kernel automatically synchronizes with the disk, writing the changes in its buffer to the disk. You can also force the kernel to do this at any time by running the sync command, which by default synchronizes all the disks on the system. If for some reason you can’t unmount a filesystem before you turn off the system, be sure to run sync first.
In addition, the kernel uses RAM to cache blocks as they’re read from a disk. Therefore, if one or more processes repeatedly access a file, the kernel doesn’t have to go to the disk again and again—it can simply read from the cache and save time and resources.
4.2.6 Filesystem Mount Options
There are many ways to change the mount command behavior, which you’ll often need to do when working with removable media or performing system maintenance. In fact, the total number of mount options is staggering. The extensive mount(8) manual page is a good reference, but it’s hard to know where to start and what you can safely ignore. You’ll see the most useful options in this section.
Options fall into two rough categories: general and filesystem-specific. General options typically work for all filesystem types and include -t for specifying the filesystem type, as shown earlier. In contrast, a filesystem-specific option pertains only to certain filesystem types.
To activate a filesystem option, use the -o switch followed by the option. For example, -o remount,rw remounts a filesystem already mounted as read-only in read-write mode.
Short General Options
General options have a short syntax. The most important are:
-r The -r option mounts the filesystem in read-only mode. This has a number of uses, from write protection to bootstrapping. You don’t need to specify this option when accessing a read-only device, such as a CD-ROM; the system will do it for you (and will also tell you about the read-only status).
-n The -n option ensures that mount does not try to update the system runtime mount database, /etc/mtab. By default, the mount operation fails when it cannot write to this file, so this option is important at boot time because the root partition (including the system mount database) is p. 87 read-only at first. You’ll also find this option handy when trying to fix a system problem in single-user mode, because the system mount database may not be available at the time.
-t The -t type option specifies the filesystem type.
Long Options
Short options like -r are too limited for the ever-increasing number of mount options; there are too few letters in the alphabet to accommodate all possible options. Short options are also troublesome because it’s difficult to determine an option’s meaning based on a single letter. Many general options and all filesystem-specific options use a longer, more flexible option format.
To use long options with mount on the command line, start with -o followed by the appropriate keywords separated by commas. Here’s a complete example, with the long options following -o:
# mount -t vfat /dev/sde1 /dos -o ro,uid=1000
The two long options here are ro and uid=1000. The ro option specifies read-only mode and is the same as the -r short option. The uid=1000 option tells the kernel to treat all files on the filesystem as if user ID 1000 is the owner.
The most useful long options are these:
exec, noexec Enables or disables execution of programs on the filesystem.
suid, nosuid Enables or disables setuid programs.
ro Mounts the filesystem in read-only mode (as does the -r short option).
rw Mounts the filesystem in read-write mode.
There is a difference between Unix and DOS text files, principally in how lines end. In Unix, only a linefeed (\n, ASCII 0x0A) marks the end of a line, but DOS uses a carriage return (\r, ASCII 0x0D) followed by a linefeed. There have been many attempts at automatic conversion at the filesystem level, but these are always problematic. Text editors such as vim can automatically detect the newline style of a file and maintain it appropriately. It’s easier to keep the styles uniform this way.
4.2.7 Remounting a Filesystem
There will be times when you need to change the mount options for a currently mounted filesystem; the most common situation is when you need to make a read-only filesystem writable during crash recovery. In that case, you need to reattach the filesystem at the same mount point.
The following command remounts the root directory in read-write mode (you need the -n option because the mount command can’t write to the system mount database when the root is read-only):
# mount -n -o remount /
This command assumes that the correct device listing for / is in /etc/fstab (as discussed in the next section). If it isn’t, you must specify the device as an additional option.
4.2.8 The /etc/fstab Filesystem Table
To mount filesystems at boot time and take the drudgery out of the mount command, Linux systems keep a permanent list of filesystems and options in /etc/fstab. This is a plaintext file in a very simple format, as Listing 4-1 shows.
UUID=70ccd6e7-6ae6-44f6-812c-51aab8036d29 / ext4 errors=remount-ro 0 1
UUID=592dcfd1-58da-4769-9ea8-5f412a896980 none swap sw 0 0
/dev/sr0 /cdrom iso9660 ro,user,nosuid,noauto 0 0
Listing 4-1: List of filesystems and options in /etc/fstab
Each line corresponds to one filesystem and is broken into six fields. From left to right, these fields are:
The device or UUID Most current Linux systems no longer use the device in /etc/fstab, preferring the UUID.
The mount point Indicates where to attach the filesystem.
The filesystem type You may not recognize swap in this list; this is a swap partition (see Section 4.3).
Options Long options, separated by commas.
Backup information for use by the dump command The dump command is a long-obsolete backup utility; this field is no longer relevant. You should always set it to 0.
The filesystem integrity test order To ensure that fsck always runs on the root first, always set this to 1 for the root filesystem and 2 for any other locally attached filesystems on a hard disk or SSD. Use 0 to disable the bootup check for every other filesystem, including read-only devices, swap, and the /proc filesystem (see the fsck command in Section 4.2.11).
When using mount, you can take some shortcuts if the filesystem you want to work with is in /etc/fstab. For example, if you were using Listing 4-1 and mounting a CD-ROM, you would simply run mount /cdrom.
You can also try to simultaneously mount all entries in /etc/fstab that do not contain the noauto option with this command:
# mount -a
Listing 4-1 introduces some new options—namely, errors, noauto, and user, because they don’t apply outside the /etc/fstab file. In addition, you’ll often see the defaults option here. These options are defined as follows:
defaults This sets the mount defaults: read-write mode, enable device files, executables, the setuid bit, and so on. Use this when you don’t want to give the filesystem any special options but you do want to fill all fields in /etc/fstab.
errors This ext2/3/4-specific parameter sets the kernel behavior when the system has trouble mounting a filesystem. The default is normally errors=continue, meaning that the kernel should return an error code and keep running. To have the kernel try the mount again in read-only mode, use errors=remount-ro. The errors=panic setting tells the kernel (and your system) to halt when there’s a problem with the mount.
noauto This option tells a mount -a command to ignore the entry. Use this to prevent a boot-time mount of a removable-media device, such as a flash storage device.
user This option allows unprivileged users to run mount on a particular entry, which can be handy for allowing certain kinds of access to removable media. Because users can put a setuid-root file on removable media with another system, this option also sets nosuid, noexec, and nodev (to bar special device files). Keep in mind that for removable media and other general cases, this option is now of limited use, because most systems use ubus along with other mechanisms to automatically mount inserted media. However, this option can be useful in special cases when you want to grant control over mounting specific directories.
4.2.9 Alternatives to /etc/fstab
Although the /etc/fstab file has been the traditional way to represent filesystems and their mount points, there are two alternatives. The first is an /etc/fstab.d directory, which contains individual filesystem configuration files (one file for each filesystem). The idea is very similar to many other configuration directories that you’ll see throughout this book.
A second alternative is to configure systemd units for the filesystems. You’ll learn more about systemd and its units in Chapter 6. However, the systemd unit configuration is often generated from (or based on) the /etc/fstab file, so you may find some overlap on your system.
4.2.10 Filesystem Capacity
To view the size and utilization of your currently mounted filesystems, use the df command. The output can be very extensive (and it gets longer all the time, thanks to specialized filesystems), but it should include information on your actual storage devices.
$ df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 214234312 127989560 75339204 63% /
/dev/sdd2 3043836 4632 2864872 1% /media/user/uuid
Here’s a brief description of the fields in the df output:
Filesystem The filesystem device
1K-blocks The total capacity of the filesystem in blocks of 1,024 bytes
Used The number of occupied blocks
Available The number of free blocks
Use% The percentage of blocks in use
Mounted on The mount point
If you’re having trouble finding the correct line in the df output corresponding to a particular directory, run the df dir command, where dir is the directory you want to examine. This limits output to the filesystem for that directory. A very common use is df ., which limits the output to the device holding your current directory.
It should be easy to see that the two filesystems here are roughly 215GB and 3GB in size. However, the capacity numbers may look a little strange because 127,989,560 plus 75,339,204 does not equal 214,234,312, and 127,989,560 is not 63 percent of 214,234,312. In both cases, 5 percent of the total capacity is unaccounted for. In fact, the space is there, but it’s hidden in reserved blocks. Only the superuser can use the reserved blocks of the filesystem when it starts to fill up. This feature keeps system servers from immediately failing when they run out of disk space.
If your disk fills up and you need to know where all of those space-hogging media files are, use the du command. With no arguments, du prints the disk usage of every directory in the directory hierarchy, starting at the current working directory. (That can be a long listing; if you want to see an example, just run cd /; du to get the idea. Press CTRL-C when you get bored.) The du -s command turns on summary mode to print only the grand total. To evaluate everything (files and subdirectories) in a particular directory, change to that directory and run du -s *, keeping in mind that there can be some dot directories that this command won’t catch.
The POSIX standard defines a block size of 512 bytes. However, this size is harder to read, so by default, the df and du output in most Linux distributions is in 1,024-byte blocks. If you insist on displaying the numbers in 512-byte blocks, set the p. 91 POSIXLY_CORRECT environment variable. To explicitly specify 1,024-byte blocks, use the -k option (both utilities support this). The df and du programs also have a -m option to list capacities in 1MB blocks and a -h option to take a best guess at what’s easiest for a person to read, based on the overall sizes of the filesystems.
4.2.11 Checking and Repairing Filesystems
The optimizations that Unix filesystems offer are made possible by a sophisticated database mechanism. For filesystems to work seamlessly, the kernel has to trust that a mounted filesystem has no errors and also that the hardware stores data reliably. If errors exist, data loss and system crashes may result.
Aside from hardware problems, filesystem errors are usually due to a user shutting down the system in a rude way (for example, by pulling out the power cord). In such cases, the previous filesystem cache in memory may not match the data on the disk, and the system also may be in the process of altering the filesystem when you happen to give the computer a kick. Although many filesystems support journals to make filesystem corruption far less common, you should always shut down the system properly. Regardless of the filesystem in use, filesystem checks are still necessary every now and then to make sure that everything is still in order.
The tool to check a filesystem is fsck. As with the mkfs program, there’s a different version of fsck for each filesystem type that Linux supports. For example, when run on an Extended filesystem series (ext2/ext3/ext4), fsck recognizes the filesystem type and starts the e2fsck utility. Therefore, you generally don’t need to type e2fsck, unless fsck can’t figure out the filesystem type or you’re looking for the e2fsck manual page.
The information presented in this section is specific to the Extended filesystem series and e2fsck.
To run fsck in interactive manual mode, give the device or the mount point (as listed in /etc/fstab) as the argument. For example:
# fsck /dev/sdb1
Never use fsck on a mounted filesystem—the kernel may alter the disk data as you run the check, causing runtime mismatches that can crash your system and corrupt files. There is only one exception: if you mount the root partition read-only in single-user mode, you may use fsck on it.
In manual mode, fsck prints verbose status reports on its passes, which should look something like this when there are no problems:
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/sdb1: 11/1976 files (0.0% non-contiguous), 265/7891 blocks
If fsck finds a problem in manual mode, it stops and asks a question relevant to fixing the problem. These questions deal with the internal structure of the filesystem, such as reconnecting loose inodes and clearing blocks (inodes are building blocks of the filesystem; you’ll see how they work in Section 4.6). When fsck asks you about reconnecting an inode, it has found a file that doesn’t appear to have a name. When reconnecting such a file, fsck places the file in the lost+found directory in the filesystem, with a number as the filename. If this happens, you need to guess the name based on the file’s contents; the original filename is probably gone.
In general, it’s pointless to sit through the fsck repair process if you’ve just uncleanly shut down the system, because fsck may have a lot of minor errors to fix. Fortunately, e2fsck has a -p option that automatically fixes ordinary problems without asking and aborts when there’s a serious error. In fact, Linux distributions run a variant of fsck -p at boot time. (You may also see fsck -a, which does the same thing.)
If you suspect a major disaster on your system, such as a hardware failure or device misconfiguration, you need to decide on a course of action, because fsck can really mess up a filesystem that has larger problems. (One telltale sign that your system has a serious problem is if fsck asks a lot of questions in manual mode.)
If you think that something really bad has happened, try running fsck -n to check the filesystem without modifying anything. If there’s a problem with the device configuration that you think you can fix (such as loose cables or an incorrect number of blocks in the partition table), fix it before running fsck for real, or you’re likely to lose a lot of data.
If you suspect that only the superblock is corrupt (for example, because someone wrote to the beginning of the disk partition), you might be able to recover the filesystem with one of the superblock backups that mkfs creates. Use fsck -b num to replace the corrupted superblock with an alternate at block num and hope for the best.
If you don’t know where to find a backup superblock, you might be able to run mkfs -n on the device to view a list of superblock backup numbers without destroying your data. (Again, make sure that you’re using -n, or you’ll really tear up the filesystem.)
Checking ext3 and ext4 Filesystems
You normally do not need to check ext3 and ext4 filesystems manually because the journal ensures data integrity (recall that the journal is a small data cache that has not yet been written to a specific spot in the filesystem). If you don’t shut your system down cleanly, you can expect the journal to contain some data. To flush the journal in an ext3 or ext4 filesystem to the regular filesystem database, run e2fsck as follows:
# e2fsck –fy /dev/disk_device
However, you may want to mount a broken ext3 or ext4 filesystem in ext2 mode, because the kernel won’t mount an ext3 or ext4 filesystem with a nonempty journal.
The Worst Case
Disk problems that are more severe leave you with few choices:
- You can try to extract the entire filesystem image from the disk with
ddand transfer it to a partition on another disk of the same size. - You can try to patch the filesystem as much as possible, mount it in read-only mode, and salvage what you can.
- You can try
debugfs.
In the first two cases, you still need to repair the filesystem before you mount it, unless you feel like picking through the raw data by hand. If you like, you can choose to answer y to all of the fsck questions by entering fsck -y, but do this as a last resort because issues may come up during the repair process that you would rather handle manually.
The debugfs tool allows you to look through the files on a filesystem and copy them elsewhere. By default, it opens filesystems in read-only mode. If you’re recovering data, it’s probably a good idea to keep your files intact to avoid messing things up further.
Now, if you’re really desperate—say with a catastrophic disk failure on your hands and no backups—there isn’t a lot you can do other than hope a professional service can “scrape the platters.”
4.2.12 Special-Purpose Filesystems
Not all filesystems represent storage on physical media. Most versions of Unix have filesystems that serve as system interfaces. That is, rather than serving only as a means to store data on a device, a filesystem can represent system information, such as process IDs and kernel diagnostics. This idea goes back to the /dev mechanism, which is an early model of using files for I/O interfaces. The /proc idea came from the eighth edition of research Unix, implemented by Tom J. Killian and accelerated when Bell Labs (including many of the original Unix designers) created Plan 9—a research operating system that took filesystem abstraction to a whole new level (https://en.wikipedia.org/wiki/Plan_9_from_Bell_Labs).
Some of the special filesystem types in common use on Linux include:
proc Mounted on /proc. The name proc is an abbreviation for process. Each numbered directory inside /proc refers to the ID of a current process on the system; the files in each directory represent various aspects of that process. The directory /proc/self represents the current process. The Linux proc filesystem includes a great deal of additional kernel and hardware information in files like /proc/cpuinfo. Keep in mind that the kernel design guidelines recommend moving information unrelated to processes out of /proc and into /sys, so system information in /proc might not be the most current interface.
sysfs Mounted on /sys. (You saw this in Chapter 3.)
tmpfs Mounted on /run and other locations. With tmpfs, you can use your physical memory and swap space as temporary storage. You can p. 94 mount tmpfs where you like, using the size and nr_blocks long options to control the maximum size. However, be careful not to pour things constantly into a tmpfs location, because your system will eventually run out of memory and programs will start to crash.
squashfs A type of read-only filesystem where content is stored in a compressed format and extracted on demand through a loopback device. One example use is in the snap package management system that mounts packages under the /snap directory.
overlay A filesystem that merges directories into a composite. Containers often use overlay filesystems; you’ll see how they work in Chapter 17.