Linux I/O Explained: File I/O, Page Cache, VFS, and Block I/O

Introduction

When operating Linux servers, you may encounter problems such as slow disk I/O, high I/O wait, or file reads and writes that take too long. However, it is not always easy to visualize what happens inside Linux after an application calls read() or write().

When an application reads or writes a file, data does not normally move directly between the application and storage. It passes through several Linux kernel mechanisms, including system calls, the VFS, filesystem, Page Cache, Block Layer, and device drivers.

Understanding these components makes it easier to determine whether an I/O performance problem originates in the application, memory, filesystem, or storage. It also helps explain why reading a file does not always cause storage I/O and why a successful write() does not necessarily mean that the data has already been persisted to storage.

This article explains the fundamentals of Linux file I/O and follows the flow of read() and write() from an application through the Linux kernel to storage.

What Is I/O in Linux?

I/O stands for Input / Output. It refers to the transfer of data between a program and another endpoint.

Linux handles many kinds of I/O through file descriptors. Applications can access regular files, terminals, pipes, sockets, and devices through common interfaces such as read() and write().

• File I/O: Reading or writing files
• Storage I/O: Reading or writing data on block devices such as SSDs and HDDs
• Network I/O: Sending or receiving data through sockets

File I/O and storage I/O are not necessarily the same operation. For example, if requested file data is already available in the Page Cache, Linux can satisfy a read from memory without issuing storage I/O at that time.

Overview of Linux I/O

Typical buffered file I/O can be viewed as the following path.

An application calls a system call such as read() or write(). The request enters the Linux kernel and is handled through mechanisms such as the VFS and the target filesystem.

With normal buffered I/O, the Page Cache plays an important role between file operations and storage. If requested data is already cached, a read may complete without accessing storage. For writes, data is normally placed in the Page Cache first and written to storage later.

When physical storage access is required, block I/O requests pass through the Block Layer and the appropriate device driver before reaching the storage device.

This is a simplified model intended to explain the overall architecture. Actual I/O paths can differ when mechanisms such as mmap(), Direct I/O, asynchronous I/O, Device Mapper, software RAID, network filesystems, or virtualization are involved.

read(), write(), and System Calls

read() and write() are representative system calls used to transfer data through a file descriptor.

• read(): Reads kernel-managed data into a User Space buffer
• write(): Passes data from a User Space buffer to the kernel

When an application invokes a system call, execution enters kernel mode so that the Linux kernel can perform the requested operation. After the kernel finishes the necessary processing, control returns to the application in User Space.

How File Reads Work

1. The Application Calls read()

The application calls read() using a file descriptor previously obtained through open() or a similar operation, a destination buffer, and the number of bytes to read.

A file descriptor is an integer that identifies an open file, socket, pipe, or another kernel-managed object belonging to the process.

2. Processing Enters the Kernel

The system call transfers control to the kernel. Linux validates the request and identifies the kernel object associated with the file descriptor.

3. VFS and Filesystem Processing

The VFS (Virtual File System) provides a common interface for file operations and routes the request to the filesystem that manages the target file, such as ext4 or XFS.

The filesystem uses information such as file metadata and the current file offset to determine which part of the file is required.

4. Page Cache Hit or Miss

The kernel checks whether the requested file data is already available in the Page Cache.

If the data is cached, Linux can copy it from memory to the application’s buffer without issuing storage I/O for that data at that time. This is called a cache hit.

If the data is not cached, storage access is required. The filesystem and Block Layer participate in issuing the necessary I/O request to the underlying block device.

During a synchronous read(), the calling process may wait while the storage I/O completes. While that process is waiting, the CPU can execute other runnable tasks.

5. Data Returns to the Application

Data retrieved from storage is normally placed in the Page Cache and then made available to the application’s buffer. A later read of the same data may therefore be satisfied from memory and complete much faster.

How File Writes Work

1. The Application Calls write()

The application calls write() with a file descriptor, a buffer containing the data, and the number of bytes to write.

The request enters the kernel and is processed through the VFS and the target filesystem.

2. Data Is Written to the Page Cache

With normal buffered I/O, data passed to write() is normally copied into the Page Cache first.

Memory pages that contain modified file data that has not yet been written to storage are called dirty pages.

3. write() Can Return Before Storage I/O Completes

A successful write() does not necessarily mean that the data has already reached persistent storage.

With normal buffered I/O, write() can return after the data has been copied into kernel-managed memory while the actual storage write occurs later.

This distinction is important when analyzing both application performance and data durability.

4. Dirty Pages Are Written Back to Storage

The Linux kernel later writes dirty pages to storage through a mechanism called writeback.

Separating the application’s write() from the physical storage operation allows Linux to buffer and combine writes, which can improve I/O efficiency.

5. fsync() and Data Persistence

Applications that require data to be persisted can use operations such as fsync() or fdatasync().

These calls request that relevant buffered data be synchronized with storage. However, actual durability can also depend on the filesystem, storage-device caches, controllers, connection method, and storage configuration.

The Roles of the VFS and Filesystem

What Is the VFS?

VFS stands for Virtual File System. It is an abstraction layer in the Linux kernel that provides a common interface for filesystem operations.

Applications can use operations such as open(), read(), write(), and close() without having to implement different logic for every filesystem.

The VFS represents filesystem information using common objects such as:

• inode: Represents filesystem objects and their metadata
• file: Represents an opened file
• dentry: Represents directory entries used for pathname lookup
• superblock: Represents a mounted filesystem

Relationship with ext4 and XFS

ext4 and XFS are concrete filesystems that implement filesystem-specific operations behind the common VFS interface.

If a target file resides on ext4, the request is handled by the corresponding ext4 implementation. If it resides on XFS, XFS-specific code performs the filesystem processing.

This abstraction allows applications to use largely the same system-call interface across different filesystems.

The Role of the Page Cache

What Is the Page Cache?

The Page Cache is a Linux kernel mechanism that keeps file data in memory.

It can contain file data that has previously been read from storage as well as modified data waiting to be written to storage.

Because RAM is much faster than persistent storage, using available memory as a cache can greatly reduce the number of physical storage accesses.

Page Cache and Linux Memory

Linux actively uses available memory for caching instead of leaving it completely unused.

When applications require additional memory, clean and reclaimable cache pages can be released and the memory reused. Therefore, a small amount of completely free memory does not by itself indicate that the system is running out of memory.

read(), write(), and the Page Cache

Operation	Role of the Page Cache
read()	Cached data can be returned from memory. On a cache miss, data is retrieved from storage and normally placed in the Page Cache.
write()	With buffered I/O, data is normally written to the Page Cache first, becomes dirty, and is later written to storage through writeback.

For reads, the Page Cache can reduce latency by avoiding storage access. For buffered writes, it allows write() to complete without waiting for every physical storage write to finish immediately.

Buffered I/O and Direct I/O

The I/O flow described so far mainly represents normal buffered I/O. Linux also supports Direct I/O, which is commonly requested using O_DIRECT.

Buffered I/O

With buffered I/O, file data normally passes through the Page Cache.

Application
│
▼
Page Cache
│
▼
Filesystem / Block Layer
│
▼
Storage

This allows Linux to cache reads and buffer writes, reducing the number of immediate storage operations.

Direct I/O

Direct I/O generally bypasses the Page Cache for file data and transfers data between application memory and storage without using the normal buffered-I/O cache path.

Application
│
▼
Filesystem / Block Layer
│
▼
Storage

Direct I/O can be useful for applications such as database systems that perform their own caching and I/O management. Its exact behavior and requirements depend on the filesystem, kernel, device, and I/O method being used.

The main difference between Buffered I/O and Direct I/O is who manages caching. With Buffered I/O, Linux uses the Page Cache, while Direct I/O allows applications to bypass the Page Cache for file data.

Direct I/O is often used by database systems that maintain their own buffer caches. This can help avoid caching the same data in both the database cache and the Linux Page Cache. However, the appropriate I/O method depends on the application, filesystem, and storage configuration.

The Role of Block I/O

What Is a Block Device?

A block device provides random access to data in addressable blocks. Typical examples include HDDs, SATA SSDs, NVMe devices, SAN LUNs, and virtual disks.

Linux exposes block devices through device files such as /dev/sda and /dev/nvme0n1.

Applications usually access files through a filesystem rather than directly specifying blocks on the device. The filesystem maps file data to the underlying storage.

From the Filesystem to the Block Layer

When storage access is required, block I/O requests are passed toward the underlying block device through the Linux Block Layer.

The Block Layer provides common infrastructure for block I/O requests and their queues. Modern Linux uses the multi-queue block layer (blk-mq), which is designed to support parallel I/O processing efficiently on modern storage devices.

Depending on the device and I/O scheduler configuration, Linux can also perform operations such as request merging and scheduling before requests reach the device driver.

From the Device Driver to Storage

The Role of the Device Driver

The device driver connects the Linux kernel’s block I/O mechanisms with the actual storage device or controller.

It handles device-specific operations such as submitting commands, managing queues, processing errors, and handling I/O completion.

HDD, SATA SSD, and NVMe SSD

An HDD stores data on rotating magnetic platters. Mechanical seek time and rotational latency make random I/O relatively slow.

A SATA SSD uses NAND flash and has no mechanical seek operation, providing much lower latency than an HDD. However, communication is performed through the SATA interface.

An NVMe SSD also uses NAND flash but communicates through PCI Express using the NVMe protocol. NVMe is designed for parallel I/O with multiple queues and can provide high throughput and low latency.

DMA and Data Transfer

Storage controllers generally use DMA (Direct Memory Access) to transfer data between devices and memory. The CPU and operating system set up and manage the I/O operation, while DMA allows bulk data transfer without requiring the CPU to copy every byte itself.

How I/O Completion Returns to the Application

When storage completes an I/O request, completion is communicated back through the device and kernel I/O stack.

For a synchronous read that had to wait for storage, the waiting process can become runnable after the required data is available. The scheduler eventually assigns CPU time to the process again, allowing read() to complete and return control to the application.

This means that application-visible I/O latency can include not only storage-device latency but also time spent waiting in queues and waiting for the process to run again.

For buffered writes, the process that called write() and the later kernel activity that writes dirty pages to storage do not necessarily execute at the same time. This asynchronous writeback behavior is an important consideration when analyzing write performance.

Conclusion

Linux file I/O involves multiple layers between an application and physical storage.

Application
│
▼
System Call
│
▼
VFS
│
▼
Filesystem
│
▼
Page Cache
│
▼
Block Layer
│
▼
Device Driver
│
▼
Storage

For reads, Linux can return cached file data from memory without accessing storage. If the data is not cached, Linux issues storage I/O and normally places the retrieved data in the Page Cache.

For buffered writes, data is normally placed in the Page Cache first and becomes dirty. The kernel later writes those dirty pages to storage through writeback. Therefore, successful completion of write() does not necessarily mean that the data has already been persisted.

It is also important to distinguish file I/O from physical storage I/O and buffered I/O from Direct I/O. The actual path depends on the application’s I/O method and the underlying Linux storage stack.

When investigating Linux I/O performance, do not treat the path from application to storage as a single operation. Breaking it down into the Page Cache, filesystem, Block Layer, device driver, and storage makes it much easier to identify where latency and waiting occur.