Introduction
Applications running on Linux perform file I/O in many situations, such as reading configuration files, writing logs, and updating database files. We normally interact with files only through commands or programs, but behind the scenes the Linux kernel coordinates several mechanisms.
For example, calling read() does not necessarily cause data to be read from storage every time. If the required data is already in the Page Cache in memory, the read can complete without accessing storage. Likewise, a successful write() does not necessarily mean that the data has already been persisted to an SSD or HDD.
To understand this distinction, you need to view system calls such as open(), read(), write(), fsync(), and close() together with mechanisms such as File Descriptors, the VFS, filesystems, the Page Cache, Dirty Pages, and Writeback as one continuous flow.
This article explains what happens in Linux from opening a file through reading, writing, synchronizing, and closing it.
What Is File I/O in Linux?
File I/O Basics
File I/O is the process by which a program reads data from a file or writes data to a file. I/O stands for Input / Output: reading from a file is input, and writing to a file is output.
In Linux, not only regular files but also directories, terminals, pipes, sockets, and devices are handled through a common file-like interface. This is sometimes described as “Everything is a file,” but it does not mean that everything is identical to a regular disk file. Rather, these objects can use file descriptors and a common set of operations.
open / read / write / close
The following system calls are commonly used to work with files in Linux.
| System Call | Primary Role |
|---|---|
| open() / openat() | Open a file specified by a path and obtain a File Descriptor |
| read() | Read data from an open file |
| write() | Write data to an open file |
| close() | Close a File Descriptor and release the process reference to it |
First, the file is opened with open() or a related call. The returned File Descriptor is then passed to read() or write(). When processing is complete, close() closes the File Descriptor. This flow allows the kernel to track which process is operating on which file.
User Space and Kernel Space
Applications run in User Space, while the Linux kernel runs in Kernel Space. A User Space program cannot freely manipulate internal filesystem data or storage devices. This isolation prevents a process malfunction or unauthorized operation from affecting the entire system.
When an application performs file I/O, it requests the operation from the kernel through a system call. The CPU switches from user mode to kernel mode, and the kernel validates permissions and arguments before processing the request. When the operation finishes, a return value is sent to the application and execution resumes in user mode.
What Happens When You Open a File?
How open() Works
When open() is executed, the kernel resolves the file path, identifies the target file, and returns a File Descriptor that the application can use to operate on it.
The process is explained below by following the diagram.
1. Application
The application calls open() with a file path and open flags. For example, the following opens /var/log/app.log in read-only mode. A write operation would require a separately opened writable descriptor.
int fd = open(“/var/log/app.log”, O_RDONLY);
O_RDONLY is the flag that specifies that the file is opened for reading only.
2. Path Resolution (VFS)
After receiving open(), the kernel resolves the specified file path through the VFS (Virtual File System). For /var/log/app.log, it follows the path from the root directory through var, log, and app.log to identify the target file.
This path-resolution process uses objects such as dentries and inodes.
3. dentry
A dentry (directory entry) represents a component of a pathname and connects a name to its parent directory and, when the entry is positive, an inode. In this simplified diagram, the dentry for app.log refers to inode #12345.
Dentries are cached, which also speeds up path resolution for files and directories that have already been accessed.
4. inode
An inode manages file metadata. It primarily contains information such as the following.
- Owner
- Permissions
- File size
- Timestamps
- Information used to locate the file data
The filename is handled by the dentry, not by the inode. In the diagram, the app.log dentry corresponds to inode #12345.
5. struct file
When the target file is opened, the kernel manages the state of the open file in a struct file.
A struct file stores the current file offset, file status flags, and references to the path and file-operation functions. In other words, the inode describes the file itself, while the struct file describes the state of an open instance of that file.
6. File Descriptor
Finally, the kernel returns a File Descriptor to the application. A File Descriptor is an integer that a process uses to operate on an open file.
By convention, 0, 1, and 2 are used for standard I/O. A newly opened file is therefore normally assigned a File Descriptor of 3 or greater.
0 stdin Standard input
1 stdout Standard output
2 stderr Standard error
3 app.log
In the diagram, app.log is assigned fd = 3.
Because this example uses O_RDONLY, the returned descriptor can be used for read() but not write(). A separate descriptor opened with O_WRONLY or O_RDWR is required for writing.
read(3, buf, size);
close(3);
In this way, open() resolves a file path through the VFS and identifies the target file through its dentry and inode. The kernel then manages the state of the open file in a struct file and returns the File Descriptor used by the application.
This flow is simplified to make the main elements involved in open() easier to understand.
What Happens When You Read a File?
The read() System Call
After opening a file, the application uses read() to retrieve data. The read() flow is shown below.
1. Call read()
After opening the file, the application calls read() to read data.
ssize_t size = read(fd, buffer, sizeof(buffer));
read() accepts the File Descriptor (fd) that identifies the target file, the User Space buffer that will receive the data, and the maximum number of bytes to read.
2. Identify the File from the File Descriptor
The kernel looks up the specified File Descriptor in the process File Descriptor Table.
Through the File Descriptor, it references the struct file and determines the target file and current file offset.
With a normal read(), a successful operation advances the file offset by the number of bytes read. Repeated read() calls on the same File Descriptor therefore read subsequent portions of the file in order.
3. Check the Page Cache
After identifying the target file, normal Buffered I/O checks whether the required data exists in the Page Cache.
With a Cache Hit, the data is already in memory, so storage access is unnecessary.
If the required data is not in the Page Cache (a Cache Miss), the kernel normally retrieves it from storage.
4. Read from Storage on a Cache Miss
On a Cache Miss, the file position is mapped to blocks on storage and a read request is issued.
With a synchronous read(), the process may wait until the data arrives.
Data read from storage is stored in the Page Cache. For sequential access, Linux may also perform Readahead to fetch data likely to be needed next.
5. Copy the Data to the User Space Buffer
Once the required data is available in the Page Cache, the kernel copies it to the User Space buffer.
In other words, with a typical Buffered I/O read(), data is not transferred directly from storage to the application.
Storage → Page Cache → User Space Buffer
The data moves through this sequence.
6. Return the read() Result
Finally, read() returns the number of bytes actually read to the application.
- A value greater than 0: Number of bytes actually read
- 0: End of file (EOF)
- -1: Error
Because read() may return fewer bytes than requested, the application must check the return value from read().
A normal read() therefore follows this flow: identify the file from the File Descriptor → check the Page Cache → read from storage if necessary → copy to User Space.
What Happens When You Write to a File?
The write() System Call
The following diagram illustrates a typical buffered write to a regular file. It separates the application’s write() call from the later Writeback of Dirty Pages to storage.
1. Call write()
The application calls write() with a writable File Descriptor, a source buffer in User Space, and the number of bytes it wants to write.
ssize_t written = write(fd, buffer, sizeof(buffer));In this example, fd must have been opened with write permission, such as O_WRONLY or O_RDWR.
2. Identify the File from the File Descriptor
The kernel looks up fd in the process’s File Descriptor Table and follows it to the corresponding struct file, which maintains the open file’s state, including its current offset.
For an ordinary positional write to a regular file, a successful operation normally advances the offset by the number of bytes written. Flags such as O_APPEND can change how the write position is selected.
3. Prepare the Page Cache
For normal Buffered I/O, the kernel locates or prepares the Page Cache pages covering the target file range. If the required page is already cached, Linux can use it. If it is not cached, Linux prepares a page and may need to read existing file data first—for example, when updating only part of an existing page.
A Cache Miss therefore does not necessarily mean that the application’s new data is immediately written to storage.
4. Copy Data to the Page Cache
The kernel copies data from the User Space buffer into the relevant Page Cache pages. Modified pages are marked as Dirty Pages because their contents may not yet match the data on storage.
At this stage, the write may be accepted in memory without waiting for the SSD or HDD to finish writing the data.
5. Return the write() Result
On success, write() returns the number of bytes written; on error, it returns -1. A successful call can write fewer bytes than requested, so applications must check the return value and handle partial writes.
A buffered write() can return before the data is persistent. However, it is not guaranteed to return immediately: preparing pages, reading existing data, or throttling when too many Dirty Pages accumulate may cause the call to wait.
6. Write Dirty Pages to Storage (Writeback)
The kernel later writes Dirty Pages to storage through Writeback, passing requests through the filesystem, Block Layer, and device driver. Writeback may run independently of the application and may even start before write() returns; the diagram shows the typical conceptual separation, not a strict timing guarantee.
When an application must wait for its file changes to be synchronized, it can call fsync() or a similar function. Durability depends on the filesystem and underlying storage correctly honoring synchronization requests.
Key point: Successful write() completion does not necessarily mean the data has been persisted to storage.
How Buffered I/O Works
Why Use the Page Cache?
Memory access is generally faster than storage access. Linux uses the Page Cache to serve repeated reads from memory and to buffer file writes, reducing unnecessary storage I/O and allowing Writeback to occur separately from many application writes.
| Operation | Role of the Page Cache |
|---|---|
read() | Returns cached data without storage access on a Cache Hit. |
write() | Holds modified data as Dirty Pages until Writeback. |
Here, Buffered I/O refers to the Linux kernel Page Cache. It is distinct from User Space buffering in C stdio functions such as fread() and fwrite().
Memory and Dirty Pages
Linux uses available memory for the Page Cache. Clean cached pages can usually be reclaimed when memory is needed. Dirty Pages contain changes that generally need to be written back before their memory can be reclaimed.
Buffered I/O vs. Direct I/O
Buffered I/O normally uses the Linux Page Cache. Direct I/O generally bypasses the Page Cache for file data, allowing applications to manage their own caching. Databases with their own buffer caches are one example where Direct I/O may be useful to avoid duplicated caching. Direct I/O has alignment and filesystem-dependent constraints and is not automatically faster.
What Happens When You Run fsync()?
When Should You Call fsync()?
A successful buffered write() does not necessarily mean that the data has reached persistent storage. An application calls fsync() when it needs to wait for changes to a file—including metadata needed to retrieve its data—to be synchronized.
The kernel may write Dirty Pages to storage without an explicit fsync(). Calling fsync() asks the kernel to synchronize the target file’s changes and wait for completion. Frequent calls may increase I/O latency.
How fsync() Reaches Storage
The filesystem writes the required dirty data and metadata through the storage stack. A device with a volatile write cache may also need a flush or an equivalent operation before synchronization is complete.
The expected durability depends on the filesystem, mount configuration, virtualization layer, and device correctly honoring synchronization requests. If a new filename or rename must also survive a crash, synchronizing the parent directory may be necessary.
fdatasync() is an alternative that synchronizes file data and the metadata needed to retrieve it, while potentially omitting other metadata updates.
What Happens When You Close a File?
The close() System Call
After finishing with a file, the application uses close() to close its File Descriptor.
close(fd);
When close() runs, the kernel removes the specified File Descriptor from the File Descriptor Table and releases that descriptor’s reference to the struct file.
The overall flow is as follows.
1. Call close()
The application calls close() with a File Descriptor it no longer needs.
close(fd);
2. Locate the File Descriptor
The kernel looks up the specified File Descriptor in the process File Descriptor Table.
It identifies the corresponding struct file from the File Descriptor.
3. Detach the File Descriptor
The target File Descriptor is detached from the File Descriptor Table.
The descriptor no longer refers to this open file. Its numeric value may be reused by a later open(), so the application must not continue using the closed descriptor.
4. Release the Reference to struct file
Detaching the File Descriptor releases its reference to the struct file.
At this point, the same struct file may still be referenced by another File Descriptor or another source.
5. Check for Other References
If references to the same struct file remain, the struct file stays alive.
For example, a File Descriptor duplicated with dup() or a process created by fork() may reference the same struct file.
If no other references remain, the struct file is no longer needed.
6. Complete close()
If no other references remain, the struct file and associated internal kernel resources are released.
If other references remain, close() completes while leaving the struct file in place.
Thus, close() detaches the File Descriptor from the File Descriptor Table and releases its reference to the struct file. The struct file remains while other references exist and is released once no references remain.
close() and Data Persistence
close() and fsync() serve different purposes.
close() closes a File Descriptor; it is not a system call that guarantees data persistence to storage.
With normal Buffered I/O, data changed by write() may remain in the Page Cache as Dirty Pages. Even after close(), Writeback will later write those Dirty Pages to storage.
To synchronize important data before closing a file, call fsync() before close() and check both calls for errors. A close operation can also report delayed write errors, but it is not a substitute for fsync().
In short, close() detaches the File Descriptor from the File Descriptor Table and releases its reference to the struct file. When persistence is required, fsync() or a similar mechanism must be used separately from close().
Summary
This article explained Linux file I/O through the processing flows of open(), read(), write(), fsync(), and close().
In Linux, open() obtains a File Descriptor, and read() or write() follows that descriptor to a struct file to operate on the target file.
With normal Buffered I/O, the Page Cache mediates file data access. read() can return cached data without accessing storage. For write(), Linux prepares the necessary cache pages, copies data into them, and marks modified pages dirty. Writeback subsequently transfers dirty data to storage.
A successful write() therefore does not necessarily mean that persistence to storage is complete. Use fsync() or a similar mechanism when synchronization must be confirmed.
Finally, close() detaches the File Descriptor from the File Descriptor Table and releases the reference to the struct file. If no other references exist, the struct file is also released.
When investigating Linux file I/O, it is important to understand the entire flow from system calls through File Descriptors, struct file, the Page Cache, Dirty Pages, Writeback, and storage.
English 
Japanese
コメント