Linux Interrupts and SoftIRQ Explained | IRQ, NAPI & Troubleshooting

Introduction

In Linux, the CPU must handle not only applications but also hardware-generated events, such as incoming network packets and the completion of storage I/O operations.

This is where interrupts come into play. When an interrupt occurs, the CPU temporarily pauses its current task, performs the required processing, and then returns to the original task.

Linux also handles interrupts efficiently by deferring time-consuming work to mechanisms such as SoftIRQ.

This article clearly explains how hardware interrupts and SoftIRQs work, how interrupt handling flows through the system, and how to inspect interrupts in Linux.

How Hardware Interrupts Work

A hardware interrupt is a mechanism through which a device, such as a NIC or storage device, tells the CPU that it requires attention.

For example, the CPU cannot know in advance when a NIC will receive a network packet. Devices such as NICs therefore use interrupts to notify the CPU when an event occurs.

The general flow is as follows.

The Role of the Interrupt Controller

In practice, an interrupt from a device is not always delivered directly to the CPU.

An interrupt controller manages interrupts from multiple devices and routes them to the appropriate CPU.

In simplified form, the relationship looks like this.

The interrupt controller receives device-generated interrupts and notifies a CPU according to the interrupt type and system configuration.

On a multicore CPU, which CPU handles an interrupt also matters. Linux lets you use IRQ affinity to control which CPU processes a particular interrupt.

What Is an IRQ?

A term that frequently appears when discussing Linux interrupts is IRQ (Interrupt Request). An IRQ is a request from a device or another source for the CPU to perform interrupt handling. Linux assigns IRQ numbers to identify and manage interrupts.

In Linux, you can inspect current interrupt activity in /proc/interrupts.

IRQ numbers appear on the left, followed by the interrupt counts for each CPU.

For example, the virtio0 row above shows that almost all NIC-related interrupts are being handled by CPU0.

The CPU Receives an Interrupt

When the interrupt controller signals an interrupt, the CPU transitions from normal instruction execution to interrupt handling.

For example, if the CPU receives an interrupt from the NIC while running an application in user mode, the conceptual flow is as follows.

The CPU saves the execution state needed to return to the original task, then transfers control to the kernel’s interrupt-handling code.

The Interrupt Handler Processes the Interrupt

When the CPU receives an interrupt, the kernel executes the corresponding handler. The component responsible for this device-specific work is generally called an interrupt handler. For a NIC, the device driver’s interrupt handler runs.

An interrupt handler performs only the work required immediately after the interrupt and finishes as quickly as possible. The division of responsibilities with deferred processing is explained in the next section, Top Half / Bottom Half.

Linux Interrupt Handling: Top Half and Bottom Half

When a hardware interrupt occurs, Linux does not perform all required work immediately. Spending too long on interrupt-related work could delay other interrupts and normal tasks.

It is therefore helpful to divide Linux interrupt handling conceptually into the Top Half (urgent work) and the Bottom Half (work that can be deferred).

What Is the Top Half?

The Top Half runs immediately after a hardware interrupt is received. The device’s interrupt handler performs the work that must be completed right away.

For a NIC, receiving a packet generates an interrupt. The Linux kernel runs the NIC driver’s interrupt handler and performs the minimum work required for that interrupt.

The key point is that the Top Half should avoid time-consuming work whenever possible.

Because lengthy interrupt handling can monopolize the CPU and affect other work, tasks that can wait are deferred to the Bottom Half.

What Is the Bottom Half?

The Bottom Half is a mechanism for performing work later when it does not need to run immediately after an interrupt.

Modern Linux uses mechanisms such as SoftIRQ to implement Bottom Half processing.

The following diagram shows the high-level relationship between immediate interrupt handling and deferred network processing. A more detailed receive flow appears later in this article.

Separating immediate interrupt handling from deferred work prevents the CPU from being tied up in interrupt context for too long.

Differences Between the Top Half and Bottom Half

The differences can be summarized as follows.

Category	Top Half	Bottom Half
Execution timing	Immediately after a hardware interrupt	After the Top Half
Primary role	Urgent processing	Deferrable processing
Key point	Finish quickly	Handle relatively heavy deferred work
Linux example	IRQ handler	SoftIRQ, etc.

Note that Top Half and Bottom Half are conceptual terms used to explain how processing is divided. In modern Linux, Bottom Half processing is implemented using mechanisms such as SoftIRQ.

SoftIRQs are particularly important for network processing. In environments handling large numbers of packets, CPU %soft time and ksoftirqd CPU usage may increase.

What Is SoftIRQ?

SoftIRQ is a Linux kernel mechanism for deferred processing, allowing work that does not need to be completed in the hardware interrupt handler to be processed later. It is commonly used for Bottom Half processing such as network receive processing.

Differences Between Hardware Interrupts and SoftIRQs

The differences between hardware interrupts and SoftIRQs are summarized below.

Type	Primary role
Hardware Interrupt	Notify the CPU of a hardware event
SoftIRQ	Perform deferred work inside the kernel

Types of SoftIRQ

SoftIRQs are divided into several types according to their purpose. In Linux, you can view current SoftIRQ event and processing counts in /proc/softirqs.

Common types include the following.

SoftIRQ	Primary function
NET_RX	Network receive processing
NET_TX	Network transmit processing
TIMER	Timer-related processing
BLOCK	Block-device processing
SCHED	Scheduler-related processing

When investigating network performance, NET_RX and NET_TX are especially useful.

Network Reception and NET_RX_SOFTIRQ

Linux uses NAPI for network packet processing. In its basic receive path, a device interrupt schedules a NAPI instance, whose poll method can then process multiple packets efficiently. The name NAPI is historical and no longer stands for a particular phrase.

After NAPI is scheduled, drivers typically keep interrupts for the corresponding queue masked while NAPI Poll processes packets in batches. This reduces unnecessary interrupt overhead, especially under high network load.

In the basic NAPI receive path, interrupts notify the host of new work and NAPI polling performs the packet processing. NAPI can also be invoked through mechanisms such as busy polling, so an interrupt is not required for every NAPI poll.

What Is ksoftirqd?

SoftIRQs can run in interrupt-related kernel context, including after hardware-interrupt handling. If SoftIRQ work remains pending or consumes too much processing time, Linux can defer further work so that normal tasks are not indefinitely starved.

Linux provides per-CPU ksoftirqd kernel threads that can process pending SoftIRQ work when it is deferred from immediate processing. On a four-CPU system, for example, you can see the following threads.

A ksoftirqd thread exists for each CPU. If SoftIRQ work cannot be completed immediately, the remaining work may be handled by that CPU’s ksoftirqd.

Under heavy network load, if ksoftirqd/0 CPU usage rises while NET_RX counts and %soft also increase, CPU0 may be spending a significant amount of CPU time on SoftIRQ work such as network reception.

SoftIRQ processing consumes CPU time, so under high load, %soft and ksoftirqd CPU usage may rise. The second half of this article explains how to check it.

Interrupt Handling Through a Network Receive Example

So far, we have covered hardware interrupts, the Top Half and Bottom Half, and SoftIRQs. Now let’s see how these mechanisms work together when a NIC receives a network packet.

1. The NIC Receives a Packet

When an Ethernet frame arrives from an external source, the NIC receives the data.

At this point, the application has not received the packet directly. Receive processing begins with the NIC and its device driver.

2. Transfer Data to Memory with DMA

Data received by the NIC is generally transferred via DMA (Direct Memory Access) to a receive buffer in main memory. More precisely, the NIC uses DMA to write to a receive buffer referenced by a descriptor in the RX ring, but this explanation is simplified to focus on interrupt handling.

DMA eliminates the need for the CPU to copy packet data from the NIC byte by byte. The CPU handles subsequent packet processing rather than the transfer itself.

3. A Hardware Interrupt Occurs

After receiving a packet, the NIC uses a hardware interrupt to notify the CPU that the packet requires processing.

Real systems may use interrupt coalescing* to report multiple packets together, so one packet does not necessarily equal one interrupt.

* Interrupt coalescing lets a NIC notify the CPU once for a group of packets instead of generating an interrupt for every received packet.

4. Run the NIC Driver’s Interrupt Handler

When the CPU receives an interrupt from the NIC, the NIC driver’s interrupt handler runs. It does not perform all packet processing at this stage.

To avoid spending too long in interrupt context, the handler performs only the minimum required work and hands subsequent receive processing to NAPI.

5. Schedule NAPI

In the normal non-threaded receive path, when the NIC driver’s interrupt handler schedules NAPI, the NAPI instance becomes eligible to run from networking SoftIRQ processing, where its poll method is invoked.

In a typical NIC driver, interrupts for the target queue are kept masked after NAPI is scheduled, and the NAPI poll method processes packets in a batch. Interrupts are normally re-enabled after polling completes.

This prevents a flood of hardware interrupts under heavy load.

6. NET_RX_SOFTIRQ Runs

In the normal non-threaded receive path, NAPI receive processing runs from networking SoftIRQ context (NET_RX_SOFTIRQ). From this point onward, packet processing is deferred work rather than immediate hardware-interrupt handling.

In a high-network-load environment, NET_RX processing counts and CPU %soft time may increase significantly.

7. Retrieve Packets with NAPI Poll

NET_RX_SOFTIRQ invokes NAPI’s poll routine.

NAPI Poll processes multiple received packets within a configured limit called the budget. Even when many packets arrive, it can process them in batches instead of handling a hardware interrupt for every packet.

8. Process Packets in the Linux Network Stack

Packets retrieved by NAPI are passed to the Linux network stack, where they are processed according to protocols such as Ethernet, IP, and TCP/UDP.

For TCP traffic, processing generally proceeds from Ethernet to IP to TCP. The destination IP address, port number, and other information identify the appropriate socket, and the received data is placed in that socket’s receive queue.

9. Store Data in the Socket Receive Buffer

After transport-layer processing, the received data is queued for the target socket and remains in kernel memory until the application reads it.

The data is still not automatically copied into the application’s user space. It remains in the kernel until the application reads it.

10. The Application Retrieves the Data

When the application calls a system call such as recv() or read(), data in the socket receive buffer is copied to user space and becomes available to the application.

Distinguishing Interrupts from SoftIRQs

A particularly important point in network receive processing is that hardware interrupts and SoftIRQs are separate processing stages.

In the first stage, the NIC reports an event to the CPU through a hardware interrupt. In the second stage, NAPI and NET_RX_SOFTIRQ perform the actual packet processing.

Understanding this mechanism makes it easier to interpret /proc/interrupts, /proc/softirqs, %irq, %soft, and ksoftirqd when network load rises on Linux.

Checking Interrupts in Linux

Linux exposes hardware-interrupt and SoftIRQ activity through files under /proc and commands such as top and mpstat. To investigate interrupt-related CPU load, check the following information.

• /proc/interrupts: hardware interrupts
• /proc/softirqs: SoftIRQs
• top: overall CPU hi and si
• mpstat: per-CPU %irq and %soft

Check Hardware Interrupts with /proc/interrupts

You can inspect hardware-interrupt activity in /proc/interrupts.

The CPU0 through CPU3 columns show the cumulative number of interrupts handled by each CPU.

The row corresponding to virtio, for example, shows which CPU handles interrupts generated by the NIC.

NIC interrupt counts do not necessarily match the number of received packets because interrupt coalescing and NAPI may process multiple packets together.

If interrupt counts rise sharply on only one CPU, interrupt processing may be concentrated on that CPU.

Check SoftIRQs with /proc/softirqs

You can inspect SoftIRQ event and processing counts in /proc/softirqs.

When investigating network reception, pay particular attention to NET_RX.

These are the cumulative NET_RX SoftIRQ counts processed by each CPU. NET_RX increases as network load rises, but a large value alone does not prove that current load is high because the counters are cumulative. It is important to compare changes over a fixed interval.

The NET_RX value is not a received-packet counter. It counts NET_RX SoftIRQ invocations, so it should not be compared directly with packet counters from the NIC or network stack.

Check irq / softirq CPU Usage with top

The top command provides a quick view of how much CPU time interrupt handling consumes.

When examining interrupt processing, focus mainly on hi and si.

Category	Meaning
hi	CPU time used for hardware-interrupt handling
si	CPU time used for SoftIRQ processing

In the displayed CPU statistics, 7.0 hi and 9.6 si indicate that approximately 7.0% and 9.6% of CPU time are being spent on hardware-interrupt and SoftIRQ processing, respectively.

Under very heavy network load, si may rise significantly.

Check Per-CPU Interrupt Load with mpstat

While top provides an overview of the entire CPU, mpstat is useful for identifying which CPU has a high interrupt-processing load.

Example output is shown below.

Check these fields in particular.

• %irq: CPU time used for hardware-interrupt handling
• %soft: CPU time used for SoftIRQ processing

In this example, %soft is higher on CPU1 and CPU3 than on the other CPUs.

Check ksoftirqd CPU Usage

When SoftIRQ load is high, use top to inspect ksoftirqd resource usage as well.

If ksoftirqd CPU usage is high under heavy network load, also check NET_RX in /proc/softirqs and %soft in mpstat.

Correlate Multiple Metrics When Investigating Interrupts

When investigating interrupt-related CPU load, correlate multiple sources of information instead of relying on a single value.

For interrupt load caused by network reception, the following investigation flow is useful.

A high interrupt count alone does not indicate a performance problem. Interrupt and SoftIRQ processing naturally increase as network traffic grows.

The key point is that whether rising interrupt or SoftIRQ activity coincides with high CPU usage or degraded application performance.

Conclusion

This article explained how hardware interrupts and SoftIRQs work in Linux.

When Linux receives a hardware interrupt from a device such as a NIC, it performs only the minimum required work and defers subsequent processing to mechanisms such as SoftIRQ. For network reception in particular, NAPI and NET_RX_SOFTIRQ process large volumes of packets efficiently.

To investigate interrupt-related CPU load, correlate /proc/interrupts, /proc/softirqs, top, mpstat, and ksoftirqd.

Understanding these mechanisms makes it easier to isolate the cause of high %irq or %soft values.