35 Kernel arguments for low latency and high performance #

Remove kthread_cpus when using SUSE real-time kernel. This parameter controls on which CPUs kernel threads are created. It also controls which CPUs are allowed for PID 1 and for loading kernel modules (the kmod user-space helper). This parameter is not recognized and does not have any effect.

Isolate the CPU cores using isolcpus, nohz_full, rcu_nocbs, and irqaffinity. For a comprehensive list of CPU pinning techniques, refer to CPU Pinning on Host (Chapter 36, CPU Pinning on Host) chapter.

Add domain,nohz,managed_irq flags to isolcpus kernel argument. Without any flags, isolcpus is equivalent to specifying only the domain flag. This isolates the specified CPUs from scheduling, including kernel tasks. The nohz flag stops the scheduler tick on the specified CPUs (if only one task is runnable on a CPU), and the managed_irq flag avoids routing managed external (device) interrupts at the specified CPUs. Note that the IRQ lines of NVMe devices are fully managed by the kernel and will be routed to the non-isolated (housekeeping) cores as a consequence. For example, the command line provided at the end of this section will result in only four queues (plus an admin/control queue) allocated on the system:

for I in $(grep nvme0 /proc/interrupts | cut -d ':' -f1); do cat /proc/irq/${I}/effective_affinity_list; done | column
39      0       19      20      39

This behavior prevents any disruption caused by disk I/O to any time sensitive application running on the isolated cores, but might require attention and careful design for storage focused workloads.

Tune the ticks (kernel’s periodic timer interrupts):

Disable Machine Check Exception (MCE) handling by specifying mce=off. MCEs are hardware errors detected by the processor and disabling them can avoid noisy logs.

Add nowatchdog to disable the soft-lockup watchdog which is implemented as a timer running in the timer hard-interrupt context. When it expires (i.e. a soft lockup is detected), it will print a warning (in the hard interrupt context), running any latency targets. Even if it never expires, it goes onto the timer list, slightly increasing the overhead of every timer interrupt. This option also disables the NMI watchdog, so NMIs cannot interfere.

nmi_watchdog=0 disables the NMI (Non-Maskable Interrupt) watchdog. This can be omitted when nowatchdog is used.

RCU (Read-Copy-Update) is a kernel mechanism that enables concurrent, lock-free access for many readers to shared data. An RCU callback, a function triggered after a 'grace period', ensures all previous readers have finished so old data can be safely reclaimed. We fine-tune RCU, particularly for sensitive workloads, to offload these callbacks from dedicated (pinned) CPUs, preventing kernel operations from interfering with critical, time-sensitive tasks.

Add ignition.platform.id=openstack for Metal3 and Cluster API to successfully provision/deprovision the cluster. This is used by Metal3 Python agent, which originated from Openstack Ironic.

Enable Predictable Network Interface Naming via net.ifnames=1. From SUSE Linux Micro 6.2 onwards, this is enabled by default and explicit configuration is not required. For versions prior to 6.2, this must be explicitly set as a kernel argument. This aligns with the predictableNicNames configuration in the Management Cluster’s Metal³ Helm chart, which is required for Directed Network Provisioning to function correctly. Consistent interface naming is also critical when SR-IOV is utilized.

Remove intel_pstate=passive. This option configures intel_pstate to work with generic cpufreq governors, but to make this work, it disables hardware-managed P-states (HWP) as a side effect. To reduce the hardware latency, this option is not recommended for real-time workloads.

Replace intel_idle.max_cstate=0 processor.max_cstate=1 with idle=poll. To avoid C-State transitions, the idle=poll option is used to disable the C-State transitions and keep the CPU in the highest C-State. The intel_idle.max_cstate=0 option disables intel_idle, so acpi_idle is used, and acpi_idle.max_cstate=1 then sets max C-state for acpi_idle. On AMD64/Intel 64 architectures, the first ACPI C-State is always POLL, but it uses a poll_idle() function, which may introduce some tiny latency by reading the clock periodically, and restarting the main loop in do_idle() after a timeout (this also involves clearing and setting the TIF_POLL task flag). In contrast, idle=poll runs in a tight loop, busy-waiting for a task to be rescheduled. This minimizes the latency of exiting the idle state, but at the cost of keeping the CPU running at full speed in the idle thread.

Disable C1E in BIOS. This option is important to disable the C1E state in the BIOS to avoid the CPU from entering the C1E state when idle. The C1E state is a low-power state that can introduce latency when the CPU is idle.