1                NO_HZ: Reducing Scheduling-Clock Ticks
   4This document describes Kconfig options and boot parameters that can
   5reduce the number of scheduling-clock interrupts, thereby improving energy
   6efficiency and reducing OS jitter.  Reducing OS jitter is important for
   7some types of computationally intensive high-performance computing (HPC)
   8applications and for real-time applications.
  10There are three main ways of managing scheduling-clock interrupts
  11(also known as "scheduling-clock ticks" or simply "ticks"):
  131.      Never omit scheduling-clock ticks (CONFIG_HZ_PERIODIC=y or
  14        CONFIG_NO_HZ=n for older kernels).  You normally will -not-
  15        want to choose this option.
  172.      Omit scheduling-clock ticks on idle CPUs (CONFIG_NO_HZ_IDLE=y or
  18        CONFIG_NO_HZ=y for older kernels).  This is the most common
  19        approach, and should be the default.
  213.      Omit scheduling-clock ticks on CPUs that are either idle or that
  22        have only one runnable task (CONFIG_NO_HZ_FULL=y).  Unless you
  23        are running realtime applications or certain types of HPC
  24        workloads, you will normally -not- want this option.
  26These three cases are described in the following three sections, followed
  27by a third section on RCU-specific considerations and a fourth and final
  28section listing known issues.
  33Very old versions of Linux from the 1990s and the very early 2000s
  34are incapable of omitting scheduling-clock ticks.  It turns out that
  35there are some situations where this old-school approach is still the
  36right approach, for example, in heavy workloads with lots of tasks
  37that use short bursts of CPU, where there are very frequent idle
  38periods, but where these idle periods are also quite short (tens or
  39hundreds of microseconds).  For these types of workloads, scheduling
  40clock interrupts will normally be delivered any way because there
  41will frequently be multiple runnable tasks per CPU.  In these cases,
  42attempting to turn off the scheduling clock interrupt will have no effect
  43other than increasing the overhead of switching to and from idle and
  44transitioning between user and kernel execution.
  46This mode of operation can be selected using CONFIG_HZ_PERIODIC=y (or
  47CONFIG_NO_HZ=n for older kernels).
  49However, if you are instead running a light workload with long idle
  50periods, failing to omit scheduling-clock interrupts will result in
  51excessive power consumption.  This is especially bad on battery-powered
  52devices, where it results in extremely short battery lifetimes.  If you
  53are running light workloads, you should therefore read the following
  56In addition, if you are running either a real-time workload or an HPC
  57workload with short iterations, the scheduling-clock interrupts can
  58degrade your applications performance.  If this describes your workload,
  59you should read the following two sections.
  64If a CPU is idle, there is little point in sending it a scheduling-clock
  65interrupt.  After all, the primary purpose of a scheduling-clock interrupt
  66is to force a busy CPU to shift its attention among multiple duties,
  67and an idle CPU has no duties to shift its attention among.
  69The CONFIG_NO_HZ_IDLE=y Kconfig option causes the kernel to avoid sending
  70scheduling-clock interrupts to idle CPUs, which is critically important
  71both to battery-powered devices and to highly virtualized mainframes.
  72A battery-powered device running a CONFIG_HZ_PERIODIC=y kernel would
  73drain its battery very quickly, easily 2-3 times as fast as would the
  74same device running a CONFIG_NO_HZ_IDLE=y kernel.  A mainframe running
  751,500 OS instances might find that half of its CPU time was consumed by
  76unnecessary scheduling-clock interrupts.  In these situations, there
  77is strong motivation to avoid sending scheduling-clock interrupts to
  78idle CPUs.  That said, dyntick-idle mode is not free:
  801.      It increases the number of instructions executed on the path
  81        to and from the idle loop.
  832.      On many architectures, dyntick-idle mode also increases the
  84        number of expensive clock-reprogramming operations.
  86Therefore, systems with aggressive real-time response constraints often
  87run CONFIG_HZ_PERIODIC=y kernels (or CONFIG_NO_HZ=n for older kernels)
  88in order to avoid degrading from-idle transition latencies.
  90An idle CPU that is not receiving scheduling-clock interrupts is said to
  91be "dyntick-idle", "in dyntick-idle mode", "in nohz mode", or "running
  92tickless".  The remainder of this document will use "dyntick-idle mode".
  94There is also a boot parameter "nohz=" that can be used to disable
  95dyntick-idle mode in CONFIG_NO_HZ_IDLE=y kernels by specifying "nohz=off".
  96By default, CONFIG_NO_HZ_IDLE=y kernels boot with "nohz=on", enabling
  97dyntick-idle mode.
 102If a CPU has only one runnable task, there is little point in sending it
 103a scheduling-clock interrupt because there is no other task to switch to.
 104Note that omitting scheduling-clock ticks for CPUs with only one runnable
 105task implies also omitting them for idle CPUs.
 107The CONFIG_NO_HZ_FULL=y Kconfig option causes the kernel to avoid
 108sending scheduling-clock interrupts to CPUs with a single runnable task,
 109and such CPUs are said to be "adaptive-ticks CPUs".  This is important
 110for applications with aggressive real-time response constraints because
 111it allows them to improve their worst-case response times by the maximum
 112duration of a scheduling-clock interrupt.  It is also important for
 113computationally intensive short-iteration workloads:  If any CPU is
 114delayed during a given iteration, all the other CPUs will be forced to
 115wait idle while the delayed CPU finishes.  Thus, the delay is multiplied
 116by one less than the number of CPUs.  In these situations, there is
 117again strong motivation to avoid sending scheduling-clock interrupts.
 119By default, no CPU will be an adaptive-ticks CPU.  The "nohz_full="
 120boot parameter specifies the adaptive-ticks CPUs.  For example,
 121"nohz_full=1,6-8" says that CPUs 1, 6, 7, and 8 are to be adaptive-ticks
 122CPUs.  Note that you are prohibited from marking all of the CPUs as
 123adaptive-tick CPUs:  At least one non-adaptive-tick CPU must remain
 124online to handle timekeeping tasks in order to ensure that system calls
 125like gettimeofday() returns accurate values on adaptive-tick CPUs.
 126(This is not an issue for CONFIG_NO_HZ_IDLE=y because there are no
 127running user processes to observe slight drifts in clock rate.)
 128Therefore, the boot CPU is prohibited from entering adaptive-ticks
 129mode.  Specifying a "nohz_full=" mask that includes the boot CPU will
 130result in a boot-time error message, and the boot CPU will be removed
 131from the mask.
 133Alternatively, the CONFIG_NO_HZ_FULL_ALL=y Kconfig parameter specifies
 134that all CPUs other than the boot CPU are adaptive-ticks CPUs.  This
 135Kconfig parameter will be overridden by the "nohz_full=" boot parameter,
 136so that if both the CONFIG_NO_HZ_FULL_ALL=y Kconfig parameter and
 137the "nohz_full=1" boot parameter is specified, the boot parameter will
 138prevail so that only CPU 1 will be an adaptive-ticks CPU.
 140Finally, adaptive-ticks CPUs must have their RCU callbacks offloaded.
 141This is covered in the "RCU IMPLICATIONS" section below.
 143Normally, a CPU remains in adaptive-ticks mode as long as possible.
 144In particular, transitioning to kernel mode does not automatically change
 145the mode.  Instead, the CPU will exit adaptive-ticks mode only if needed,
 146for example, if that CPU enqueues an RCU callback.
 148Just as with dyntick-idle mode, the benefits of adaptive-tick mode do
 149not come for free:
 1511.      CONFIG_NO_HZ_FULL selects CONFIG_NO_HZ_COMMON, so you cannot run
 152        adaptive ticks without also running dyntick idle.  This dependency
 153        extends down into the implementation, so that all of the costs
 154        of CONFIG_NO_HZ_IDLE are also incurred by CONFIG_NO_HZ_FULL.
 1562.      The user/kernel transitions are slightly more expensive due
 157        to the need to inform kernel subsystems (such as RCU) about
 158        the change in mode.
 1603.      POSIX CPU timers on adaptive-tick CPUs may miss their deadlines
 161        (perhaps indefinitely) because they currently rely on
 162        scheduling-tick interrupts.  This will likely be fixed in
 163        one of two ways: (1) Prevent CPUs with POSIX CPU timers from
 164        entering adaptive-tick mode, or (2) Use hrtimers or other
 165        adaptive-ticks-immune mechanism to cause the POSIX CPU timer to
 166        fire properly.
 1684.      If there are more perf events pending than the hardware can
 169        accommodate, they are normally round-robined so as to collect
 170        all of them over time.  Adaptive-tick mode may prevent this
 171        round-robining from happening.  This will likely be fixed by
 172        preventing CPUs with large numbers of perf events pending from
 173        entering adaptive-tick mode.
 1755.      Scheduler statistics for adaptive-tick CPUs may be computed
 176        slightly differently than those for non-adaptive-tick CPUs.
 177        This might in turn perturb load-balancing of real-time tasks.
 1796.      The LB_BIAS scheduler feature is disabled by adaptive ticks.
 181Although improvements are expected over time, adaptive ticks is quite
 182useful for many types of real-time and compute-intensive applications.
 183However, the drawbacks listed above mean that adaptive ticks should not
 184(yet) be enabled by default.
 189There are situations in which idle CPUs cannot be permitted to
 190enter either dyntick-idle mode or adaptive-tick mode, the most
 191common being when that CPU has RCU callbacks pending.
 193The CONFIG_RCU_FAST_NO_HZ=y Kconfig option may be used to cause such CPUs
 194to enter dyntick-idle mode or adaptive-tick mode anyway.  In this case,
 195a timer will awaken these CPUs every four jiffies in order to ensure
 196that the RCU callbacks are processed in a timely fashion.
 198Another approach is to offload RCU callback processing to "rcuo" kthreads
 199using the CONFIG_RCU_NOCB_CPU=y Kconfig option.  The specific CPUs to
 200offload may be selected via several methods:
 2021.      One of three mutually exclusive Kconfig options specify a
 203        build-time default for the CPUs to offload:
 205        a.      The CONFIG_RCU_NOCB_CPU_NONE=y Kconfig option results in
 206                no CPUs being offloaded.
 208        b.      The CONFIG_RCU_NOCB_CPU_ZERO=y Kconfig option causes
 209                CPU 0 to be offloaded.
 211        c.      The CONFIG_RCU_NOCB_CPU_ALL=y Kconfig option causes all
 212                CPUs to be offloaded.  Note that the callbacks will be
 213                offloaded to "rcuo" kthreads, and that those kthreads
 214                will in fact run on some CPU.  However, this approach
 215                gives fine-grained control on exactly which CPUs the
 216                callbacks run on, along with their scheduling priority
 217                (including the default of SCHED_OTHER), and it further
 218                allows this control to be varied dynamically at runtime.
 2202.      The "rcu_nocbs=" kernel boot parameter, which takes a comma-separated
 221        list of CPUs and CPU ranges, for example, "1,3-5" selects CPUs 1,
 222        3, 4, and 5.  The specified CPUs will be offloaded in addition to
 223        any CPUs specified as offloaded by CONFIG_RCU_NOCB_CPU_ZERO=y or
 224        CONFIG_RCU_NOCB_CPU_ALL=y.  This means that the "rcu_nocbs=" boot
 225        parameter has no effect for kernels built with RCU_NOCB_CPU_ALL=y.
 227The offloaded CPUs will never queue RCU callbacks, and therefore RCU
 228never prevents offloaded CPUs from entering either dyntick-idle mode
 229or adaptive-tick mode.  That said, note that it is up to userspace to
 230pin the "rcuo" kthreads to specific CPUs if desired.  Otherwise, the
 231scheduler will decide where to run them, which might or might not be
 232where you want them to run.
 237o       Dyntick-idle slows transitions to and from idle slightly.
 238        In practice, this has not been a problem except for the most
 239        aggressive real-time workloads, which have the option of disabling
 240        dyntick-idle mode, an option that most of them take.  However,
 241        some workloads will no doubt want to use adaptive ticks to
 242        eliminate scheduling-clock interrupt latencies.  Here are some
 243        options for these workloads:
 245        a.      Use PMQOS from userspace to inform the kernel of your
 246                latency requirements (preferred).
 248        b.      On x86 systems, use the "idle=mwait" boot parameter.
 250        c.      On x86 systems, use the "intel_idle.max_cstate=" to limit
 251        `       the maximum C-state depth.
 253        d.      On x86 systems, use the "idle=poll" boot parameter.
 254                However, please note that use of this parameter can cause
 255                your CPU to overheat, which may cause thermal throttling
 256                to degrade your latencies -- and that this degradation can
 257                be even worse than that of dyntick-idle.  Furthermore,
 258                this parameter effectively disables Turbo Mode on Intel
 259                CPUs, which can significantly reduce maximum performance.
 261o       Adaptive-ticks slows user/kernel transitions slightly.
 262        This is not expected to be a problem for computationally intensive
 263        workloads, which have few such transitions.  Careful benchmarking
 264        will be required to determine whether or not other workloads
 265        are significantly affected by this effect.
 267o       Adaptive-ticks does not do anything unless there is only one
 268        runnable task for a given CPU, even though there are a number
 269        of other situations where the scheduling-clock tick is not
 270        needed.  To give but one example, consider a CPU that has one
 271        runnable high-priority SCHED_FIFO task and an arbitrary number
 272        of low-priority SCHED_OTHER tasks.  In this case, the CPU is
 273        required to run the SCHED_FIFO task until it either blocks or
 274        some other higher-priority task awakens on (or is assigned to)
 275        this CPU, so there is no point in sending a scheduling-clock
 276        interrupt to this CPU.  However, the current implementation
 277        nevertheless sends scheduling-clock interrupts to CPUs having a
 278        single runnable SCHED_FIFO task and multiple runnable SCHED_OTHER
 279        tasks, even though these interrupts are unnecessary.
 281        And even when there are multiple runnable tasks on a given CPU,
 282        there is little point in interrupting that CPU until the current
 283        running task's timeslice expires, which is almost always way
 284        longer than the time of the next scheduling-clock interrupt.
 286        Better handling of these sorts of situations is future work.
 288o       A reboot is required to reconfigure both adaptive idle and RCU
 289        callback offloading.  Runtime reconfiguration could be provided
 290        if needed, however, due to the complexity of reconfiguring RCU at
 291        runtime, there would need to be an earthshakingly good reason.
 292        Especially given that you have the straightforward option of
 293        simply offloading RCU callbacks from all CPUs and pinning them
 294        where you want them whenever you want them pinned.
 296o       Additional configuration is required to deal with other sources
 297        of OS jitter, including interrupts and system-utility tasks
 298        and processes.  This configuration normally involves binding
 299        interrupts and tasks to particular CPUs.
 301o       Some sources of OS jitter can currently be eliminated only by
 302        constraining the workload.  For example, the only way to eliminate
 303        OS jitter due to global TLB shootdowns is to avoid the unmapping
 304        operations (such as kernel module unload operations) that
 305        result in these shootdowns.  For another example, page faults
 306        and TLB misses can be reduced (and in some cases eliminated) by
 307        using huge pages and by constraining the amount of memory used
 308        by the application.  Pre-faulting the working set can also be
 309        helpful, especially when combined with the mlock() and mlockall()
 310        system calls.
 312o       Unless all CPUs are idle, at least one CPU must keep the
 313        scheduling-clock interrupt going in order to support accurate
 314        timekeeping.
 316o       If there might potentially be some adaptive-ticks CPUs, there
 317        will be at least one CPU keeping the scheduling-clock interrupt
 318        going, even if all CPUs are otherwise idle.
 320        Better handling of this situation is ongoing work.
 322o       Some process-handling operations still require the occasional
 323        scheduling-clock tick.  These operations include calculating CPU
 324        load, maintaining sched average, computing CFS entity vruntime,
 325        computing avenrun, and carrying out load balancing.  They are
 326        currently accommodated by scheduling-clock tick every second
 327        or so.  On-going work will eliminate the need even for these
 328        infrequent scheduling-clock ticks.
 329 kindly hosted by Redpill Linpro AS, provider of Linux consulting and operations services since 1995.