1 NO_HZ: Reducing Scheduling-Clock Ticks 2 3 4This document describes Kconfig options and boot parameters that can 5reduce the number of scheduling-clock interrupts, thereby improving energy 6efficiency and reducing OS jitter. Reducing OS jitter is important for 7some types of computationally intensive high-performance computing (HPC) 8applications and for real-time applications. 9 10There are three main ways of managing scheduling-clock interrupts 11(also known as "scheduling-clock ticks" or simply "ticks"): 12 131. Never omit scheduling-clock ticks (CONFIG_HZ_PERIODIC=y or 14 CONFIG_NO_HZ=n for older kernels). You normally will -not- 15 want to choose this option. 16 172. Omit scheduling-clock ticks on idle CPUs (CONFIG_NO_HZ_IDLE=y or 18 CONFIG_NO_HZ=y for older kernels). This is the most common 19 approach, and should be the default. 20 213. Omit scheduling-clock ticks on CPUs that are either idle or that 22 have only one runnable task (CONFIG_NO_HZ_FULL=y). Unless you 23 are running realtime applications or certain types of HPC 24 workloads, you will normally -not- want this option. 25 26These three cases are described in the following three sections, followed 27by a third section on RCU-specific considerations, a fourth section 28discussing testing, and a fifth and final section listing known issues. 29 30 31NEVER OMIT SCHEDULING-CLOCK TICKS 32 33Very old versions of Linux from the 1990s and the very early 2000s 34are incapable of omitting scheduling-clock ticks. It turns out that 35there are some situations where this old-school approach is still the 36right approach, for example, in heavy workloads with lots of tasks 37that use short bursts of CPU, where there are very frequent idle 38periods, but where these idle periods are also quite short (tens or 39hundreds of microseconds). For these types of workloads, scheduling 40clock interrupts will normally be delivered any way because there 41will frequently be multiple runnable tasks per CPU. In these cases, 42attempting to turn off the scheduling clock interrupt will have no effect 43other than increasing the overhead of switching to and from idle and 44transitioning between user and kernel execution. 45 46This mode of operation can be selected using CONFIG_HZ_PERIODIC=y (or 47CONFIG_NO_HZ=n for older kernels). 48 49However, if you are instead running a light workload with long idle 50periods, failing to omit scheduling-clock interrupts will result in 51excessive power consumption. This is especially bad on battery-powered 52devices, where it results in extremely short battery lifetimes. If you 53are running light workloads, you should therefore read the following 54section. 55 56In addition, if you are running either a real-time workload or an HPC 57workload with short iterations, the scheduling-clock interrupts can 58degrade your applications performance. If this describes your workload, 59you should read the following two sections. 60 61 62OMIT SCHEDULING-CLOCK TICKS FOR IDLE CPUs 63 64If a CPU is idle, there is little point in sending it a scheduling-clock 65interrupt. After all, the primary purpose of a scheduling-clock interrupt 66is to force a busy CPU to shift its attention among multiple duties, 67and an idle CPU has no duties to shift its attention among. 68 69The CONFIG_NO_HZ_IDLE=y Kconfig option causes the kernel to avoid sending 70scheduling-clock interrupts to idle CPUs, which is critically important 71both to battery-powered devices and to highly virtualized mainframes. 72A battery-powered device running a CONFIG_HZ_PERIODIC=y kernel would 73drain its battery very quickly, easily 2-3 times as fast as would the 74same device running a CONFIG_NO_HZ_IDLE=y kernel. A mainframe running 751,500 OS instances might find that half of its CPU time was consumed by 76unnecessary scheduling-clock interrupts. In these situations, there 77is strong motivation to avoid sending scheduling-clock interrupts to 78idle CPUs. That said, dyntick-idle mode is not free: 79 801. It increases the number of instructions executed on the path 81 to and from the idle loop. 82 832. On many architectures, dyntick-idle mode also increases the 84 number of expensive clock-reprogramming operations. 85 86Therefore, systems with aggressive real-time response constraints often 87run CONFIG_HZ_PERIODIC=y kernels (or CONFIG_NO_HZ=n for older kernels) 88in order to avoid degrading from-idle transition latencies. 89 90An idle CPU that is not receiving scheduling-clock interrupts is said to 91be "dyntick-idle", "in dyntick-idle mode", "in nohz mode", or "running 92tickless". The remainder of this document will use "dyntick-idle mode". 93 94There is also a boot parameter "nohz=" that can be used to disable 95dyntick-idle mode in CONFIG_NO_HZ_IDLE=y kernels by specifying "nohz=off". 96By default, CONFIG_NO_HZ_IDLE=y kernels boot with "nohz=on", enabling 97dyntick-idle mode. 98 99 100OMIT SCHEDULING-CLOCK TICKS FOR CPUs WITH ONLY ONE RUNNABLE TASK 101 102If a CPU has only one runnable task, there is little point in sending it 103a scheduling-clock interrupt because there is no other task to switch to. 104Note that omitting scheduling-clock ticks for CPUs with only one runnable 105task implies also omitting them for idle CPUs. 106 107The CONFIG_NO_HZ_FULL=y Kconfig option causes the kernel to avoid 108sending scheduling-clock interrupts to CPUs with a single runnable task, 109and such CPUs are said to be "adaptive-ticks CPUs". This is important 110for applications with aggressive real-time response constraints because 111it allows them to improve their worst-case response times by the maximum 112duration of a scheduling-clock interrupt. It is also important for 113computationally intensive short-iteration workloads: If any CPU is 114delayed during a given iteration, all the other CPUs will be forced to 115wait idle while the delayed CPU finishes. Thus, the delay is multiplied 116by one less than the number of CPUs. In these situations, there is 117again strong motivation to avoid sending scheduling-clock interrupts. 118 119By default, no CPU will be an adaptive-ticks CPU. The "nohz_full=" 120boot parameter specifies the adaptive-ticks CPUs. For example, 121"nohz_full=1,6-8" says that CPUs 1, 6, 7, and 8 are to be adaptive-ticks 122CPUs. Note that you are prohibited from marking all of the CPUs as 123adaptive-tick CPUs: At least one non-adaptive-tick CPU must remain 124online to handle timekeeping tasks in order to ensure that system 125calls like gettimeofday() returns accurate values on adaptive-tick CPUs. 126(This is not an issue for CONFIG_NO_HZ_IDLE=y because there are no running 127user processes to observe slight drifts in clock rate.) Therefore, the 128boot CPU is prohibited from entering adaptive-ticks mode. Specifying a 129"nohz_full=" mask that includes the boot CPU will result in a boot-time 130error message, and the boot CPU will be removed from the mask. Note that 131this means that your system must have at least two CPUs in order for 132CONFIG_NO_HZ_FULL=y to do anything for you. 133 134Finally, adaptive-ticks CPUs must have their RCU callbacks offloaded. 135This is covered in the "RCU IMPLICATIONS" section below. 136 137Normally, a CPU remains in adaptive-ticks mode as long as possible. 138In particular, transitioning to kernel mode does not automatically change 139the mode. Instead, the CPU will exit adaptive-ticks mode only if needed, 140for example, if that CPU enqueues an RCU callback. 141 142Just as with dyntick-idle mode, the benefits of adaptive-tick mode do 143not come for free: 144 1451. CONFIG_NO_HZ_FULL selects CONFIG_NO_HZ_COMMON, so you cannot run 146 adaptive ticks without also running dyntick idle. This dependency 147 extends down into the implementation, so that all of the costs 148 of CONFIG_NO_HZ_IDLE are also incurred by CONFIG_NO_HZ_FULL. 149 1502. The user/kernel transitions are slightly more expensive due 151 to the need to inform kernel subsystems (such as RCU) about 152 the change in mode. 153 1543. POSIX CPU timers prevent CPUs from entering adaptive-tick mode. 155 Real-time applications needing to take actions based on CPU time 156 consumption need to use other means of doing so. 157 1584. If there are more perf events pending than the hardware can 159 accommodate, they are normally round-robined so as to collect 160 all of them over time. Adaptive-tick mode may prevent this 161 round-robining from happening. This will likely be fixed by 162 preventing CPUs with large numbers of perf events pending from 163 entering adaptive-tick mode. 164 1655. Scheduler statistics for adaptive-tick CPUs may be computed 166 slightly differently than those for non-adaptive-tick CPUs. 167 This might in turn perturb load-balancing of real-time tasks. 168 1696. The LB_BIAS scheduler feature is disabled by adaptive ticks. 170 171Although improvements are expected over time, adaptive ticks is quite 172useful for many types of real-time and compute-intensive applications. 173However, the drawbacks listed above mean that adaptive ticks should not 174(yet) be enabled by default. 175 176 177RCU IMPLICATIONS 178 179There are situations in which idle CPUs cannot be permitted to 180enter either dyntick-idle mode or adaptive-tick mode, the most 181common being when that CPU has RCU callbacks pending. 182 183The CONFIG_RCU_FAST_NO_HZ=y Kconfig option may be used to cause such CPUs 184to enter dyntick-idle mode or adaptive-tick mode anyway. In this case, 185a timer will awaken these CPUs every four jiffies in order to ensure 186that the RCU callbacks are processed in a timely fashion. 187 188Another approach is to offload RCU callback processing to "rcuo" kthreads 189using the CONFIG_RCU_NOCB_CPU=y Kconfig option. The specific CPUs to 190offload may be selected using The "rcu_nocbs=" kernel boot parameter, 191which takes a comma-separated list of CPUs and CPU ranges, for example, 192"1,3-5" selects CPUs 1, 3, 4, and 5. 193 194The offloaded CPUs will never queue RCU callbacks, and therefore RCU 195never prevents offloaded CPUs from entering either dyntick-idle mode 196or adaptive-tick mode. That said, note that it is up to userspace to 197pin the "rcuo" kthreads to specific CPUs if desired. Otherwise, the 198scheduler will decide where to run them, which might or might not be 199where you want them to run. 200 201 202TESTING 203 204So you enable all the OS-jitter features described in this document, 205but do not see any change in your workload's behavior. Is this because 206your workload isn't affected that much by OS jitter, or is it because 207something else is in the way? This section helps answer this question 208by providing a simple OS-jitter test suite, which is available on branch 209master of the following git archive: 210 211git://git.kernel.org/pub/scm/linux/kernel/git/frederic/dynticks-testing.git 212 213Clone this archive and follow the instructions in the README file. 214This test procedure will produce a trace that will allow you to evaluate 215whether or not you have succeeded in removing OS jitter from your system. 216If this trace shows that you have removed OS jitter as much as is 217possible, then you can conclude that your workload is not all that 218sensitive to OS jitter. 219 220Note: this test requires that your system have at least two CPUs. 221We do not currently have a good way to remove OS jitter from single-CPU 222systems. 223 224 225KNOWN ISSUES 226 227o Dyntick-idle slows transitions to and from idle slightly. 228 In practice, this has not been a problem except for the most 229 aggressive real-time workloads, which have the option of disabling 230 dyntick-idle mode, an option that most of them take. However, 231 some workloads will no doubt want to use adaptive ticks to 232 eliminate scheduling-clock interrupt latencies. Here are some 233 options for these workloads: 234 235 a. Use PMQOS from userspace to inform the kernel of your 236 latency requirements (preferred). 237 238 b. On x86 systems, use the "idle=mwait" boot parameter. 239 240 c. On x86 systems, use the "intel_idle.max_cstate=" to limit 241 ` the maximum C-state depth. 242 243 d. On x86 systems, use the "idle=poll" boot parameter. 244 However, please note that use of this parameter can cause 245 your CPU to overheat, which may cause thermal throttling 246 to degrade your latencies -- and that this degradation can 247 be even worse than that of dyntick-idle. Furthermore, 248 this parameter effectively disables Turbo Mode on Intel 249 CPUs, which can significantly reduce maximum performance. 250 251o Adaptive-ticks slows user/kernel transitions slightly. 252 This is not expected to be a problem for computationally intensive 253 workloads, which have few such transitions. Careful benchmarking 254 will be required to determine whether or not other workloads 255 are significantly affected by this effect. 256 257o Adaptive-ticks does not do anything unless there is only one 258 runnable task for a given CPU, even though there are a number 259 of other situations where the scheduling-clock tick is not 260 needed. To give but one example, consider a CPU that has one 261 runnable high-priority SCHED_FIFO task and an arbitrary number 262 of low-priority SCHED_OTHER tasks. In this case, the CPU is 263 required to run the SCHED_FIFO task until it either blocks or 264 some other higher-priority task awakens on (or is assigned to) 265 this CPU, so there is no point in sending a scheduling-clock 266 interrupt to this CPU. However, the current implementation 267 nevertheless sends scheduling-clock interrupts to CPUs having a 268 single runnable SCHED_FIFO task and multiple runnable SCHED_OTHER 269 tasks, even though these interrupts are unnecessary. 270 271 And even when there are multiple runnable tasks on a given CPU, 272 there is little point in interrupting that CPU until the current 273 running task's timeslice expires, which is almost always way 274 longer than the time of the next scheduling-clock interrupt. 275 276 Better handling of these sorts of situations is future work. 277 278o A reboot is required to reconfigure both adaptive idle and RCU 279 callback offloading. Runtime reconfiguration could be provided 280 if needed, however, due to the complexity of reconfiguring RCU at 281 runtime, there would need to be an earthshakingly good reason. 282 Especially given that you have the straightforward option of 283 simply offloading RCU callbacks from all CPUs and pinning them 284 where you want them whenever you want them pinned. 285 286o Additional configuration is required to deal with other sources 287 of OS jitter, including interrupts and system-utility tasks 288 and processes. This configuration normally involves binding 289 interrupts and tasks to particular CPUs. 290 291o Some sources of OS jitter can currently be eliminated only by 292 constraining the workload. For example, the only way to eliminate 293 OS jitter due to global TLB shootdowns is to avoid the unmapping 294 operations (such as kernel module unload operations) that 295 result in these shootdowns. For another example, page faults 296 and TLB misses can be reduced (and in some cases eliminated) by 297 using huge pages and by constraining the amount of memory used 298 by the application. Pre-faulting the working set can also be 299 helpful, especially when combined with the mlock() and mlockall() 300 system calls. 301 302o Unless all CPUs are idle, at least one CPU must keep the 303 scheduling-clock interrupt going in order to support accurate 304 timekeeping. 305 306o If there might potentially be some adaptive-ticks CPUs, there 307 will be at least one CPU keeping the scheduling-clock interrupt 308 going, even if all CPUs are otherwise idle. 309 310 Better handling of this situation is ongoing work. 311 312o Some process-handling operations still require the occasional 313 scheduling-clock tick. These operations include calculating CPU 314 load, maintaining sched average, computing CFS entity vruntime, 315 computing avenrun, and carrying out load balancing. They are 316 currently accommodated by scheduling-clock tick every second 317 or so. On-going work will eliminate the need even for these 318 infrequent scheduling-clock ticks. 319
