[PORT FROM R2] nohz: delay going tickless under CPU load to favor deeper C states

[PORT FROM R2] nohz: delay going tickless under CPU load to favor deeper C states

BZ: 23293

While there is CPU load, continue the periodic tick in order to give
CPUidle another opportunity to pick a deeper C-state instead of
spending potentially long idle times in a shallow C-state.

Long winded version:

When going idle with a high load average, CPUidle menu governor will
decide to pick a shallow C-state since one of the guiding principles
of the menu governor is "The busier the system, the less impact of C
states is acceptable" (taken from cpuidle/governors/menu.c.)  That
makes perfect sense.

However, there are missed power-saving opportunities for bursty
workloads with long idle times (e.g. MP3 playback.)  Given such a
workload, because of the load average, CPUidle tends to pick a shallow
C-state.  Because we also go tickless, this shallow C-state is used
for the duration of the idle period.  If the idle period is long, a
deeper C state would've resulted in better power savings.

This patch delays going tickless when there is a load such that on the
next tick, the CPUidle governor will have another opportunity to to
pick a deeper C-state.  Since the system will have been idle for
potentially a full tick, the load average will drop and a deeper C
state will most likely be chosen.

Delaying NOHZ decisions until the load is zero improved the load
estimation on our ARM/OMAP4 platform where HZ=128 and increased the
time spent in deep C-states (~50% of idle time in C-states deeper than
C1).  A power saving of ~20mA at battery level is observed during MP3
playback on OMAP4/Blaze board.

Change-Id: I574a4a70ec90c19aef94fd7ebd4904b2cef94d30
Orig-Change-Id: I941465f66fa69e9e9a59d2de7211a8d6f519a715
Signed-off-by: Nicole Chalhoub <n-chalhoub@ti.com>
Signed-off-by: Vincent Bour <v-bour@ti.com>
Signed-off-by: Kevin Hilman <khilman@ti.com>
Signed-off-by: Sundar Iyer <sundar.iyer@intel.com>
Reviewed-on: http://android.intel.com:8080/35157
Reviewed-by: Martin, LoicX <loicx.martin@intel.com>
Reviewed-by: Gross, Mark <mark.gross@intel.com>
Tested-by: Martin, LoicX <loicx.martin@intel.com>
Reviewed-by: buildbot <buildbot@intel.com>
Tested-by: buildbot <buildbot@intel.com>