[PATCH] Optimize calc_load()

calc_load() is called by timer interrupt to update avenrun[].  It currently
calls nr_active() at each timer tick (HZ per second), while the update of
avenrun[] is done only once every 5 seconds.  (LOAD_FREQ=5 Hz)

nr_active() is quite expensive on SMP machines, since it has to sum up
nr_running and nr_uninterruptible of all online CPUS, bringing foreign
dirty cache lines.

This patch is an optimization of calc_load() so that nr_active() is called
only if we need it.

The use of unlikely() is welcome since the condition is true only once every
5*HZ time.

Signed-off-by: Eric Dumazet <dada1@cosmosbay.com>
Cc: Ingo Molnar <mingo@elte.hu>
Acked-by: "Siddha, Suresh B" <suresh.b.siddha@intel.com>
Cc: Nick Piggin <nickpiggin@yahoo.com.au>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>

diff --git a/kernel/timer.c b/kernel/timer.c
index 0256ab4..feddf81 100644 (file)
--- a/kernel/timer.c
+++ b/kernel/timer.c
@@ -1146,11 +1146,15 @@ static inline void calc_load(unsigned long ticks)
        unsigned long active_tasks; /* fixed-point */
        static int count = LOAD_FREQ;
 
-       active_tasks = count_active_tasks();
-       for (count -= ticks; count < 0; count += LOAD_FREQ) {
-               CALC_LOAD(avenrun[0], EXP_1, active_tasks);
-               CALC_LOAD(avenrun[1], EXP_5, active_tasks);
-               CALC_LOAD(avenrun[2], EXP_15, active_tasks);
+       count -= ticks;
+       if (unlikely(count < 0)) {
+               active_tasks = count_active_tasks();
+               do {
+                       CALC_LOAD(avenrun[0], EXP_1, active_tasks);
+                       CALC_LOAD(avenrun[1], EXP_5, active_tasks);
+                       CALC_LOAD(avenrun[2], EXP_15, active_tasks);
+                       count += LOAD_FREQ;
+               } while (count < 0);
        }
 }