From: Aubrey Li Date: Wed, 24 Feb 2021 08:15:49 +0000 (+0800) Subject: sched/fair: Reduce long-tail newly idle balance cost X-Git-Tag: v5.15~1260^2~33 X-Git-Url: http://review.tizen.org/git/?a=commitdiff_plain;h=acb4decc1e900468d51b33c5f1ee445278e716a7;p=platform%2Fkernel%2Flinux-starfive.git sched/fair: Reduce long-tail newly idle balance cost A long-tail load balance cost is observed on the newly idle path, this is caused by a race window between the first nr_running check of the busiest runqueue and its nr_running recheck in detach_tasks. Before the busiest runqueue is locked, the tasks on the busiest runqueue could be pulled by other CPUs and nr_running of the busiest runqueu becomes 1 or even 0 if the running task becomes idle, this causes detach_tasks breaks with LBF_ALL_PINNED flag set, and triggers load_balance redo at the same sched_domain level. In order to find the new busiest sched_group and CPU, load balance will recompute and update the various load statistics, which eventually leads to the long-tail load balance cost. This patch clears LBF_ALL_PINNED flag for this race condition, and hence reduces the long-tail cost of newly idle balance. Signed-off-by: Aubrey Li Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Vincent Guittot Link: https://lkml.kernel.org/r/1614154549-116078-1-git-send-email-aubrey.li@intel.com --- diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index aaa0dfa..6d73bdb 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -7687,6 +7687,15 @@ static int detach_tasks(struct lb_env *env) lockdep_assert_held(&env->src_rq->lock); + /* + * Source run queue has been emptied by another CPU, clear + * LBF_ALL_PINNED flag as we will not test any task. + */ + if (env->src_rq->nr_running <= 1) { + env->flags &= ~LBF_ALL_PINNED; + return 0; + } + if (env->imbalance <= 0) return 0;