[PORT FROM R4]sched: Fix cgroup movement of forking process

BZ: 42195

(backport of upstream commit 4fc420c91, merged in v3.3-rc1)

There is a small race between task_fork_fair() and sched_move_task(),
which is trying to move the parent.

        task_fork_fair()                 sched_move_task()
 --------------------------------+---------------------------------
  cfs_rq = task_cfs_rq(current)
    -> cfs_rq is the "old" one.
  curr = cfs_rq->curr
    -> curr is set to the parent.
                                    task_rq_lock()
                                    dequeue_task()
                                      ->parent.se.vruntime -= (old)cfs_rq->min_vruntime
                                    enqueue_task()
                                      ->parent.se.vruntime += (new)cfs_rq->min_vruntime
                                    task_rq_unlock()
  raw_spin_lock_irqsave(rq->lock)
  se->vruntime = curr->vruntime
    -> vruntime of the child is set to that of the parent
       which has already been updated by sched_move_task().
  se->vruntime -= (old)cfs_rq->min_vruntime.
  raw_spin_unlock_irqrestore(rq->lock)

As a result, vruntime of the child becomes far bigger than expected,
if (new)cfs_rq->min_vruntime >> (old)cfs_rq->min_vruntime.

This patch fixes this problem by setting "cfs_rq" and "curr" after
holding the rq->lock.

Signed-off-by: Daisuke Nishimura <nishimura@mxp.nes.nec.co.jp>
Acked-by: Paul Turner <pjt@google.com>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Tejun Heo <tj@kernel.org>
Link: http://lkml.kernel.org/r/20111215143655.662676b0.nishimura@mxp.nes.nec.co.jp
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Change-Id: I5fdea6d67116199553e0b608603e237e4e995946
Orig-Change-Id: I1ee474f2b786b962bbdf6409c7690abcea88bd2c
Signed-off-by: German Monroy <german.monroy@intel.com>
Reviewed-on: http://android.intel.com:8080/54404
Reviewed-by: Yang, Fei <fei.yang@intel.com>
Tested-by: Ng, Cheon-woei <cheon-woei.ng@intel.com>
Reviewed-by: buildbot <buildbot@intel.com>
Tested-by: buildbot <buildbot@intel.com>

diff --git a/kernel/sched_fair.c b/kernel/sched_fair.c
index c768588..983c5ae 100644 (file)
--- a/kernel/sched_fair.c
+++ b/kernel/sched_fair.c
@@ -4129,8 +4129,8 @@ static void task_tick_fair(struct rq *rq, struct task_struct *curr, int queued)
  */
 static void task_fork_fair(struct task_struct *p)
 {
-       struct cfs_rq *cfs_rq = task_cfs_rq(current);
-       struct sched_entity *se = &p->se, *curr = cfs_rq->curr;
+       struct cfs_rq *cfs_rq;
+       struct sched_entity *se = &p->se, *curr;
        int this_cpu = smp_processor_id();
        struct rq *rq = this_rq();
        unsigned long flags;
@@ -4139,6 +4139,9 @@ static void task_fork_fair(struct task_struct *p)
 
        update_rq_clock(rq);
 
+       cfs_rq = task_cfs_rq(current);
+       curr = cfs_rq->curr;
+
        if (unlikely(task_cpu(p) != this_cpu)) {
                rcu_read_lock();
                __set_task_cpu(p, this_cpu);