mm, sched: Drop voluntary schedule from might_fault()
authorMichael S. Tsirkin <mst@redhat.com>
Sun, 26 May 2013 14:32:13 +0000 (17:32 +0300)
committerIngo Molnar <mingo@kernel.org>
Tue, 28 May 2013 07:41:11 +0000 (09:41 +0200)
commit114276ac0a3beb9c391a410349bd770653e185ce
treed5bfaac722054c5a576647edf8642058fae298a7
parent016be2e55d98aee0b97b94b200d6e0e110c8392a
mm, sched: Drop voluntary schedule from might_fault()

might_fault() is called from functions like copy_to_user()
which most callers expect to be very fast, like a couple of
instructions.

So functions like memcpy_toiovec() call them many times in a loop.

But might_fault() calls might_sleep() and with CONFIG_PREEMPT_VOLUNTARY
this results in a function call.

Let's not do this - just call __might_sleep() that produces
a diagnostic for sleep within atomic, but drop
might_preempt().

Here's a test sending traffic between the VM and the host,
host is built with CONFIG_PREEMPT_VOLUNTARY:

 before:
incoming: 7122.77   Mb/s
outgoing: 8480.37   Mb/s

 after:
incoming: 8619.24   Mb/s
outgoing: 9455.42   Mb/s

As a side effect, this fixes an issue pointed
out by Ingo: might_fault might schedule differently
depending on PROVE_LOCKING. Now there's no
preemption point in both cases, so it's consistent.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1369577426-26721-10-git-send-email-mst@redhat.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
include/linux/kernel.h
mm/memory.c