locking/static_keys: Add a new static_key interface
authorPeter Zijlstra <peterz@infradead.org>
Fri, 24 Jul 2015 13:09:55 +0000 (15:09 +0200)
committerIngo Molnar <mingo@kernel.org>
Mon, 3 Aug 2015 09:34:15 +0000 (11:34 +0200)
commit11276d5306b8e5b438a36bbff855fe792d7eaa61
tree9ddfc5e1bec6174e838cce8eb67af9e911c4e5f8
parent706249c222f68471b6f8e9e8e9b77665c404b226
locking/static_keys: Add a new static_key interface

There are various problems and short-comings with the current
static_key interface:

 - static_key_{true,false}() read like a branch depending on the key
   value, instead of the actual likely/unlikely branch depending on
   init value.

 - static_key_{true,false}() are, as stated above, tied to the
   static_key init values STATIC_KEY_INIT_{TRUE,FALSE}.

 - we're limited to the 2 (out of 4) possible options that compile to
   a default NOP because that's what our arch_static_branch() assembly
   emits.

So provide a new static_key interface:

  DEFINE_STATIC_KEY_TRUE(name);
  DEFINE_STATIC_KEY_FALSE(name);

Which define a key of different types with an initial true/false
value.

Then allow:

   static_branch_likely()
   static_branch_unlikely()

to take a key of either type and emit the right instruction for the
case.

This means adding a second arch_static_branch_jump() assembly helper
which emits a JMP per default.

In order to determine the right instruction for the right state,
encode the branch type in the LSB of jump_entry::key.

This is the final step in removing the naming confusion that has led to
a stream of avoidable bugs such as:

  a833581e372a ("x86, perf: Fix static_key bug in load_mm_cr4()")

... but it also allows new static key combinations that will give us
performance enhancements in the subsequent patches.

Tested-by: Rabin Vincent <rabin@rab.in> # arm
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Michael Ellerman <mpe@ellerman.id.au> # ppc
Acked-by: Heiko Carstens <heiko.carstens@de.ibm.com> # s390
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
arch/arm/include/asm/jump_label.h
arch/arm64/include/asm/jump_label.h
arch/mips/include/asm/jump_label.h
arch/powerpc/include/asm/jump_label.h
arch/s390/include/asm/jump_label.h
arch/sparc/include/asm/jump_label.h
arch/x86/include/asm/jump_label.h
include/linux/jump_label.h
kernel/jump_label.c