ARM: smp: Store current pointer in TPIDRURO register if available
authorArd Biesheuvel <ardb@kernel.org>
Sat, 18 Sep 2021 08:44:37 +0000 (10:44 +0200)
committerArd Biesheuvel <ardb@kernel.org>
Mon, 27 Sep 2021 14:54:02 +0000 (16:54 +0200)
commit50596b7559bf226bb35ad55855ee979453ec06a1
tree844d4e625c603c1c76ebfd1e21abe9e657e531e0
parent3855ab614df4818c833864572559a97fd9f9a299
ARM: smp: Store current pointer in TPIDRURO register if available

Now that the user space TLS register is assigned on every return to user
space, we can use it to keep the 'current' pointer while running in the
kernel. This removes the need to access it via thread_info, which is
located at the base of the stack, but will be moved out of there in a
subsequent patch.

Use the __builtin_thread_pointer() helper when available - this will
help GCC understand that reloading the value within the same function is
not necessary, even when using the per-task stack protector (which also
generates accesses via the TLS register). For example, the generated
code below loads TPIDRURO only once, and uses it to access both the
stack canary and the preempt_count fields.

<do_one_initcall>:
       e92d 41f0       stmdb   sp!, {r4, r5, r6, r7, r8, lr}
       ee1d 4f70       mrc     15, 0, r4, cr13, cr0, {3}
       4606            mov     r6, r0
       b094            sub     sp, #80 ; 0x50
       f8d4 34e8       ldr.w   r3, [r4, #1256] ; 0x4e8  <- stack canary
       9313            str     r3, [sp, #76]   ; 0x4c
       f8d4 8004       ldr.w   r8, [r4, #4]             <- preempt count

Co-developed-by: Keith Packard <keithpac@amazon.com>
Signed-off-by: Keith Packard <keithpac@amazon.com>
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Tested-by: Amit Daniel Kachhap <amit.kachhap@arm.com>
12 files changed:
arch/arm/Kconfig
arch/arm/Makefile
arch/arm/include/asm/assembler.h
arch/arm/include/asm/current.h [new file with mode: 0644]
arch/arm/include/asm/switch_to.h
arch/arm/include/asm/thread_info.h
arch/arm/kernel/entry-armv.S
arch/arm/kernel/entry-common.S
arch/arm/kernel/head-common.S
arch/arm/kernel/process.c
arch/arm/kernel/smp.c
arch/arm/mm/proc-macros.S