powerpc/64s/interrupt: Perf NMI should not take normal exit path

NMI interrupts should exit with EXCEPTION_RESTORE_REGS not with
interrupt_return_srr, which is what the perf NMI handler currently does.
This breaks if a PMI hits after interrupt_exit_user_prepare_main() has
switched the context tracking to user mode, then the CT_WARN_ON() in
interrupt_exit_kernel_prepare() fires because it returns to kernel with
context set to user.

This could possibly be solved by soft-disabling PMIs in the exit path,
but that reduces our ability to profile that code. The warning could be
removed, but it's potentially useful.

All other NMIs and soft-NMIs return using EXCEPTION_RESTORE_REGS, so
this makes perf interrupts consistent with that and seems like the best
fix.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
[mpe: Squash in fixups from Nick]
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
Link: https://lore.kernel.org/r/20221006140413.126443-3-npiggin@gmail.com

diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
index 5381a43..651c36b 100644 (file)
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -2357,9 +2357,21 @@ EXC_VIRT_END(performance_monitor, 0x4f00, 0x20)
 EXC_COMMON_BEGIN(performance_monitor_common)
        GEN_COMMON performance_monitor
        addi    r3,r1,STACK_FRAME_OVERHEAD
-       bl      performance_monitor_exception
+       lbz     r4,PACAIRQSOFTMASK(r13)
+       cmpdi   r4,IRQS_ENABLED
+       bne     1f
+       bl      performance_monitor_exception_async
        b       interrupt_return_srr
+1:
+       bl      performance_monitor_exception_nmi
+       /* Clear MSR_RI before setting SRR0 and SRR1. */
+       li      r9,0
+       mtmsrd  r9,1
 
+       kuap_kernel_restore r9, r10
+
+       EXCEPTION_RESTORE_REGS hsrr=0
+       RFI_TO_KERNEL
 
 /**
  * Interrupt 0xf20 - Vector Unavailable Interrupt.

diff --git a/arch/powerpc/kernel/interrupt.c b/arch/powerpc/kernel/interrupt.c
index 7bc9336..fc6631a 100644 (file)
--- a/arch/powerpc/kernel/interrupt.c
+++ b/arch/powerpc/kernel/interrupt.c
@@ -377,13 +377,15 @@ notrace unsigned long interrupt_exit_kernel_prepare(struct pt_regs *regs)
         * CT_WARN_ON comes here via program_check_exception, so avoid
         * recursion.
         *
-        * Skip the assertion on PMIs to work around a problem caused by NMI
-        * PMIs incorrectly taking this interrupt return path, it's possible
-        * for this to hit after interrupt exit to user switches context to
-        * user. See also the comment in the performance monitor handler in
-        * exceptions-64e/s.S
+        * Skip the assertion on PMIs on 64e to work around a problem caused
+        * by NMI PMIs incorrectly taking this interrupt return path, it's
+        * possible for this to hit after interrupt exit to user switches
+        * context to user. See also the comment in the performance monitor
+        * handler in exceptions-64e.S
         */
-       if (TRAP(regs) != INTERRUPT_PROGRAM && TRAP(regs) != INTERRUPT_PERFMON)
+       if (!IS_ENABLED(CONFIG_PPC_BOOK3E_64) &&
+           TRAP(regs) != INTERRUPT_PROGRAM &&
+           TRAP(regs) != INTERRUPT_PERFMON)
                CT_WARN_ON(ct_state() == CONTEXT_USER);
 
        kuap = kuap_get_and_assert_locked();