Merge tag 'smp-core-2023-06-26' of ssh://gitolite.kernel.org/pub/scm/linux/kernel...

Pull SMP updates from Thomas Gleixner:
 "A large update for SMP management:

   - Parallel CPU bringup

     The reason why people are interested in parallel bringup is to
     shorten the (kexec) reboot time of cloud servers to reduce the
     downtime of the VM tenants.

     The current fully serialized bringup does the following per AP:

       1) Prepare callbacks (allocate, intialize, create threads)
       2) Kick the AP alive (e.g. INIT/SIPI on x86)
       3) Wait for the AP to report alive state
       4) Let the AP continue through the atomic bringup
       5) Let the AP run the threaded bringup to full online state

     There are two significant delays:

       #3 The time for an AP to report alive state in start_secondary()
          on x86 has been measured in the range between 350us and 3.5ms
          depending on vendor and CPU type, BIOS microcode size etc.

       #4 The atomic bringup does the microcode update. This has been
          measured to take up to ~8ms on the primary threads depending
          on the microcode patch size to apply.

     On a two socket SKL server with 56 cores (112 threads) the boot CPU
     spends on current mainline about 800ms busy waiting for the APs to
     come up and apply microcode. That's more than 80% of the actual
     onlining procedure.

     This can be reduced significantly by splitting the bringup
     mechanism into two parts:

       1) Run the prepare callbacks and kick the AP alive for each AP
          which needs to be brought up.

          The APs wake up, do their firmware initialization and run the
          low level kernel startup code including microcode loading in
          parallel up to the first synchronization point. (#1 and #2
          above)

       2) Run the rest of the bringup code strictly serialized per CPU
          (#3 - #5 above) as it's done today.

          Parallelizing that stage of the CPU bringup might be possible
          in theory, but it's questionable whether required surgery
          would be justified for a pretty small gain.

     If the system is large enough the first AP is already waiting at
     the first synchronization point when the boot CPU finished the
     wake-up of the last AP. That reduces the AP bringup time on that
     SKL from ~800ms to ~80ms, i.e. by a factor ~10x.

     The actual gain varies wildly depending on the system, CPU,
     microcode patch size and other factors. There are some
     opportunities to reduce the overhead further, but that needs some
     deep surgery in the x86 CPU bringup code.

     For now this is only enabled on x86, but the core functionality
     obviously works for all SMP capable architectures.

   - Enhancements for SMP function call tracing so it is possible to
     locate the scheduling and the actual execution points. That allows
     to measure IPI delivery time precisely"

* tag 'smp-core-2023-06-26' of ssh://gitolite.kernel.org/pub/scm/linux/kernel/git/tip/tip: (45 commits)
  trace,smp: Add tracepoints for scheduling remotelly called functions
  trace,smp: Add tracepoints around remotelly called functions
  MAINTAINERS: Add CPU HOTPLUG entry
  x86/smpboot: Fix the parallel bringup decision
  x86/realmode: Make stack lock work in trampoline_compat()
  x86/smp: Initialize cpu_primary_thread_mask late
  cpu/hotplug: Fix off by one in cpuhp_bringup_mask()
  x86/apic: Fix use of X{,2}APIC_ENABLE in asm with older binutils
  x86/smpboot/64: Implement arch_cpuhp_init_parallel_bringup() and enable it
  x86/smpboot: Support parallel startup of secondary CPUs
  x86/smpboot: Implement a bit spinlock to protect the realmode stack
  x86/apic: Save the APIC virtual base address
  cpu/hotplug: Allow "parallel" bringup up to CPUHP_BP_KICK_AP_STATE
  x86/apic: Provide cpu_primary_thread mask
  x86/smpboot: Enable split CPU startup
  cpu/hotplug: Provide a split up CPUHP_BRINGUP mechanism
  cpu/hotplug: Reset task stack state in _cpu_up()
  cpu/hotplug: Remove unused state functions
  riscv: Switch to hotplug core state synchronization
  parisc: Switch to hotplug core state synchronization
  ...

diff --cc Documentation/admin-guide/kernel-parameters.txt
Simple merge

diff --cc MAINTAINERS
Simple merge

diff --cc arch/Kconfig
Simple merge

diff --cc arch/arm/Kconfig
Simple merge

diff --cc arch/arm64/Kconfig
Simple merge

diff --cc arch/mips/Kconfig
Simple merge

diff --cc arch/parisc/Kconfig
Simple merge

diff --cc arch/parisc/kernel/process.c
Simple merge

diff --cc arch/riscv/Kconfig
Simple merge

diff --cc arch/x86/Kconfig
Simple merge

diff --cc arch/x86/kernel/cpu/common.c
Simple merge

diff --cc arch/x86/kernel/head_64.S
Simple merge

diff --cc arch/x86/kernel/smpboot.c
index 352f0ce,aaa876c..8de8060
--- 1/arch/x86/kernel/smpboot.c
--- 2/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@@ -226,24 -226,62 +226,63 @@@ static void notrace start_secondary(voi
         */
        cr4_init();
  
- #ifdef CONFIG_X86_32
-       /* switch away from the initial page table */
-       load_cr3(swapper_pg_dir);
-       __flush_tlb_all();
- #endif
-       cpu_init_secondary();
+       /*
+        * 32-bit specific. 64-bit reaches this code with the correct page
+        * table established. Yet another historical divergence.
+        */
+       if (IS_ENABLED(CONFIG_X86_32)) {
+               /* switch away from the initial page table */
+               load_cr3(swapper_pg_dir);
+               __flush_tlb_all();
+       }
+ 
+       cpu_init_exception_handling();
+ 
+       /*
+        * 32-bit systems load the microcode from the ASM startup code for
+        * historical reasons.
+        *
+        * On 64-bit systems load it before reaching the AP alive
+        * synchronization point below so it is not part of the full per
+        * CPU serialized bringup part when "parallel" bringup is enabled.
+        *
+        * That's even safe when hyperthreading is enabled in the CPU as
+        * the core code starts the primary threads first and leaves the
+        * secondary threads waiting for SIPI. Loading microcode on
+        * physical cores concurrently is a safe operation.
+        *
+        * This covers both the Intel specific issue that concurrent
+        * microcode loading on SMT siblings must be prohibited and the
+        * vendor independent issue`that microcode loading which changes
+        * CPUID, MSRs etc. must be strictly serialized to maintain
+        * software state correctness.
+        */
+       if (IS_ENABLED(CONFIG_X86_64))
+               load_ucode_ap();
+ 
+       /*
+        * Synchronization point with the hotplug core. Sets this CPUs
+        * synchronization state to ALIVE and spin-waits for the control CPU to
+        * release this CPU for further bringup.
+        */
+       cpuhp_ap_sync_alive();
+ 
+       cpu_init();
++      fpu__init_cpu();
        rcu_cpu_starting(raw_smp_processor_id());
        x86_cpuinit.early_percpu_clock_init();
-       smp_callin();
  
-       enable_start_cpu0 = 0;
+       ap_starting();
+ 
+       /* Check TSC synchronization with the control CPU. */
+       check_tsc_sync_target();
  
-       /* otherwise gcc will move up smp_processor_id before the cpu_init */
-       barrier();
        /*
-        * Check TSC synchronization with the boot CPU:
+        * Calibrate the delay loop after the TSC synchronization check.
+        * This allows to skip the calibration when TSC is synchronized
+        * across sockets.
         */
-       check_tsc_sync_target();
+       ap_calibrate_delay();
  
        speculative_store_bypass_ht_init();
  

diff --cc include/linux/cpu.h
index 4893c4a,d321dbd..6e6e57e
--- 1/include/linux/cpu.h
--- 2/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@@ -184,14 -184,6 +184,12 @@@ void arch_cpu_idle_enter(void)
  void arch_cpu_idle_exit(void);
  void __noreturn arch_cpu_idle_dead(void);
  
- int cpu_report_state(int cpu);
- int cpu_check_up_prepare(int cpu);
 +#ifdef CONFIG_ARCH_HAS_CPU_FINALIZE_INIT
 +void arch_cpu_finalize_init(void);
 +#else
 +static inline void arch_cpu_finalize_init(void) { }
 +#endif
 +
  void cpu_set_state_online(int cpu);
  void play_idle_precise(u64 duration_ns, u64 latency_ns);
  

diff --cc include/linux/cpuhotplug.h
Simple merge