Description: read only
This sysfs file exposes the cpumask which is designated to make
HCALLs to retrieve hv-gpci pmu event counter data.
+
+What: /sys/devices/hv_gpci/interface/processor_bus_topology
+Date: July 2023
+Contact: Linux on PowerPC Developer List <linuxppc-dev@lists.ozlabs.org>
+Description: admin read only
+ This sysfs file exposes the system topology information by making HCALL
+ H_GET_PERF_COUNTER_INFO. The HCALL is made with counter request value
+ PROCESSOR_BUS_TOPOLOGY(0xD0).
+
+ * This sysfs file will be created only for power10 and above platforms.
+
+ * User needs root privileges to read data from this sysfs file.
+
+ * This sysfs file will be created, only when the HCALL returns "H_SUCCESS",
+ "H_AUTHORITY" or "H_PARAMETER" as the return type.
+
+ HCALL with return error type "H_AUTHORITY" can be resolved during
+ runtime by setting "Enable Performance Information Collection" option.
+
+ * The end user reading this sysfs file must decode the content as per
+ underlying platform/firmware.
+
+ Possible error codes while reading this sysfs file:
+
+ * "-EPERM" : Partition is not permitted to retrieve performance information,
+ required to set "Enable Performance Information Collection" option.
+
+ * "-EIO" : Can't retrieve system information because of invalid buffer length/invalid address
+ or because of some hardware error. Refer to getPerfCountInfo documentation for
+ more information.
+
+ * "-EFBIG" : System information exceeds PAGE_SIZE.
+
+What: /sys/devices/hv_gpci/interface/processor_config
+Date: July 2023
+Contact: Linux on PowerPC Developer List <linuxppc-dev@lists.ozlabs.org>
+Description: admin read only
+ This sysfs file exposes the system topology information by making HCALL
+ H_GET_PERF_COUNTER_INFO. The HCALL is made with counter request value
+ PROCESSOR_CONFIG(0x90).
+
+ * This sysfs file will be created only for power10 and above platforms.
+
+ * User needs root privileges to read data from this sysfs file.
+
+ * This sysfs file will be created, only when the HCALL returns "H_SUCCESS",
+ "H_AUTHORITY" or "H_PARAMETER" as the return type.
+
+ HCALL with return error type "H_AUTHORITY" can be resolved during
+ runtime by setting "Enable Performance Information Collection" option.
+
+ * The end user reading this sysfs file must decode the content as per
+ underlying platform/firmware.
+
+ Possible error codes while reading this sysfs file:
+
+ * "-EPERM" : Partition is not permitted to retrieve performance information,
+ required to set "Enable Performance Information Collection" option.
+
+ * "-EIO" : Can't retrieve system information because of invalid buffer length/invalid address
+ or because of some hardware error. Refer to getPerfCountInfo documentation for
+ more information.
+
+ * "-EFBIG" : System information exceeds PAGE_SIZE.
+
+What: /sys/devices/hv_gpci/interface/affinity_domain_via_virtual_processor
+Date: July 2023
+Contact: Linux on PowerPC Developer List <linuxppc-dev@lists.ozlabs.org>
+Description: admin read only
+ This sysfs file exposes the system topology information by making HCALL
+ H_GET_PERF_COUNTER_INFO. The HCALL is made with counter request value
+ AFFINITY_DOMAIN_INFORMATION_BY_VIRTUAL_PROCESSOR(0xA0).
+
+ * This sysfs file will be created only for power10 and above platforms.
+
+ * User needs root privileges to read data from this sysfs file.
+
+ * This sysfs file will be created, only when the HCALL returns "H_SUCCESS",
+ "H_AUTHORITY" or "H_PARAMETER" as the return type.
+
+ HCALL with return error type "H_AUTHORITY" can be resolved during
+ runtime by setting "Enable Performance Information Collection" option.
+
+ * The end user reading this sysfs file must decode the content as per
+ underlying platform/firmware.
+
+ Possible error codes while reading this sysfs file:
+
+ * "-EPERM" : Partition is not permitted to retrieve performance information,
+ required to set "Enable Performance Information Collection" option.
+
+ * "-EIO" : Can't retrieve system information because of invalid buffer length/invalid address
+ or because of some hardware error. Refer to getPerfCountInfo documentation for
+ more information.
+
+ * "-EFBIG" : System information exceeds PAGE_SIZE.
+
+What: /sys/devices/hv_gpci/interface/affinity_domain_via_domain
+Date: July 2023
+Contact: Linux on PowerPC Developer List <linuxppc-dev@lists.ozlabs.org>
+Description: admin read only
+ This sysfs file exposes the system topology information by making HCALL
+ H_GET_PERF_COUNTER_INFO. The HCALL is made with counter request value
+ AFFINITY_DOMAIN_INFORMATION_BY_DOMAIN(0xB0).
+
+ * This sysfs file will be created only for power10 and above platforms.
+
+ * User needs root privileges to read data from this sysfs file.
+
+ * This sysfs file will be created, only when the HCALL returns "H_SUCCESS",
+ "H_AUTHORITY" or "H_PARAMETER" as the return type.
+
+ HCALL with return error type "H_AUTHORITY" can be resolved during
+ runtime by setting "Enable Performance Information Collection" option.
+
+ * The end user reading this sysfs file must decode the content as per
+ underlying platform/firmware.
+
+ Possible error codes while reading this sysfs file:
+
+ * "-EPERM" : Partition is not permitted to retrieve performance information,
+ required to set "Enable Performance Information Collection" option.
+
+ * "-EIO" : Can't retrieve system information because of invalid buffer length/invalid address
+ or because of some hardware error. Refer to getPerfCountInfo documentation for
+ more information.
+
+ * "-EFBIG" : System information exceeds PAGE_SIZE.
+
+What: /sys/devices/hv_gpci/interface/affinity_domain_via_partition
+Date: July 2023
+Contact: Linux on PowerPC Developer List <linuxppc-dev@lists.ozlabs.org>
+Description: admin read only
+ This sysfs file exposes the system topology information by making HCALL
+ H_GET_PERF_COUNTER_INFO. The HCALL is made with counter request value
+ AFFINITY_DOMAIN_INFORMATION_BY_PARTITION(0xB1).
+
+ * This sysfs file will be created only for power10 and above platforms.
+
+ * User needs root privileges to read data from this sysfs file.
+
+ * This sysfs file will be created, only when the HCALL returns "H_SUCCESS",
+ "H_AUTHORITY" or "H_PARAMETER" as the return type.
+
+ HCALL with return error type "H_AUTHORITY" can be resolved during
+ runtime by setting "Enable Performance Information Collection" option.
+
+ * The end user reading this sysfs file must decode the content as per
+ underlying platform/firmware.
+
+ Possible error codes while reading this sysfs file:
+
+ * "-EPERM" : Partition is not permitted to retrieve performance information,
+ required to set "Enable Performance Information Collection" option.
+
+ * "-EIO" : Can't retrieve system information because of invalid buffer length/invalid address
+ or because of some hardware error. Refer to getPerfCountInfo documentation for
+ more information.
+
+ * "-EFBIG" : System information exceeds PAGE_SIZE.
nohibernate [HIBERNATION] Disable hibernation and resume.
- nohlt [ARM,ARM64,MICROBLAZE,MIPS,SH] Forces the kernel to
+ nohlt [ARM,ARM64,MICROBLAZE,MIPS,PPC,SH] Forces the kernel to
busy wait in do_idle() and not use the arch_cpu_idle()
implementation; requires CONFIG_GENERIC_IDLE_POLL_SETUP
to be effective. This is useful on platforms where the
nosmp [SMP] Tells an SMP kernel to act as a UP kernel,
and disable the IO APIC. legacy for "maxcpus=0".
- nosmt [KNL,MIPS,S390] Disable symmetric multithreading (SMT).
+ nosmt [KNL,MIPS,PPC,S390] Disable symmetric multithreading (SMT).
Equivalent to smt=1.
- [KNL,X86] Disable symmetric multithreading (SMT).
+ [KNL,X86,PPC] Disable symmetric multithreading (SMT).
nosmt=force: Force disable SMT, cannot be undone
via the sysfs control file.
that GDB doesn't need to special-case each of them. We added the
following 3 new ptrace requests.
-1. PTRACE_PPC_GETHWDEBUGINFO
+1. PPC_PTRACE_GETHWDBGINFO
============================
Query for GDB to discover the hardware debug features. The main info to
#define PPC_DEBUG_FEATURE_DATA_BP_DAWR 0x10
#define PPC_DEBUG_FEATURE_DATA_BP_ARCH_31 0x20
-2. PTRACE_SETHWDEBUG
+2. PPC_PTRACE_SETHWDEBUG
Sets a hardware breakpoint or watchpoint, according to the provided structure::
are not contemplated, but that is out of the scope of this work.
ptrace will return an integer (handle) uniquely identifying the breakpoint or
-watchpoint just created. This integer will be used in the PTRACE_DELHWDEBUG
+watchpoint just created. This integer will be used in the PPC_PTRACE_DELHWDEBUG
request to ask for its removal. Return -ENOSPC if the requested breakpoint
can't be allocated on the registers.
p.addr2 = (uint64_t) end_range;
p.condition_value = 0;
-3. PTRACE_DELHWDEBUG
+3. PPC_PTRACE_DELHWDEBUG
Takes an integer which identifies an existing breakpoint or watchpoint
(i.e., the value returned from PTRACE_SETHWDEBUG), and deletes the
select DYNAMIC_FTRACE if FUNCTION_TRACER
select EDAC_ATOMIC_SCRUB
select EDAC_SUPPORT
+ select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY if ARCH_USING_PATCHABLE_FUNCTION_ENTRY
select GENERIC_ATOMIC64 if PPC32
select GENERIC_CLOCKEVENTS_BROADCAST if SMP
select GENERIC_CMOS_UPDATE
select GENERIC_CPU_VULNERABILITIES if PPC_BARRIER_NOSPEC
select GENERIC_EARLY_IOREMAP
select GENERIC_GETTIMEOFDAY
+ select GENERIC_IDLE_POLL_SETUP
select GENERIC_IOREMAP
select GENERIC_IRQ_SHOW
select GENERIC_IRQ_SHOW_LEVEL
select HAVE_DEBUG_KMEMLEAK
select HAVE_DEBUG_STACKOVERFLOW
select HAVE_DYNAMIC_FTRACE
- select HAVE_DYNAMIC_FTRACE_WITH_ARGS if MPROFILE_KERNEL || PPC32
- select HAVE_DYNAMIC_FTRACE_WITH_REGS if MPROFILE_KERNEL || PPC32
+ select HAVE_DYNAMIC_FTRACE_WITH_ARGS if ARCH_USING_PATCHABLE_FUNCTION_ENTRY || MPROFILE_KERNEL || PPC32
+ select HAVE_DYNAMIC_FTRACE_WITH_REGS if ARCH_USING_PATCHABLE_FUNCTION_ENTRY || MPROFILE_KERNEL || PPC32
select HAVE_EBPF_JIT
select HAVE_EFFICIENT_UNALIGNED_ACCESS
select HAVE_FAST_GUP
select HAVE_MOD_ARCH_SPECIFIC
select HAVE_NMI if PERF_EVENTS || (PPC64 && PPC_BOOK3S)
select HAVE_OPTPROBES
- select HAVE_OBJTOOL if PPC32 || MPROFILE_KERNEL
+ select HAVE_OBJTOOL if ARCH_USING_PATCHABLE_FUNCTION_ENTRY || MPROFILE_KERNEL || PPC32
select HAVE_OBJTOOL_MCOUNT if HAVE_OBJTOOL
select HAVE_PERF_EVENTS
select HAVE_PERF_EVENTS_NMI if PPC64
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_VIRT_CPU_ACCOUNTING
select HAVE_VIRT_CPU_ACCOUNTING_GEN
+ select HOTPLUG_SMT if HOTPLUG_CPU
+ select SMT_NUM_THREADS_DYNAMIC
select HUGETLB_PAGE_SIZE_VARIABLE if PPC_BOOK3S_64 && HUGETLB_PAGE
select IOMMU_HELPER if PPC64
select IRQ_DOMAIN
def_bool $(success,$(srctree)/arch/powerpc/tools/gcc-check-mprofile-kernel.sh $(CC) -mlittle-endian) if CPU_LITTLE_ENDIAN
def_bool $(success,$(srctree)/arch/powerpc/tools/gcc-check-mprofile-kernel.sh $(CC) -mbig-endian) if CPU_BIG_ENDIAN
+config ARCH_USING_PATCHABLE_FUNCTION_ENTRY
+ depends on FUNCTION_TRACER && (PPC32 || PPC64_ELF_ABI_V2)
+ depends on $(cc-option,-fpatchable-function-entry=2)
+ def_bool y if PPC32
+ def_bool $(success,$(srctree)/arch/powerpc/tools/gcc-check-fpatchable-function-entry.sh $(CC) -mlittle-endian) if PPC64 && CPU_LITTLE_ENDIAN
+ def_bool $(success,$(srctree)/arch/powerpc/tools/gcc-check-fpatchable-function-entry.sh $(CC) -mbig-endian) if PPC64 && CPU_BIG_ENDIAN
+
config HOTPLUG_CPU
bool "Support for enabling/disabling CPUs"
depends on SMP && (PPC_PSERIES || \
help
Freescale General-purpose Timers support
-config PCI_8260
- bool
- depends on PCI && 8260
- select PPC_INDIRECT_PCI
- default y
-
config FSL_RIO
bool "Freescale Embedded SRIO Controller support"
depends on RAPIDIO = y && HAVE_RAPIDIO
CFLAGS-$(CONFIG_PPC32) += $(call cc-option,-mno-readonly-in-sdata)
ifdef CONFIG_FUNCTION_TRACER
+ifdef CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY
+KBUILD_CPPFLAGS += -DCC_USING_PATCHABLE_FUNCTION_ENTRY
+CC_FLAGS_FTRACE := -fpatchable-function-entry=2
+else
CC_FLAGS_FTRACE := -pg
ifdef CONFIG_MPROFILE_KERNEL
CC_FLAGS_FTRACE += -mprofile-kernel
endif
endif
+endif
CFLAGS-$(CONFIG_TARGET_CPU_BOOL) += -mcpu=$(CONFIG_TARGET_CPU)
AFLAGS-$(CONFIG_TARGET_CPU_BOOL) += -mcpu=$(CONFIG_TARGET_CPU)
-CFLAGS-$(CONFIG_POWERPC64_CPU) += $(call cc-option,-mtune=power10, \
- $(call cc-option,-mtune=power9, \
- $(call cc-option,-mtune=power8)))
+CFLAGS-y += $(CONFIG_TUNE_CPU)
asinstr := $(call as-instr,lis 9$(comma)foo@high,-DHAVE_AS_ATHIGH=1)
reg = <0x80000 0x20000>;
ranges = <0x0 0x80000 0x20000>;
- jr@1000{
+ jr@1000 {
interrupts = <45 2 0 0>;
};
- jr@2000{
+ jr@2000 {
interrupts = <57 2 0 0>;
};
};
reg = <0xa0000 0x20000>;
ranges = <0x0 0xa0000 0x20000>;
- jr@1000{
+ jr@1000 {
interrupts = <49 2 0 0>;
};
- jr@2000{
+ jr@2000 {
interrupts = <50 2 0 0>;
};
};
reg = <0xc0000 0x20000>;
ranges = <0x0 0xc0000 0x20000>;
- jr@1000{
+ jr@1000 {
interrupts = <55 2 0 0>;
};
- jr@2000{
+ jr@2000 {
interrupts = <56 2 0 0>;
};
};
compatible = "st,m41t62";
reg = <0x68>;
};
- adt7461@4c{
+ adt7461@4c {
compatible = "adi,adt7461";
reg = <0x4c>;
};
- zl6100@21{
+ zl6100@21 {
compatible = "isil,zl6100";
reg = <0x21>;
};
- zl6100@24{
+ zl6100@24 {
compatible = "isil,zl6100";
reg = <0x24>;
};
- zl6100@26{
+ zl6100@26 {
compatible = "isil,zl6100";
reg = <0x26>;
};
- zl6100@29{
+ zl6100@29 {
compatible = "isil,zl6100";
reg = <0x29>;
};
fsl,has-rstcr;
};
- power@e0070{
+ power@e0070 {
compatible = "fsl,mpc8536-pmc", "fsl,mpc8548-pmc";
reg = <0xe0070 0x20>;
};
#size-cells = <2>;
interrupt-parent = <&mpic>;
- aliases{
+ aliases {
phy_rgmii_0 = &phy_rgmii_0;
phy_rgmii_1 = &phy_rgmii_1;
phy_sgmii_1c = &phy_sgmii_1c;
};
};
- fman@400000{
+ fman@400000 {
ethernet@e0000 {
phy-handle = <&phy_sgmii_1c>;
phy-connection-type = "sgmii";
#size-cells = <2>;
interrupt-parent = <&mpic>;
- aliases{
+ aliases {
phy_sgmii_slot2_1c = &phy_sgmii_slot2_1c;
phy_sgmii_slot2_1d = &phy_sgmii_slot2_1d;
phy_sgmii_slot2_1e = &phy_sgmii_slot2_1e;
#size-cells = <2>;
interrupt-parent = <&mpic>;
- aliases{
+ aliases {
phy_rgmii1 = &phyrgmii1;
phy_rgmii2 = &phyrgmii2;
phy_sgmii3 = &phy3;
};
/* Power Management Controller */
- pmc@1000{
+ pmc@1000 {
compatible = "fsl,mpc5121-pmc";
reg = <0x1000 0x100>;
interrupts = <83 0x8>;
clock-names = "osc";
};
- pmc@1000{ // Power Management Controller
+ pmc@1000 { // Power Management Controller
compatible = "fsl,mpc5121-pmc";
reg = <0x1000 0x100>;
interrupts = <83 0x2>;
# CONFIG_SERIO_I8042 is not set
# CONFIG_SERIO_SERPORT is not set
CONFIG_SERIAL_8250=m
-CONFIG_SERIAL_PMACZILOG=m
+CONFIG_SERIAL_PMACZILOG=y
CONFIG_SERIAL_PMACZILOG_TTYS=y
+CONFIG_SERIAL_PMACZILOG_CONSOLE=y
CONFIG_NVRAM=y
CONFIG_I2C_CHARDEV=m
CONFIG_APM_POWER=y
CONFIG_CRYPTO_WP512=m
CONFIG_CRYPTO_LZO=m
CONFIG_CRYPTO_CRC32C_VPMSUM=m
+CONFIG_CRYPTO_CRCT10DIF_VPMSUM=m
+CONFIG_CRYPTO_VPMSUM_TESTER=m
CONFIG_CRYPTO_MD5_PPC=m
CONFIG_CRYPTO_SHA1_PPC=m
+CONFIG_CRYPTO_AES_GCM_P10=m
CONFIG_CRYPTO_DEV_NX=y
CONFIG_CRYPTO_DEV_NX_ENCRYPT=m
CONFIG_CRYPTO_DEV_VMX=y
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
CONFIG_IP_NF_MANGLE=m
-CONFIG_IP_NF_TARGET_CLUSTERIP=m
CONFIG_IP_NF_TARGET_ECN=m
CONFIG_IP_NF_TARGET_TTL=m
CONFIG_IP_NF_RAW=m
# CONFIG_XZ_DEC_SPARC is not set
CONFIG_PRINTK_TIME=y
CONFIG_MAGIC_SYSRQ=y
-CONFIG_SLUB_DEBUG_ON=y
CONFIG_SCHED_STACK_END_CHECK=y
CONFIG_DEBUG_STACKOVERFLOW=y
CONFIG_PANIC_ON_OOPS=y
select CRYPTO_LIB_AES
select CRYPTO_ALGAPI
select CRYPTO_AEAD
- default m
+ select CRYPTO_SKCIPHER
help
AEAD cipher: AES cipher algorithms (FIPS-197)
GCM (Galois/Counter Mode) authenticated encryption mode (NIST SP800-38D)
cpm8xx_t im_cpm; /* Communication processor */
} immap_t;
+extern immap_t __iomem *mpc8xx_immr;
+
#endif /* __IMMAP_8XX__ */
#endif /* __KERNEL__ */
generated-y += syscall_table_64.h
generated-y += syscall_table_spu.h
generic-y += agp.h
-generic-y += export.h
generic-y += kvm_types.h
generic-y += mcs_spinlock.h
generic-y += qrwlock.h
#ifndef __ASSEMBLY__
-#include <linux/jump_label.h>
-
-extern struct static_key_false disable_kuap_key;
-
-static __always_inline bool kuep_is_disabled(void)
-{
- return !IS_ENABLED(CONFIG_PPC_KUEP);
-}
-
#ifdef CONFIG_PPC_KUAP
#include <linux/sched.h>
#define KUAP_NONE (~0UL)
-#define KUAP_ALL (~1UL)
-static __always_inline bool kuap_is_disabled(void)
-{
- return static_branch_unlikely(&disable_kuap_key);
-}
-
-static inline void kuap_lock_one(unsigned long addr)
+static __always_inline void kuap_lock_one(unsigned long addr)
{
mtsr(mfsr(addr) | SR_KS, addr);
isync(); /* Context sync required after mtsr() */
}
-static inline void kuap_unlock_one(unsigned long addr)
+static __always_inline void kuap_unlock_one(unsigned long addr)
{
mtsr(mfsr(addr) & ~SR_KS, addr);
isync(); /* Context sync required after mtsr() */
}
-static inline void kuap_lock_all(void)
+static __always_inline void uaccess_begin_32s(unsigned long addr)
{
- update_user_segments(mfsr(0) | SR_KS);
- isync(); /* Context sync required after mtsr() */
-}
+ unsigned long tmp;
-static inline void kuap_unlock_all(void)
-{
- update_user_segments(mfsr(0) & ~SR_KS);
- isync(); /* Context sync required after mtsr() */
+ asm volatile(ASM_MMU_FTR_IFSET(
+ "mfsrin %0, %1;"
+ "rlwinm %0, %0, 0, %2;"
+ "mtsrin %0, %1;"
+ "isync", "", %3)
+ : "=&r"(tmp)
+ : "r"(addr), "i"(~SR_KS), "i"(MMU_FTR_KUAP)
+ : "memory");
}
-void kuap_lock_all_ool(void);
-void kuap_unlock_all_ool(void);
-
-static inline void kuap_lock_addr(unsigned long addr, bool ool)
+static __always_inline void uaccess_end_32s(unsigned long addr)
{
- if (likely(addr != KUAP_ALL))
- kuap_lock_one(addr);
- else if (!ool)
- kuap_lock_all();
- else
- kuap_lock_all_ool();
-}
+ unsigned long tmp;
-static inline void kuap_unlock(unsigned long addr, bool ool)
-{
- if (likely(addr != KUAP_ALL))
- kuap_unlock_one(addr);
- else if (!ool)
- kuap_unlock_all();
- else
- kuap_unlock_all_ool();
+ asm volatile(ASM_MMU_FTR_IFSET(
+ "mfsrin %0, %1;"
+ "oris %0, %0, %2;"
+ "mtsrin %0, %1;"
+ "isync", "", %3)
+ : "=&r"(tmp)
+ : "r"(addr), "i"(SR_KS >> 16), "i"(MMU_FTR_KUAP)
+ : "memory");
}
-static inline void __kuap_lock(void)
-{
-}
-
-static inline void __kuap_save_and_lock(struct pt_regs *regs)
+static __always_inline void __kuap_save_and_lock(struct pt_regs *regs)
{
unsigned long kuap = current->thread.kuap;
return;
current->thread.kuap = KUAP_NONE;
- kuap_lock_addr(kuap, false);
+ kuap_lock_one(kuap);
}
+#define __kuap_save_and_lock __kuap_save_and_lock
-static inline void kuap_user_restore(struct pt_regs *regs)
+static __always_inline void kuap_user_restore(struct pt_regs *regs)
{
}
-static inline void __kuap_kernel_restore(struct pt_regs *regs, unsigned long kuap)
+static __always_inline void __kuap_kernel_restore(struct pt_regs *regs, unsigned long kuap)
{
if (unlikely(kuap != KUAP_NONE)) {
current->thread.kuap = KUAP_NONE;
- kuap_lock_addr(kuap, false);
+ kuap_lock_one(kuap);
}
if (likely(regs->kuap == KUAP_NONE))
current->thread.kuap = regs->kuap;
- kuap_unlock(regs->kuap, false);
+ kuap_unlock_one(regs->kuap);
}
-static inline unsigned long __kuap_get_and_assert_locked(void)
+static __always_inline unsigned long __kuap_get_and_assert_locked(void)
{
unsigned long kuap = current->thread.kuap;
return kuap;
}
+#define __kuap_get_and_assert_locked __kuap_get_and_assert_locked
-static __always_inline void __allow_user_access(void __user *to, const void __user *from,
- u32 size, unsigned long dir)
+static __always_inline void allow_user_access(void __user *to, const void __user *from,
+ u32 size, unsigned long dir)
{
BUILD_BUG_ON(!__builtin_constant_p(dir));
return;
current->thread.kuap = (__force u32)to;
- kuap_unlock_one((__force u32)to);
+ uaccess_begin_32s((__force u32)to);
}
-static __always_inline void __prevent_user_access(unsigned long dir)
+static __always_inline void prevent_user_access(unsigned long dir)
{
u32 kuap = current->thread.kuap;
return;
current->thread.kuap = KUAP_NONE;
- kuap_lock_addr(kuap, true);
+ uaccess_end_32s(kuap);
}
-static inline unsigned long __prevent_user_access_return(void)
+static __always_inline unsigned long prevent_user_access_return(void)
{
unsigned long flags = current->thread.kuap;
if (flags != KUAP_NONE) {
current->thread.kuap = KUAP_NONE;
- kuap_lock_addr(flags, true);
+ uaccess_end_32s(flags);
}
return flags;
}
-static inline void __restore_user_access(unsigned long flags)
+static __always_inline void restore_user_access(unsigned long flags)
{
if (flags != KUAP_NONE) {
current->thread.kuap = flags;
- kuap_unlock(flags, true);
+ uaccess_begin_32s(flags);
}
}
-static inline bool
+static __always_inline bool
__bad_kuap_fault(struct pt_regs *regs, unsigned long address, bool is_write)
{
unsigned long kuap = regs->kuap;
- if (!is_write || kuap == KUAP_ALL)
+ if (!is_write)
return false;
if (kuap == KUAP_NONE)
return true;
- /* If faulting address doesn't match unlocked segment, unlock all */
- if ((kuap ^ address) & 0xf0000000)
- regs->kuap = KUAP_ALL;
+ /*
+ * If faulting address doesn't match unlocked segment, change segment.
+ * In case of unaligned store crossing two segments, emulate store.
+ */
+ if ((kuap ^ address) & 0xf0000000) {
+ if (!(kuap & 0x0fffffff) && address > kuap - 4 && fix_alignment(regs)) {
+ regs_add_return_ip(regs, 4);
+ emulate_single_step(regs);
+ } else {
+ regs->kuap = address;
+ }
+ }
return false;
}
/* This low level function performs the actual PTE insertion
- * Setting the PTE depends on the MMU type and other factors. It's
- * an horrible mess that I'm not going to try to clean up now but
- * I'm keeping it in one place rather than spread around
+ * Setting the PTE depends on the MMU type and other factors.
+ *
+ * First case is 32-bit in UP mode with 32-bit PTEs, we need to preserve
+ * the _PAGE_HASHPTE bit since we may not have invalidated the previous
+ * translation in the hash yet (done in a subsequent flush_tlb_xxx())
+ * and see we need to keep track that this PTE needs invalidating.
+ *
+ * Second case is 32-bit with 64-bit PTE. In this case, we
+ * can just store as long as we do the two halves in the right order
+ * with a barrier in between. This is possible because we take care,
+ * in the hash code, to pre-invalidate if the PTE was already hashed,
+ * which synchronizes us with any concurrent invalidation.
+ * In the percpu case, we fallback to the simple update preserving
+ * the hash bits (ie, same as the non-SMP case).
+ *
+ * Third case is 32-bit in SMP mode with 32-bit PTEs. We use the
+ * helper pte_update() which does an atomic update. We need to do that
+ * because a concurrent invalidation can clear _PAGE_HASHPTE. If it's a
+ * per-CPU PTE such as a kmap_atomic, we also do a simple update preserving
+ * the hash bits instead.
*/
static inline void __set_pte_at(struct mm_struct *mm, unsigned long addr,
pte_t *ptep, pte_t pte, int percpu)
{
-#if defined(CONFIG_SMP) && !defined(CONFIG_PTE_64BIT)
- /* First case is 32-bit Hash MMU in SMP mode with 32-bit PTEs. We use the
- * helper pte_update() which does an atomic update. We need to do that
- * because a concurrent invalidation can clear _PAGE_HASHPTE. If it's a
- * per-CPU PTE such as a kmap_atomic, we do a simple update preserving
- * the hash bits instead (ie, same as the non-SMP case)
- */
- if (percpu)
- *ptep = __pte((pte_val(*ptep) & _PAGE_HASHPTE)
- | (pte_val(pte) & ~_PAGE_HASHPTE));
- else
- pte_update(mm, addr, ptep, ~_PAGE_HASHPTE, pte_val(pte), 0);
+ if ((!IS_ENABLED(CONFIG_SMP) && !IS_ENABLED(CONFIG_PTE_64BIT)) || percpu) {
+ *ptep = __pte((pte_val(*ptep) & _PAGE_HASHPTE) |
+ (pte_val(pte) & ~_PAGE_HASHPTE));
+ } else if (IS_ENABLED(CONFIG_PTE_64BIT)) {
+ if (pte_val(*ptep) & _PAGE_HASHPTE)
+ flush_hash_entry(mm, ptep, addr);
-#elif defined(CONFIG_PTE_64BIT)
- /* Second case is 32-bit with 64-bit PTE. In this case, we
- * can just store as long as we do the two halves in the right order
- * with a barrier in between. This is possible because we take care,
- * in the hash code, to pre-invalidate if the PTE was already hashed,
- * which synchronizes us with any concurrent invalidation.
- * In the percpu case, we also fallback to the simple update preserving
- * the hash bits
- */
- if (percpu) {
- *ptep = __pte((pte_val(*ptep) & _PAGE_HASHPTE)
- | (pte_val(pte) & ~_PAGE_HASHPTE));
- return;
+ asm volatile("stw%X0 %2,%0; eieio; stw%X1 %L2,%1" :
+ "=m" (*ptep), "=m" (*((unsigned char *)ptep+4)) :
+ "r" (pte) : "memory");
+ } else {
+ pte_update(mm, addr, ptep, ~_PAGE_HASHPTE, pte_val(pte), 0);
}
- if (pte_val(*ptep) & _PAGE_HASHPTE)
- flush_hash_entry(mm, ptep, addr);
- __asm__ __volatile__("\
- stw%X0 %2,%0\n\
- eieio\n\
- stw%X1 %L2,%1"
- : "=m" (*ptep), "=m" (*((unsigned char *)ptep+4))
- : "r" (pte) : "memory");
-
-#else
- /* Third case is 32-bit hash table in UP mode, we need to preserve
- * the _PAGE_HASHPTE bit since we may not have invalidated the previous
- * translation in the hash yet (done in a subsequent flush_tlb_xxx())
- * and see we need to keep track that this PTE needs invalidating
- */
- *ptep = __pte((pte_val(*ptep) & _PAGE_HASHPTE)
- | (pte_val(pte) & ~_PAGE_HASHPTE));
-#endif
}
/*
((pteflags & H_PTE_PKEY_BIT1) ? HPTE_R_KEY_BIT1 : 0x0UL) |
((pteflags & H_PTE_PKEY_BIT0) ? HPTE_R_KEY_BIT0 : 0x0UL));
- if (mmu_has_feature(MMU_FTR_BOOK3S_KUAP) ||
+ if (mmu_has_feature(MMU_FTR_KUAP) ||
mmu_has_feature(MMU_FTR_BOOK3S_KUEP)) {
if ((pte_pkey == 0) && (flags & HPTE_USE_KERNEL_KEY))
return HASH_DEFAULT_KERNEL_KEY;
mfspr \gpr2, SPRN_AMR
cmpd \gpr1, \gpr2
beq 99f
- END_MMU_FTR_SECTION_NESTED_IFCLR(MMU_FTR_BOOK3S_KUAP, 68)
+ END_MMU_FTR_SECTION_NESTED_IFCLR(MMU_FTR_KUAP, 68)
isync
mtspr SPRN_AMR, \gpr1
* No need to restore IAMR when returning to kernel space.
*/
100:
- END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_BOOK3S_KUAP, 67)
+ END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_KUAP, 67)
#endif
.endm
LOAD_REG_IMMEDIATE(\gpr2, AMR_KUAP_BLOCKED)
999: tdne \gpr1, \gpr2
EMIT_WARN_ENTRY 999b, __FILE__, __LINE__, (BUGFLAG_WARNING | BUGFLAG_ONCE)
- END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_BOOK3S_KUAP, 67)
+ END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_KUAP, 67)
#endif
.endm
#endif
*/
BEGIN_MMU_FTR_SECTION_NESTED(68)
b 100f // skip_save_amr
- END_MMU_FTR_SECTION_NESTED_IFCLR(MMU_FTR_PKEY | MMU_FTR_BOOK3S_KUAP, 68)
+ END_MMU_FTR_SECTION_NESTED_IFCLR(MMU_FTR_PKEY | MMU_FTR_KUAP, 68)
/*
* if pkey is disabled and we are entering from userspace
mtspr SPRN_AMR, \gpr2
isync
102:
- END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_BOOK3S_KUAP, 69)
+ END_MMU_FTR_SECTION_NESTED_IFSET(MMU_FTR_KUAP, 69)
/*
* if entering from kernel we don't need save IAMR
* access restrictions. Because of this ignore AMR value when accessing
* userspace via kernel thread.
*/
-static inline u64 current_thread_amr(void)
+static __always_inline u64 current_thread_amr(void)
{
if (current->thread.regs)
return current->thread.regs->amr;
return default_amr;
}
-static inline u64 current_thread_iamr(void)
+static __always_inline u64 current_thread_iamr(void)
{
if (current->thread.regs)
return current->thread.regs->iamr;
#ifdef CONFIG_PPC_KUAP
-static __always_inline bool kuap_is_disabled(void)
-{
- return !mmu_has_feature(MMU_FTR_BOOK3S_KUAP);
-}
-
-static inline void kuap_user_restore(struct pt_regs *regs)
+static __always_inline void kuap_user_restore(struct pt_regs *regs)
{
bool restore_amr = false, restore_iamr = false;
unsigned long amr, iamr;
if (!mmu_has_feature(MMU_FTR_PKEY))
return;
- if (!mmu_has_feature(MMU_FTR_BOOK3S_KUAP)) {
+ if (!mmu_has_feature(MMU_FTR_KUAP)) {
amr = mfspr(SPRN_AMR);
if (amr != regs->amr)
restore_amr = true;
*/
}
-static inline void __kuap_kernel_restore(struct pt_regs *regs, unsigned long amr)
+static __always_inline void __kuap_kernel_restore(struct pt_regs *regs, unsigned long amr)
{
if (likely(regs->amr == amr))
return;
*/
}
-static inline unsigned long __kuap_get_and_assert_locked(void)
+static __always_inline unsigned long __kuap_get_and_assert_locked(void)
{
unsigned long amr = mfspr(SPRN_AMR);
WARN_ON_ONCE(amr != AMR_KUAP_BLOCKED);
return amr;
}
+#define __kuap_get_and_assert_locked __kuap_get_and_assert_locked
-/* Do nothing, book3s/64 does that in ASM */
-static inline void __kuap_lock(void)
-{
-}
-
-static inline void __kuap_save_and_lock(struct pt_regs *regs)
-{
-}
+/* __kuap_lock() not required, book3s/64 does that in ASM */
/*
* We support individually allowing read or write, but we don't support nesting
* because that would require an expensive read/modify write of the AMR.
*/
-static inline unsigned long get_kuap(void)
+static __always_inline unsigned long get_kuap(void)
{
/*
* We return AMR_KUAP_BLOCKED when we don't support KUAP because
* This has no effect in terms of actually blocking things on hash,
* so it doesn't break anything.
*/
- if (!mmu_has_feature(MMU_FTR_BOOK3S_KUAP))
+ if (!mmu_has_feature(MMU_FTR_KUAP))
return AMR_KUAP_BLOCKED;
return mfspr(SPRN_AMR);
static __always_inline void set_kuap(unsigned long value)
{
- if (!mmu_has_feature(MMU_FTR_BOOK3S_KUAP))
+ if (!mmu_has_feature(MMU_FTR_KUAP))
return;
/*
isync();
}
-static inline bool __bad_kuap_fault(struct pt_regs *regs, unsigned long address, bool is_write)
+static __always_inline bool
+__bad_kuap_fault(struct pt_regs *regs, unsigned long address, bool is_write)
{
/*
* For radix this will be a storage protection fault (DSISR_PROTFAULT).
#else /* CONFIG_PPC_KUAP */
-static inline unsigned long get_kuap(void)
+static __always_inline unsigned long get_kuap(void)
{
return AMR_KUAP_BLOCKED;
}
-static inline void set_kuap(unsigned long value) { }
+static __always_inline void set_kuap(unsigned long value) { }
static __always_inline void allow_user_access(void __user *to, const void __user *from,
unsigned long size, unsigned long dir)
do_uaccess_flush();
}
-static inline unsigned long prevent_user_access_return(void)
+static __always_inline unsigned long prevent_user_access_return(void)
{
unsigned long flags = get_kuap();
return flags;
}
-static inline void restore_user_access(unsigned long flags)
+static __always_inline void restore_user_access(unsigned long flags)
{
set_kuap(flags);
if (static_branch_unlikely(&uaccess_flush_key) && flags == AMR_KUAP_BLOCKED)
/* Base PID to allocate from */
extern unsigned int mmu_base_pid;
-/*
- * memory block size used with radix translation.
- */
-extern unsigned long __ro_after_init radix_mem_block_size;
+extern unsigned long __ro_after_init memory_block_size;
#define PRTB_SIZE_SHIFT (mmu_pid_bits + 4)
#define PRTB_ENTRIES (1ul << mmu_pid_bits)
#define arch_clear_mm_cpumask_cpu(cpu, mm) \
do { \
if (cpumask_test_cpu(cpu, mm_cpumask(mm))) { \
- atomic_dec(&(mm)->context.active_cpus); \
+ dec_mm_active_cpus(mm); \
cpumask_clear_cpu(cpu, mm_cpumask(mm)); \
} \
} while (0)
struct pt_regs;
void hash__do_page_fault(struct pt_regs *);
void bad_page_fault(struct pt_regs *, int);
+void emulate_single_step(struct pt_regs *regs);
extern void _exception(int, struct pt_regs *, int, unsigned long);
extern void _exception_pkey(struct pt_regs *, unsigned long, int);
extern void die(const char *, struct pt_regs *, long);
#define FCC2_MEM_OFFSET FCC_MEM_OFFSET(1)
#define FCC3_MEM_OFFSET FCC_MEM_OFFSET(2)
+/* Pipeline Maximum Depth */
+#define MPC82XX_BCR_PLDP 0x00800000
+
/* Clocks and GRG's */
enum cpm_clk_dir {
* This is also required by 52xx family.
*/
#if defined(CONFIG_SMP) || defined(CONFIG_MPC10X_BRIDGE) \
- || defined(CONFIG_PPC_83xx) || defined(CONFIG_8260) \
+ || defined(CONFIG_PPC_83xx) || defined(CONFIG_PPC_82xx) \
|| defined(CONFIG_PPC_MPC52xx)
#define CPU_FTR_COMMON CPU_FTR_NEED_COHERENT
#else
extern void register_dtl_buffer(int cpu);
extern void alloc_dtl_buffers(unsigned long *time_limit);
-extern long hcall_vphn(unsigned long cpu, u64 flags, __be32 *associativity);
#endif /* _ASM_POWERPC_DTL_H */
extern long __start__btb_flush_fixup, __stop__btb_flush_fixup;
void apply_feature_fixups(void);
+void update_mmu_feature_fixups(unsigned long mask);
void setup_feature_keys(void);
#endif
#include <sysdev/fsl_soc.h>
#include <asm/time.h>
-#ifdef CONFIG_CPM2
-#include <asm/cpm2.h>
-
-#if defined(CONFIG_8260)
-#include <asm/mpc8260.h>
-#endif
-
-#define cpm2_map(member) (&cpm2_immr->member)
-#define cpm2_map_size(member, size) (&cpm2_immr->member)
-#define cpm2_unmap(addr) do {} while(0)
-#endif
-
-#ifdef CONFIG_PPC_8xx
-#include <asm/8xx_immap.h>
-
-extern immap_t __iomem *mpc8xx_immr;
-
-#define immr_map(member) (&mpc8xx_immr->member)
-#define immr_map_size(member, size) (&mpc8xx_immr->member)
-#define immr_unmap(addr) do {} while (0)
-#endif
-
static inline int uart_baudrate(void)
{
return get_baudrate();
#define HAVE_FUNCTION_GRAPH_RET_ADDR_PTR
/* Ignore unused weak functions which will have larger offsets */
-#ifdef CONFIG_MPROFILE_KERNEL
-#define FTRACE_MCOUNT_MAX_OFFSET 12
+#if defined(CONFIG_MPROFILE_KERNEL) || defined(CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY)
+#define FTRACE_MCOUNT_MAX_OFFSET 16
#elif defined(CONFIG_PPC32)
#define FTRACE_MCOUNT_MAX_OFFSET 8
#endif
static inline unsigned long ftrace_call_adjust(unsigned long addr)
{
- /* relocation of mcount call site is the same as the address */
+ if (IS_ENABLED(CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY))
+ addr += MCOUNT_INSN_SIZE;
+
return addr;
}
unsigned long prepare_ftrace_return(unsigned long parent, unsigned long ip,
unsigned long sp);
+struct module;
+struct dyn_ftrace;
struct dyn_arch_ftrace {
struct module *mod;
};
#ifdef CONFIG_DYNAMIC_FTRACE_WITH_ARGS
+#define ftrace_need_init_nop() (true)
+int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec);
+#define ftrace_init_nop ftrace_init_nop
+
struct ftrace_regs {
struct pt_regs regs;
};
{
return get_paca()->ftrace_enabled;
}
-
-void ftrace_free_init_tramp(void);
#else /* CONFIG_PPC64 */
static inline void this_cpu_disable_ftrace(void) { }
static inline void this_cpu_enable_ftrace(void) { }
static inline void this_cpu_set_ftrace_enabled(u8 ftrace_enabled) { }
static inline u8 this_cpu_get_ftrace_enabled(void) { return 1; }
-static inline void ftrace_free_init_tramp(void) { }
#endif /* CONFIG_PPC64 */
+
+#ifdef CONFIG_FUNCTION_TRACER
+extern unsigned int ftrace_tramp_text[], ftrace_tramp_init[];
+void ftrace_free_init_tramp(void);
+#else
+static inline void ftrace_free_init_tramp(void) { }
+#endif
#endif /* !__ASSEMBLY__ */
#endif /* _ASM_POWERPC_FTRACE */
u16 len; /* length of the target data symbol */
u16 hw_len; /* length programmed in hw */
u8 flags;
+ bool perf_single_step; /* temporarily uninstalled for a perf single step */
};
/* Note: Don't change the first 6 bits below as they are in the same order
#include <linux/of_device.h>
#include <linux/of_platform.h>
+struct platform_driver;
+
extern struct bus_type ibmebus_bus_type;
int ibmebus_register_driver(struct platform_driver *drv);
#define IOMMU_PAGE_MASK(tblptr) (~((1 << (tblptr)->it_page_shift) - 1))
#define IOMMU_PAGE_ALIGN(addr, tblptr) ALIGN(addr, IOMMU_PAGE_SIZE(tblptr))
+#define DIRECT64_PROPNAME "linux,direct64-ddr-window-info"
+#define DMA64_PROPNAME "linux,dma64-ddr-window-info"
+
/* Boot time flags */
extern int iommu_is_off;
extern int iommu_force_on;
#ifdef CONFIG_PPC64
static inline bool kfence_protect_page(unsigned long addr, bool protect)
{
- struct page *page = virt_to_page(addr);
+ struct page *page = virt_to_page((void *)addr);
__kernel_map_pages(page, 1, !protect);
#define KUAP_WRITE 2
#define KUAP_READ_WRITE (KUAP_READ | KUAP_WRITE)
+#ifndef __ASSEMBLY__
+#include <linux/types.h>
+
+static __always_inline bool kuap_is_disabled(void);
+#endif
+
#ifdef CONFIG_PPC_BOOK3S_64
#include <asm/book3s/64/kup.h>
#endif
#ifdef CONFIG_PPC_KUAP
void setup_kuap(bool disabled);
+
+static __always_inline bool kuap_is_disabled(void)
+{
+ return !mmu_has_feature(MMU_FTR_KUAP);
+}
#else
static inline void setup_kuap(bool disabled) { }
static __always_inline bool kuap_is_disabled(void) { return true; }
-static inline bool
+static __always_inline bool
__bad_kuap_fault(struct pt_regs *regs, unsigned long address, bool is_write)
{
return false;
}
-static inline void __kuap_lock(void) { }
-static inline void __kuap_save_and_lock(struct pt_regs *regs) { }
-static inline void kuap_user_restore(struct pt_regs *regs) { }
-static inline void __kuap_kernel_restore(struct pt_regs *regs, unsigned long amr) { }
-
-static inline unsigned long __kuap_get_and_assert_locked(void)
-{
- return 0;
-}
+static __always_inline void kuap_user_restore(struct pt_regs *regs) { }
+static __always_inline void __kuap_kernel_restore(struct pt_regs *regs, unsigned long amr) { }
/*
* book3s/64/kup-radix.h defines these functions for the !KUAP case to flush
* platforms.
*/
#ifndef CONFIG_PPC_BOOK3S_64
-static inline void __allow_user_access(void __user *to, const void __user *from,
- unsigned long size, unsigned long dir) { }
-static inline void __prevent_user_access(unsigned long dir) { }
-static inline unsigned long __prevent_user_access_return(void) { return 0UL; }
-static inline void __restore_user_access(unsigned long flags) { }
+static __always_inline void allow_user_access(void __user *to, const void __user *from,
+ unsigned long size, unsigned long dir) { }
+static __always_inline void prevent_user_access(unsigned long dir) { }
+static __always_inline unsigned long prevent_user_access_return(void) { return 0UL; }
+static __always_inline void restore_user_access(unsigned long flags) { }
#endif /* CONFIG_PPC_BOOK3S_64 */
#endif /* CONFIG_PPC_KUAP */
return __bad_kuap_fault(regs, address, is_write);
}
-static __always_inline void kuap_assert_locked(void)
-{
- if (kuap_is_disabled())
- return;
-
- if (IS_ENABLED(CONFIG_PPC_KUAP_DEBUG))
- __kuap_get_and_assert_locked();
-}
-
static __always_inline void kuap_lock(void)
{
+#ifdef __kuap_lock
if (kuap_is_disabled())
return;
__kuap_lock();
+#endif
}
static __always_inline void kuap_save_and_lock(struct pt_regs *regs)
{
+#ifdef __kuap_save_and_lock
if (kuap_is_disabled())
return;
__kuap_save_and_lock(regs);
+#endif
}
static __always_inline void kuap_kernel_restore(struct pt_regs *regs, unsigned long amr)
static __always_inline unsigned long kuap_get_and_assert_locked(void)
{
- if (kuap_is_disabled())
- return 0;
-
- return __kuap_get_and_assert_locked();
-}
-
-#ifndef CONFIG_PPC_BOOK3S_64
-static __always_inline void allow_user_access(void __user *to, const void __user *from,
- unsigned long size, unsigned long dir)
-{
- if (kuap_is_disabled())
- return;
-
- __allow_user_access(to, from, size, dir);
-}
-
-static __always_inline void prevent_user_access(unsigned long dir)
-{
- if (kuap_is_disabled())
- return;
-
- __prevent_user_access(dir);
-}
-
-static __always_inline unsigned long prevent_user_access_return(void)
-{
- if (kuap_is_disabled())
- return 0;
-
- return __prevent_user_access_return();
+#ifdef __kuap_get_and_assert_locked
+ if (!kuap_is_disabled())
+ return __kuap_get_and_assert_locked();
+#endif
+ return 0;
}
-static __always_inline void restore_user_access(unsigned long flags)
+static __always_inline void kuap_assert_locked(void)
{
- if (kuap_is_disabled())
- return;
-
- __restore_user_access(flags);
+ if (IS_ENABLED(CONFIG_PPC_KUAP_DEBUG))
+ kuap_get_and_assert_locked();
}
-#endif /* CONFIG_PPC_BOOK3S_64 */
static __always_inline void allow_read_from_user(const void __user *from, unsigned long size)
{
#ifndef _ASM_POWERPC_LPPACA_H
#define _ASM_POWERPC_LPPACA_H
-/*
- * The below VPHN macros are outside the __KERNEL__ check since these are
- * used for compiling the vphn selftest in userspace
- */
-
-/* The H_HOME_NODE_ASSOCIATIVITY h_call returns 6 64-bit registers. */
-#define VPHN_REGISTER_COUNT 6
-
-/*
- * 6 64-bit registers unpacked into up to 24 be32 associativity values. To
- * form the complete property we have to add the length in the first cell.
- */
-#define VPHN_ASSOC_BUFSIZE (VPHN_REGISTER_COUNT*sizeof(u64)/sizeof(u16) + 1)
-
-/*
- * The H_HOME_NODE_ASSOCIATIVITY hcall takes two values for flags:
- * 1 for retrieving associativity information for a guest cpu
- * 2 for retrieving associativity information for a host/hypervisor cpu
- */
-#define VPHN_FLAG_VCPU 1
-#define VPHN_FLAG_PCPU 2
-
#ifdef __KERNEL__
/*
#include <asm/types.h>
#include <asm/mmu.h>
#include <asm/firmware.h>
+#include <asm/paca.h>
/*
* The lppaca is the "virtual processor area" registered with the hypervisor,
*/
#define LPPACA_OLD_SHARED_PROC 2
-static inline bool lppaca_shared_proc(struct lppaca *l)
+#ifdef CONFIG_PPC_PSERIES
+/*
+ * All CPUs should have the same shared proc value, so directly access the PACA
+ * to avoid false positives from DEBUG_PREEMPT.
+ */
+static inline bool lppaca_shared_proc(void)
{
+ struct lppaca *l = local_paca->lppaca_ptr;
+
if (!firmware_has_feature(FW_FEATURE_SPLPAR))
return false;
return !!(l->__old_status & LPPACA_OLD_SHARED_PROC);
}
+#define get_lppaca() (get_paca()->lppaca_ptr)
+#endif
+
/*
* SLB shadow buffer structure as defined in the PAPR. The save_area
* contains adjacent ESID and VSID pairs for each shadowed SLB. The
} save_area[SLB_NUM_BOLTED];
} ____cacheline_aligned;
-extern long hcall_vphn(unsigned long cpu, u64 flags, __be32 *associativity);
-
#endif /* CONFIG_PPC_BOOK3S */
#endif /* __KERNEL__ */
#endif /* _ASM_POWERPC_LPPACA_H */
#define __MACIO_ASIC_H__
#ifdef __KERNEL__
-#include <linux/of_device.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
extern struct bus_type macio_bus_type;
* key 0 controlling userspace addresses on radix
* Key 3 on hash
*/
-#define MMU_FTR_BOOK3S_KUAP ASM_CONST(0x00000200)
+#define MMU_FTR_KUAP ASM_CONST(0x00000200)
/*
* Supports KUEP feature
typedef pte_t *pgtable_t;
-#ifdef CONFIG_PPC_E500
-#include <asm/percpu.h>
-DECLARE_PER_CPU(int, next_tlbcam_idx);
-#endif
-
enum {
MMU_FTRS_POSSIBLE =
#if defined(CONFIG_PPC_BOOK3S_604)
#endif /* CONFIG_PPC_RADIX_MMU */
#endif
#ifdef CONFIG_PPC_KUAP
- MMU_FTR_BOOK3S_KUAP |
+ MMU_FTR_KUAP |
#endif /* CONFIG_PPC_KUAP */
#ifdef CONFIG_PPC_MEM_KEYS
MMU_FTR_PKEY |
static inline void dec_mm_active_cpus(struct mm_struct *mm)
{
+ VM_WARN_ON_ONCE(atomic_read(&mm->context.active_cpus) <= 0);
atomic_dec(&mm->context.active_cpus);
}
#endif
#ifdef CONFIG_DYNAMIC_FTRACE
-# ifdef MODULE
- asm(".section .ftrace.tramp,\"ax\",@nobits; .align 3; .previous");
-# endif /* MODULE */
-
int module_trampoline_target(struct module *mod, unsigned long trampoline,
unsigned long *target);
int module_finalize_ftrace(struct module *mod, const Elf_Shdr *sechdrs);
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0 */
-/*
- * Since there are many different boards and no standard configuration,
- * we have a unique include file for each. Rather than change every
- * file that has to include MPC8260 configuration, they all include
- * this one and the configuration switching is done here.
- */
-#ifdef __KERNEL__
-#ifndef __ASM_POWERPC_MPC8260_H__
-#define __ASM_POWERPC_MPC8260_H__
-
-#define MPC82XX_BCR_PLDP 0x00800000 /* Pipeline Maximum Depth */
-
-#ifdef CONFIG_8260
-
-#ifdef CONFIG_PCI_8260
-#include <platforms/82xx/m82xx_pci.h>
-#endif
-
-#endif /* CONFIG_8260 */
-#endif /* !__ASM_POWERPC_MPC8260_H__ */
-#endif /* __KERNEL__ */
#ifndef __ASSEMBLY__
-#include <linux/jump_label.h>
-
#include <asm/reg.h>
-extern struct static_key_false disable_kuap_key;
-
-static __always_inline bool kuap_is_disabled(void)
-{
- return static_branch_unlikely(&disable_kuap_key);
-}
-
-static inline void __kuap_lock(void)
-{
-}
-
-static inline void __kuap_save_and_lock(struct pt_regs *regs)
+static __always_inline void __kuap_save_and_lock(struct pt_regs *regs)
{
regs->kuap = mfspr(SPRN_MD_AP);
mtspr(SPRN_MD_AP, MD_APG_KUAP);
}
+#define __kuap_save_and_lock __kuap_save_and_lock
-static inline void kuap_user_restore(struct pt_regs *regs)
+static __always_inline void kuap_user_restore(struct pt_regs *regs)
{
}
-static inline void __kuap_kernel_restore(struct pt_regs *regs, unsigned long kuap)
+static __always_inline void __kuap_kernel_restore(struct pt_regs *regs, unsigned long kuap)
{
mtspr(SPRN_MD_AP, regs->kuap);
}
-static inline unsigned long __kuap_get_and_assert_locked(void)
+#ifdef CONFIG_PPC_KUAP_DEBUG
+static __always_inline unsigned long __kuap_get_and_assert_locked(void)
{
- unsigned long kuap;
+ WARN_ON_ONCE(mfspr(SPRN_MD_AP) >> 16 != MD_APG_KUAP >> 16);
- kuap = mfspr(SPRN_MD_AP);
+ return 0;
+}
+#define __kuap_get_and_assert_locked __kuap_get_and_assert_locked
+#endif
- if (IS_ENABLED(CONFIG_PPC_KUAP_DEBUG))
- WARN_ON_ONCE(kuap >> 16 != MD_APG_KUAP >> 16);
+static __always_inline void uaccess_begin_8xx(unsigned long val)
+{
+ asm(ASM_MMU_FTR_IFSET("mtspr %0, %1", "", %2) : :
+ "i"(SPRN_MD_AP), "r"(val), "i"(MMU_FTR_KUAP) : "memory");
+}
- return kuap;
+static __always_inline void uaccess_end_8xx(void)
+{
+ asm(ASM_MMU_FTR_IFSET("mtspr %0, %1", "", %2) : :
+ "i"(SPRN_MD_AP), "r"(MD_APG_KUAP), "i"(MMU_FTR_KUAP) : "memory");
}
-static inline void __allow_user_access(void __user *to, const void __user *from,
- unsigned long size, unsigned long dir)
+static __always_inline void allow_user_access(void __user *to, const void __user *from,
+ unsigned long size, unsigned long dir)
{
- mtspr(SPRN_MD_AP, MD_APG_INIT);
+ uaccess_begin_8xx(MD_APG_INIT);
}
-static inline void __prevent_user_access(unsigned long dir)
+static __always_inline void prevent_user_access(unsigned long dir)
{
- mtspr(SPRN_MD_AP, MD_APG_KUAP);
+ uaccess_end_8xx();
}
-static inline unsigned long __prevent_user_access_return(void)
+static __always_inline unsigned long prevent_user_access_return(void)
{
unsigned long flags;
flags = mfspr(SPRN_MD_AP);
- mtspr(SPRN_MD_AP, MD_APG_KUAP);
+ uaccess_end_8xx();
return flags;
}
-static inline void __restore_user_access(unsigned long flags)
+static __always_inline void restore_user_access(unsigned long flags)
{
- mtspr(SPRN_MD_AP, flags);
+ uaccess_begin_8xx(flags);
}
-static inline bool
+static __always_inline bool
__bad_kuap_fault(struct pt_regs *regs, unsigned long address, bool is_write)
{
return !((regs->kuap ^ MD_APG_KUAP) & 0xff000000);
#define pmd_pfn(pmd) (pmd_val(pmd) >> PAGE_SHIFT)
#else
#define pmd_page_vaddr(pmd) \
- ((unsigned long)(pmd_val(pmd) & ~(PTE_TABLE_SIZE - 1)))
+ ((const void *)(pmd_val(pmd) & ~(PTE_TABLE_SIZE - 1)))
#define pmd_pfn(pmd) (__pa(pmd_val(pmd)) >> PAGE_SHIFT)
#endif
#define pmd_bad(pmd) (!is_kernel_addr(pmd_val(pmd)) \
|| (pmd_val(pmd) & PMD_BAD_BITS))
#define pmd_present(pmd) (!pmd_none(pmd))
-#define pmd_page_vaddr(pmd) (pmd_val(pmd) & ~PMD_MASKED_BITS)
+#define pmd_page_vaddr(pmd) ((const void *)(pmd_val(pmd) & ~PMD_MASKED_BITS))
extern struct page *pmd_page(pmd_t pmd);
#define pmd_pfn(pmd) (page_to_pfn(pmd_page(pmd)))
#define _ASM_POWERPC_KUP_BOOKE_H_
#include <asm/bug.h>
+#include <asm/mmu.h>
#ifdef CONFIG_PPC_KUAP
#else
-#include <linux/jump_label.h>
#include <linux/sched.h>
#include <asm/reg.h>
-extern struct static_key_false disable_kuap_key;
-
-static __always_inline bool kuap_is_disabled(void)
-{
- return static_branch_unlikely(&disable_kuap_key);
-}
-
-static inline void __kuap_lock(void)
+static __always_inline void __kuap_lock(void)
{
mtspr(SPRN_PID, 0);
isync();
}
+#define __kuap_lock __kuap_lock
-static inline void __kuap_save_and_lock(struct pt_regs *regs)
+static __always_inline void __kuap_save_and_lock(struct pt_regs *regs)
{
regs->kuap = mfspr(SPRN_PID);
mtspr(SPRN_PID, 0);
isync();
}
+#define __kuap_save_and_lock __kuap_save_and_lock
-static inline void kuap_user_restore(struct pt_regs *regs)
+static __always_inline void kuap_user_restore(struct pt_regs *regs)
{
if (kuap_is_disabled())
return;
/* Context synchronisation is performed by rfi */
}
-static inline void __kuap_kernel_restore(struct pt_regs *regs, unsigned long kuap)
+static __always_inline void __kuap_kernel_restore(struct pt_regs *regs, unsigned long kuap)
{
if (regs->kuap)
mtspr(SPRN_PID, current->thread.pid);
/* Context synchronisation is performed by rfi */
}
-static inline unsigned long __kuap_get_and_assert_locked(void)
+#ifdef CONFIG_PPC_KUAP_DEBUG
+static __always_inline unsigned long __kuap_get_and_assert_locked(void)
{
- unsigned long kuap = mfspr(SPRN_PID);
+ WARN_ON_ONCE(mfspr(SPRN_PID));
- if (IS_ENABLED(CONFIG_PPC_KUAP_DEBUG))
- WARN_ON_ONCE(kuap);
+ return 0;
+}
+#define __kuap_get_and_assert_locked __kuap_get_and_assert_locked
+#endif
- return kuap;
+static __always_inline void uaccess_begin_booke(unsigned long val)
+{
+ asm(ASM_MMU_FTR_IFSET("mtspr %0, %1; isync", "", %2) : :
+ "i"(SPRN_PID), "r"(val), "i"(MMU_FTR_KUAP) : "memory");
}
-static inline void __allow_user_access(void __user *to, const void __user *from,
- unsigned long size, unsigned long dir)
+static __always_inline void uaccess_end_booke(void)
{
- mtspr(SPRN_PID, current->thread.pid);
- isync();
+ asm(ASM_MMU_FTR_IFSET("mtspr %0, %1; isync", "", %2) : :
+ "i"(SPRN_PID), "r"(0), "i"(MMU_FTR_KUAP) : "memory");
}
-static inline void __prevent_user_access(unsigned long dir)
+static __always_inline void allow_user_access(void __user *to, const void __user *from,
+ unsigned long size, unsigned long dir)
{
- mtspr(SPRN_PID, 0);
- isync();
+ uaccess_begin_booke(current->thread.pid);
+}
+
+static __always_inline void prevent_user_access(unsigned long dir)
+{
+ uaccess_end_booke();
}
-static inline unsigned long __prevent_user_access_return(void)
+static __always_inline unsigned long prevent_user_access_return(void)
{
unsigned long flags = mfspr(SPRN_PID);
- mtspr(SPRN_PID, 0);
- isync();
+ uaccess_end_booke();
return flags;
}
-static inline void __restore_user_access(unsigned long flags)
+static __always_inline void restore_user_access(unsigned long flags)
{
- if (flags) {
- mtspr(SPRN_PID, current->thread.pid);
- isync();
- }
+ if (flags)
+ uaccess_begin_booke(current->thread.pid);
}
-static inline bool
+static __always_inline bool
__bad_kuap_fault(struct pt_regs *regs, unsigned long address, bool is_write)
{
return !regs->kuap;
#endif
+#include <asm/percpu.h>
+DECLARE_PER_CPU(int, next_tlbcam_idx);
+
#endif /* !__ASSEMBLY__ */
#endif /* _ASM_POWERPC_MMU_BOOK3E_H_ */
#include <linux/cache.h>
#include <linux/string.h>
#include <asm/types.h>
-#include <asm/lppaca.h>
#include <asm/mmu.h>
#include <asm/page.h>
#ifdef CONFIG_PPC_BOOK3E_64
#define get_paca() local_paca
#endif
-#ifdef CONFIG_PPC_PSERIES
-#define get_lppaca() (get_paca()->lppaca_ptr)
-#endif
-
#define get_slb_shadow() (get_paca()->slb_shadow_ptr)
struct task_struct;
struct rtas_args;
+struct lppaca;
/*
* Defines the layout of the paca.
#ifndef __ASSEMBLY__
#include <linux/types.h>
#include <linux/kernel.h>
+#include <linux/bug.h>
#else
#include <asm/types.h>
#endif
#define ARCH_PFN_OFFSET ((unsigned long)(MEMORY_START >> PAGE_SHIFT))
#endif
-#define virt_to_pfn(kaddr) (__pa(kaddr) >> PAGE_SHIFT)
-#define virt_to_page(kaddr) pfn_to_page(virt_to_pfn(kaddr))
-#define pfn_to_kaddr(pfn) __va((pfn) << PAGE_SHIFT)
-
-#define virt_addr_valid(vaddr) ({ \
- unsigned long _addr = (unsigned long)vaddr; \
- _addr >= PAGE_OFFSET && _addr < (unsigned long)high_memory && \
- pfn_valid(virt_to_pfn(_addr)); \
-})
-
/*
* On Book-E parts we need __va to parse the device tree and we can't
* determine MEMORY_START until then. However we can determine PHYSICAL_START
#endif
#endif
+#ifndef __ASSEMBLY__
+static inline unsigned long virt_to_pfn(const void *kaddr)
+{
+ return __pa(kaddr) >> PAGE_SHIFT;
+}
+
+static inline const void *pfn_to_kaddr(unsigned long pfn)
+{
+ return __va(pfn << PAGE_SHIFT);
+}
+#endif
+
+#define virt_to_page(kaddr) pfn_to_page(virt_to_pfn(kaddr))
+#define virt_addr_valid(vaddr) ({ \
+ unsigned long _addr = (unsigned long)vaddr; \
+ _addr >= PAGE_OFFSET && _addr < (unsigned long)high_memory && \
+ pfn_valid(virt_to_pfn((void *)_addr)); \
+})
+
/*
* Unfortunately the PLT is in the BSS in the PPC32 ELF ABI,
* and needs to be executable. This means the whole heap ends
#include <asm/smp.h>
#ifdef CONFIG_PPC64
#include <asm/paca.h>
+#include <asm/lppaca.h>
#include <asm/hvcall.h>
#endif
extern int pci_mmap_legacy_page_range(struct pci_bus *bus,
struct vm_area_struct *vma,
enum pci_mmap_state mmap_state);
-
+extern void pci_adjust_legacy_attr(struct pci_bus *bus,
+ enum pci_mmap_state mmap_type);
#define HAVE_PCI_LEGACY 1
extern void pcibios_claim_one_bus(struct pci_bus *b);
}
#ifndef pmd_page_vaddr
-static inline unsigned long pmd_page_vaddr(pmd_t pmd)
+static inline const void *pmd_page_vaddr(pmd_t pmd)
{
- return ((unsigned long)__va(pmd_val(pmd) & ~PMD_MASKED_BITS));
+ return __va(pmd_val(pmd) & ~PMD_MASKED_BITS);
}
#define pmd_page_vaddr pmd_page_vaddr
#endif
#include <asm/hvcall.h>
#include <asm/paca.h>
+#include <asm/lppaca.h>
#include <asm/page.h>
static inline long poll_pending(void)
#define PPC_RAW_RFCI (0x4c000066)
#define PPC_RAW_RFDI (0x4c00004e)
#define PPC_RAW_RFMCI (0x4c00004c)
+#define PPC_RAW_TLBILX_LPID (0x7c000024)
#define PPC_RAW_TLBILX(t, a, b) (0x7c000024 | __PPC_T_TLB(t) | __PPC_RA0(a) | __PPC_RB(b))
#define PPC_RAW_WAIT_v203 (0x7c00007c)
#define PPC_RAW_WAIT(w, p) (0x7c00003c | __PPC_WC(w) | __PPC_PL(p))
#define PPC_TLBILX(t, a, b) stringify_in_c(.long PPC_RAW_TLBILX(t, a, b))
#define PPC_TLBILX_ALL(a, b) PPC_TLBILX(0, a, b)
#define PPC_TLBILX_PID(a, b) PPC_TLBILX(1, a, b)
+#define PPC_TLBILX_LPID stringify_in_c(.long PPC_RAW_TLBILX_LPID)
#define PPC_TLBILX_VA(a, b) PPC_TLBILX(3, a, b)
#define PPC_WAIT_v203 stringify_in_c(.long PPC_RAW_WAIT_v203)
#define PPC_WAIT(w, p) stringify_in_c(.long PPC_RAW_WAIT(w, p))
unsigned int align_ctl; /* alignment handling control */
#ifdef CONFIG_HAVE_HW_BREAKPOINT
struct perf_event *ptrace_bps[HBP_NUM_MAX];
- /*
- * Helps identify source of single-step exception and subsequent
- * hw-breakpoint enablement
- */
- struct perf_event *last_hit_ubp[HBP_NUM_MAX];
#endif /* CONFIG_HAVE_HW_BREAKPOINT */
struct arch_hw_breakpoint hw_brk[HBP_NUM_MAX]; /* hardware breakpoint info */
unsigned long trap_nr; /* last trap # on this thread */
#define mfspr(rn) ({unsigned long rval; \
asm volatile("mfspr %0," __stringify(rn) \
: "=r" (rval)); rval;})
-#ifndef mtspr
#define mtspr(rn, v) asm volatile("mtspr " __stringify(rn) ",%0" : \
: "r" ((unsigned long)(v)) \
: "memory")
-#endif
#define wrtspr(rn) asm volatile("mtspr " __stringify(rn) ",2" : : : "memory")
static inline void wrtee(unsigned long val)
#define RTAS_USER_REGION_SIZE (64 * 1024)
/* RTAS return status codes */
+#define RTAS_HARDWARE_ERROR -1 /* Hardware Error */
#define RTAS_BUSY -2 /* RTAS Busy */
+#define RTAS_INVALID_PARAMETER -3 /* Invalid indicator/domain/sensor etc. */
#define RTAS_EXTENDED_DELAY_MIN 9900
#define RTAS_EXTENDED_DELAY_MAX 9905
extern int rtas_set_indicator_fast(int indicator, int index, int new_value);
extern void rtas_progress(char *s, unsigned short hex);
int rtas_ibm_suspend_me(int *fw_status);
+int rtas_error_rc(int rtas_rc);
struct rtc_time;
extern time64_t rtas_get_boot_time(void);
(unsigned long)_stext < end;
}
+#else
+static inline unsigned long kernel_toc_addr(void) { BUILD_BUG(); return -1UL; }
#endif
#endif /* __KERNEL__ */
extern void ppc_printk_progress(char *s, unsigned short hex);
extern unsigned long long memory_limit;
-extern void *zalloc_maybe_bootmem(size_t size, gfp_t mask);
struct device_node;
#endif
#endif
+#ifdef CONFIG_HOTPLUG_SMT
+#include <linux/cpu_smt.h>
+#include <asm/cputhreads.h>
+
+static inline bool topology_is_primary_thread(unsigned int cpu)
+{
+ return cpu == cpu_first_thread_sibling(cpu);
+}
+
+static inline bool topology_smt_thread_allowed(unsigned int cpu)
+{
+ return cpu_thread_in_core(cpu) < cpu_smt_num_threads;
+}
+#endif
+
#endif /* __KERNEL__ */
#endif /* _ASM_POWERPC_TOPOLOGY_H */
extern long __copy_from_user_flushcache(void *dst, const void __user *src,
unsigned size);
-static __must_check inline bool user_access_begin(const void __user *ptr, size_t len)
+static __must_check __always_inline bool user_access_begin(const void __user *ptr, size_t len)
{
if (unlikely(!access_ok(ptr, len)))
return false;
#define user_access_save prevent_user_access_return
#define user_access_restore restore_user_access
-static __must_check inline bool
+static __must_check __always_inline bool
user_read_access_begin(const void __user *ptr, size_t len)
{
if (unlikely(!access_ok(ptr, len)))
#define user_read_access_begin user_read_access_begin
#define user_read_access_end prevent_current_read_from_user
-static __must_check inline bool
+static __must_check __always_inline bool
user_write_access_begin(const void __user *ptr, size_t len)
{
if (unlikely(!access_ok(ptr, len)))
#ifndef _ASM_VERMAGIC_H
#define _ASM_VERMAGIC_H
-#ifdef CONFIG_MPROFILE_KERNEL
+#ifdef CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY
+#define MODULE_ARCH_VERMAGIC_FTRACE "patchable-function-entry "
+#elif defined(CONFIG_MPROFILE_KERNEL)
#define MODULE_ARCH_VERMAGIC_FTRACE "mprofile-kernel "
#else
#define MODULE_ARCH_VERMAGIC_FTRACE ""
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+#ifndef _ASM_POWERPC_VPHN_H
+#define _ASM_POWERPC_VPHN_H
+
+/* The H_HOME_NODE_ASSOCIATIVITY h_call returns 6 64-bit registers. */
+#define VPHN_REGISTER_COUNT 6
+
+/*
+ * 6 64-bit registers unpacked into up to 24 be32 associativity values. To
+ * form the complete property we have to add the length in the first cell.
+ */
+#define VPHN_ASSOC_BUFSIZE (VPHN_REGISTER_COUNT*sizeof(u64)/sizeof(u16) + 1)
+
+/*
+ * The H_HOME_NODE_ASSOCIATIVITY hcall takes two values for flags:
+ * 1 for retrieving associativity information for a guest cpu
+ * 2 for retrieving associativity information for a host/hypervisor cpu
+ */
+#define VPHN_FLAG_VCPU 1
+#define VPHN_FLAG_PCPU 2
+
+long hcall_vphn(unsigned long cpu, u64 flags, __be32 *associativity);
+
+#endif // _ASM_POWERPC_VPHN_H
#include <linux/audit.h>
#include <asm/unistd.h>
+#include "audit_32.h"
+
static unsigned dir_class[] = {
#include <asm-generic/audit_dir_write.h>
~0U
int audit_classify_syscall(int abi, unsigned syscall)
{
#ifdef CONFIG_PPC64
- extern int ppc32_classify_syscall(unsigned);
if (abi == AUDIT_ARCH_PPC)
return ppc32_classify_syscall(syscall);
#endif
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+#ifndef __AUDIT_32_H__
+#define __AUDIT_32_H__
+
+extern int ppc32_classify_syscall(unsigned);
+
+#endif
#include <linux/audit_arch.h>
#include <asm/unistd.h>
+#include "audit_32.h"
+
unsigned ppc32_dir_class[] = {
#include <asm-generic/audit_dir_write.h>
~0U
t->cpu_features |= old.cpu_features & CPU_FTR_PMAO_BUG;
}
+ /* Set kuap ON at startup, will be disabled later if cmdline has 'nosmap' */
+ if (IS_ENABLED(CONFIG_PPC_KUAP) && IS_ENABLED(CONFIG_PPC32))
+ t->mmu_features |= MMU_FTR_KUAP;
+
*PTRRELOC(&cur_cpu_spec) = &the_cpu_spec;
/*
#include <asm/asm-offsets.h>
#include <asm/unistd.h>
#include <asm/ptrace.h>
-#include <asm/export.h>
#include <asm/feature-fixups.h>
#include <asm/barrier.h>
#include <asm/kup.h>
* Copyright (C) 2012 Freescale Semiconductor, Inc.
*/
+#include <linux/export.h>
#include <linux/threads.h>
#include <asm/epapr_hcalls.h>
#include <asm/reg.h>
#include <asm/ppc_asm.h>
#include <asm/asm-compat.h>
#include <asm/asm-offsets.h>
-#include <asm/export.h>
#ifndef CONFIG_PPC64
/* epapr_ev_idle() was derived from e500_idle() */
return ret;
error_out:
fw_dump.fadump_enabled = 0;
+ fw_dump.reserve_dump_area_size = 0;
return 0;
}
* Copyright (C) 1997 Dan Malek (dmalek@jlc.net).
*/
+#include <linux/export.h>
#include <asm/reg.h>
#include <asm/page.h>
#include <asm/mmu.h>
#include <asm/ppc_asm.h>
#include <asm/asm-offsets.h>
#include <asm/ptrace.h>
-#include <asm/export.h>
#include <asm/asm-compat.h>
#include <asm/feature-fixups.h>
#include <asm/ppc_asm.h>
#include <asm/asm-offsets.h>
#include <asm/ptrace.h>
-#include <asm/export.h>
#include "head_32.h"
#include <asm/asm-offsets.h>
#include <asm/ptrace.h>
#include <asm/synch.h>
-#include <asm/export.h>
#include <asm/code-patching-asm.h>
#include "head_booke.h"
#include <asm/hw_irq.h>
#include <asm/cputhreads.h>
#include <asm/ppc-opcode.h>
-#include <asm/export.h>
#include <asm/feature-fixups.h>
#ifdef CONFIG_PPC_BOOK3S
#include <asm/exception-64s.h>
#include <asm/asm-offsets.h>
#include <asm/cache.h>
#include <asm/ptrace.h>
-#include <asm/export.h>
#include <asm/feature-fixups.h>
#include "head_booke.h"
#include <asm/ppc_asm.h>
#include <asm/asm-offsets.h>
#include <asm/ptrace.h>
-#include <asm/export.h>
#include <asm/code-patching-asm.h>
#include <asm/interrupt.h>
#include <asm/ptrace.h>
#include <asm/bug.h>
#include <asm/kvm_book3s_asm.h>
-#include <asm/export.h>
#include <asm/feature-fixups.h>
#include <asm/interrupt.h>
return 0; /* no instruction breakpoints available */
}
-static bool single_step_pending(void)
-{
- int i;
-
- for (i = 0; i < nr_wp_slots(); i++) {
- if (current->thread.last_hit_ubp[i])
- return true;
- }
- return false;
-}
/*
* Install a perf counter breakpoint.
* Do not install DABR values if the instruction must be single-stepped.
* If so, DABR will be populated in single_step_dabr_instruction().
*/
- if (!single_step_pending())
+ if (!info->perf_single_step)
__set_breakpoint(i, info);
return 0;
return bp->overflow_handler == ptrace_triggered;
}
-struct breakpoint {
- struct list_head list;
- struct perf_event *bp;
- bool ptrace_bp;
-};
-
-/*
- * While kernel/events/hw_breakpoint.c does its own synchronization, we cannot
- * rely on it safely synchronizing internals here; however, we can rely on it
- * not requesting more breakpoints than available.
- */
-static DEFINE_SPINLOCK(cpu_bps_lock);
-static DEFINE_PER_CPU(struct breakpoint *, cpu_bps[HBP_NUM_MAX]);
-static DEFINE_SPINLOCK(task_bps_lock);
-static LIST_HEAD(task_bps);
-
-static struct breakpoint *alloc_breakpoint(struct perf_event *bp)
-{
- struct breakpoint *tmp;
-
- tmp = kzalloc(sizeof(*tmp), GFP_KERNEL);
- if (!tmp)
- return ERR_PTR(-ENOMEM);
- tmp->bp = bp;
- tmp->ptrace_bp = is_ptrace_bp(bp);
- return tmp;
-}
-
-static bool bp_addr_range_overlap(struct perf_event *bp1, struct perf_event *bp2)
-{
- __u64 bp1_saddr, bp1_eaddr, bp2_saddr, bp2_eaddr;
-
- bp1_saddr = ALIGN_DOWN(bp1->attr.bp_addr, HW_BREAKPOINT_SIZE);
- bp1_eaddr = ALIGN(bp1->attr.bp_addr + bp1->attr.bp_len, HW_BREAKPOINT_SIZE);
- bp2_saddr = ALIGN_DOWN(bp2->attr.bp_addr, HW_BREAKPOINT_SIZE);
- bp2_eaddr = ALIGN(bp2->attr.bp_addr + bp2->attr.bp_len, HW_BREAKPOINT_SIZE);
-
- return (bp1_saddr < bp2_eaddr && bp1_eaddr > bp2_saddr);
-}
-
-static bool alternate_infra_bp(struct breakpoint *b, struct perf_event *bp)
-{
- return is_ptrace_bp(bp) ? !b->ptrace_bp : b->ptrace_bp;
-}
-
-static bool can_co_exist(struct breakpoint *b, struct perf_event *bp)
-{
- return !(alternate_infra_bp(b, bp) && bp_addr_range_overlap(b->bp, bp));
-}
-
-static int task_bps_add(struct perf_event *bp)
-{
- struct breakpoint *tmp;
-
- tmp = alloc_breakpoint(bp);
- if (IS_ERR(tmp))
- return PTR_ERR(tmp);
-
- spin_lock(&task_bps_lock);
- list_add(&tmp->list, &task_bps);
- spin_unlock(&task_bps_lock);
- return 0;
-}
-
-static void task_bps_remove(struct perf_event *bp)
-{
- struct list_head *pos, *q;
-
- spin_lock(&task_bps_lock);
- list_for_each_safe(pos, q, &task_bps) {
- struct breakpoint *tmp = list_entry(pos, struct breakpoint, list);
-
- if (tmp->bp == bp) {
- list_del(&tmp->list);
- kfree(tmp);
- break;
- }
- }
- spin_unlock(&task_bps_lock);
-}
-
-/*
- * If any task has breakpoint from alternate infrastructure,
- * return true. Otherwise return false.
- */
-static bool all_task_bps_check(struct perf_event *bp)
-{
- struct breakpoint *tmp;
- bool ret = false;
-
- spin_lock(&task_bps_lock);
- list_for_each_entry(tmp, &task_bps, list) {
- if (!can_co_exist(tmp, bp)) {
- ret = true;
- break;
- }
- }
- spin_unlock(&task_bps_lock);
- return ret;
-}
-
-/*
- * If same task has breakpoint from alternate infrastructure,
- * return true. Otherwise return false.
- */
-static bool same_task_bps_check(struct perf_event *bp)
-{
- struct breakpoint *tmp;
- bool ret = false;
-
- spin_lock(&task_bps_lock);
- list_for_each_entry(tmp, &task_bps, list) {
- if (tmp->bp->hw.target == bp->hw.target &&
- !can_co_exist(tmp, bp)) {
- ret = true;
- break;
- }
- }
- spin_unlock(&task_bps_lock);
- return ret;
-}
-
-static int cpu_bps_add(struct perf_event *bp)
-{
- struct breakpoint **cpu_bp;
- struct breakpoint *tmp;
- int i = 0;
-
- tmp = alloc_breakpoint(bp);
- if (IS_ERR(tmp))
- return PTR_ERR(tmp);
-
- spin_lock(&cpu_bps_lock);
- cpu_bp = per_cpu_ptr(cpu_bps, bp->cpu);
- for (i = 0; i < nr_wp_slots(); i++) {
- if (!cpu_bp[i]) {
- cpu_bp[i] = tmp;
- break;
- }
- }
- spin_unlock(&cpu_bps_lock);
- return 0;
-}
-
-static void cpu_bps_remove(struct perf_event *bp)
-{
- struct breakpoint **cpu_bp;
- int i = 0;
-
- spin_lock(&cpu_bps_lock);
- cpu_bp = per_cpu_ptr(cpu_bps, bp->cpu);
- for (i = 0; i < nr_wp_slots(); i++) {
- if (!cpu_bp[i])
- continue;
-
- if (cpu_bp[i]->bp == bp) {
- kfree(cpu_bp[i]);
- cpu_bp[i] = NULL;
- break;
- }
- }
- spin_unlock(&cpu_bps_lock);
-}
-
-static bool cpu_bps_check(int cpu, struct perf_event *bp)
-{
- struct breakpoint **cpu_bp;
- bool ret = false;
- int i;
-
- spin_lock(&cpu_bps_lock);
- cpu_bp = per_cpu_ptr(cpu_bps, cpu);
- for (i = 0; i < nr_wp_slots(); i++) {
- if (cpu_bp[i] && !can_co_exist(cpu_bp[i], bp)) {
- ret = true;
- break;
- }
- }
- spin_unlock(&cpu_bps_lock);
- return ret;
-}
-
-static bool all_cpu_bps_check(struct perf_event *bp)
-{
- int cpu;
-
- for_each_online_cpu(cpu) {
- if (cpu_bps_check(cpu, bp))
- return true;
- }
- return false;
-}
-
-int arch_reserve_bp_slot(struct perf_event *bp)
-{
- int ret;
-
- /* ptrace breakpoint */
- if (is_ptrace_bp(bp)) {
- if (all_cpu_bps_check(bp))
- return -ENOSPC;
-
- if (same_task_bps_check(bp))
- return -ENOSPC;
-
- return task_bps_add(bp);
- }
-
- /* perf breakpoint */
- if (is_kernel_addr(bp->attr.bp_addr))
- return 0;
-
- if (bp->hw.target && bp->cpu == -1) {
- if (same_task_bps_check(bp))
- return -ENOSPC;
-
- return task_bps_add(bp);
- } else if (!bp->hw.target && bp->cpu != -1) {
- if (all_task_bps_check(bp))
- return -ENOSPC;
-
- return cpu_bps_add(bp);
- }
-
- if (same_task_bps_check(bp))
- return -ENOSPC;
-
- ret = cpu_bps_add(bp);
- if (ret)
- return ret;
- ret = task_bps_add(bp);
- if (ret)
- cpu_bps_remove(bp);
-
- return ret;
-}
-
-void arch_release_bp_slot(struct perf_event *bp)
-{
- if (!is_kernel_addr(bp->attr.bp_addr)) {
- if (bp->hw.target)
- task_bps_remove(bp);
- if (bp->cpu != -1)
- cpu_bps_remove(bp);
- }
-}
-
-/*
- * Perform cleanup of arch-specific counters during unregistration
- * of the perf-event
- */
-void arch_unregister_hw_breakpoint(struct perf_event *bp)
-{
- /*
- * If the breakpoint is unregistered between a hw_breakpoint_handler()
- * and the single_step_dabr_instruction(), then cleanup the breakpoint
- * restoration variables to prevent dangling pointers.
- * FIXME, this should not be using bp->ctx at all! Sayeth peterz.
- */
- if (bp->ctx && bp->ctx->task && bp->ctx->task != ((void *)-1L)) {
- int i;
-
- for (i = 0; i < nr_wp_slots(); i++) {
- if (bp->ctx->task->thread.last_hit_ubp[i] == bp)
- bp->ctx->task->thread.last_hit_ubp[i] = NULL;
- }
- }
-}
-
/*
* Check for virtual address in kernel space.
*/
* Restores the breakpoint on the debug registers.
* Invoke this function if it is known that the execution context is
* about to change to cause loss of MSR_SE settings.
+ *
+ * The perf watchpoint will simply re-trigger once the thread is started again,
+ * and the watchpoint handler will set up MSR_SE and perf_single_step as
+ * needed.
*/
void thread_change_pc(struct task_struct *tsk, struct pt_regs *regs)
{
int i;
for (i = 0; i < nr_wp_slots(); i++) {
- if (unlikely(tsk->thread.last_hit_ubp[i]))
+ struct perf_event *bp = __this_cpu_read(bp_per_reg[i]);
+
+ if (unlikely(bp && counter_arch_bp(bp)->perf_single_step))
goto reset;
}
return;
for (i = 0; i < nr_wp_slots(); i++) {
info = counter_arch_bp(__this_cpu_read(bp_per_reg[i]));
__set_breakpoint(i, info);
- tsk->thread.last_hit_ubp[i] = NULL;
+ info->perf_single_step = false;
}
}
* We've failed in reliably handling the hw-breakpoint. Unregister
* it and throw a warning message to let the user know about it.
*/
-static void handler_error(struct perf_event *bp, struct arch_hw_breakpoint *info)
+static void handler_error(struct perf_event *bp)
{
WARN(1, "Unable to handle hardware breakpoint. Breakpoint at 0x%lx will be disabled.",
- info->address);
+ counter_arch_bp(bp)->address);
perf_event_disable_inatomic(bp);
}
-static void larx_stcx_err(struct perf_event *bp, struct arch_hw_breakpoint *info)
+static void larx_stcx_err(struct perf_event *bp)
{
printk_ratelimited("Breakpoint hit on instruction that can't be emulated. Breakpoint at 0x%lx will be disabled.\n",
- info->address);
+ counter_arch_bp(bp)->address);
perf_event_disable_inatomic(bp);
}
static bool stepping_handler(struct pt_regs *regs, struct perf_event **bp,
- struct arch_hw_breakpoint **info, int *hit,
- ppc_inst_t instr)
+ int *hit, ppc_inst_t instr)
{
int i;
int stepped;
for (i = 0; i < nr_wp_slots(); i++) {
if (!hit[i])
continue;
- current->thread.last_hit_ubp[i] = bp[i];
- info[i] = NULL;
+
+ counter_arch_bp(bp[i])->perf_single_step = true;
+ bp[i] = NULL;
}
regs_set_return_msr(regs, regs->msr | MSR_SE);
return false;
for (i = 0; i < nr_wp_slots(); i++) {
if (!hit[i])
continue;
- handler_error(bp[i], info[i]);
- info[i] = NULL;
+ handler_error(bp[i]);
+ bp[i] = NULL;
}
return false;
}
return true;
}
-static void handle_p10dd1_spurious_exception(struct arch_hw_breakpoint **info,
+static void handle_p10dd1_spurious_exception(struct perf_event **bp,
int *hit, unsigned long ea)
{
int i;
* spurious exception.
*/
for (i = 0; i < nr_wp_slots(); i++) {
- if (!info[i])
+ struct arch_hw_breakpoint *info;
+
+ if (!bp[i])
continue;
- hw_end_addr = ALIGN(info[i]->address + info[i]->len, HW_BREAKPOINT_SIZE);
+ info = counter_arch_bp(bp[i]);
+
+ hw_end_addr = ALIGN(info->address + info->len, HW_BREAKPOINT_SIZE);
/*
* Ending address of DAWR range is less than starting
return;
for (i = 0; i < nr_wp_slots(); i++) {
- if (info[i]) {
+ if (bp[i]) {
hit[i] = 1;
- info[i]->type |= HW_BRK_TYPE_EXTRANEOUS_IRQ;
+ counter_arch_bp(bp[i])->type |= HW_BRK_TYPE_EXTRANEOUS_IRQ;
}
}
}
int rc = NOTIFY_STOP;
struct perf_event *bp[HBP_NUM_MAX] = { NULL };
struct pt_regs *regs = args->regs;
- struct arch_hw_breakpoint *info[HBP_NUM_MAX] = { NULL };
int i;
int hit[HBP_NUM_MAX] = {0};
int nr_hit = 0;
wp_get_instr_detail(regs, &instr, &type, &size, &ea);
for (i = 0; i < nr_wp_slots(); i++) {
+ struct arch_hw_breakpoint *info;
+
bp[i] = __this_cpu_read(bp_per_reg[i]);
if (!bp[i])
continue;
- info[i] = counter_arch_bp(bp[i]);
- info[i]->type &= ~HW_BRK_TYPE_EXTRANEOUS_IRQ;
+ info = counter_arch_bp(bp[i]);
+ info->type &= ~HW_BRK_TYPE_EXTRANEOUS_IRQ;
- if (wp_check_constraints(regs, instr, ea, type, size, info[i])) {
+ if (wp_check_constraints(regs, instr, ea, type, size, info)) {
if (!IS_ENABLED(CONFIG_PPC_8xx) &&
ppc_inst_equal(instr, ppc_inst(0))) {
- handler_error(bp[i], info[i]);
- info[i] = NULL;
+ handler_error(bp[i]);
+ bp[i] = NULL;
err = 1;
continue;
}
/* Workaround for Power10 DD1 */
if (!IS_ENABLED(CONFIG_PPC_8xx) && mfspr(SPRN_PVR) == 0x800100 &&
is_octword_vsx_instr(type, size)) {
- handle_p10dd1_spurious_exception(info, hit, ea);
+ handle_p10dd1_spurious_exception(bp, hit, ea);
} else {
rc = NOTIFY_DONE;
goto out;
*/
if (ptrace_bp) {
for (i = 0; i < nr_wp_slots(); i++) {
- if (!hit[i])
+ if (!hit[i] || !is_ptrace_bp(bp[i]))
continue;
perf_bp_event(bp[i], regs);
- info[i] = NULL;
+ bp[i] = NULL;
}
rc = NOTIFY_DONE;
goto reset;
for (i = 0; i < nr_wp_slots(); i++) {
if (!hit[i])
continue;
- larx_stcx_err(bp[i], info[i]);
- info[i] = NULL;
+ larx_stcx_err(bp[i]);
+ bp[i] = NULL;
}
goto reset;
}
- if (!stepping_handler(regs, bp, info, hit, instr))
+ if (!stepping_handler(regs, bp, hit, instr))
goto reset;
}
for (i = 0; i < nr_wp_slots(); i++) {
if (!hit[i])
continue;
- if (!(info[i]->type & HW_BRK_TYPE_EXTRANEOUS_IRQ))
+ if (!(counter_arch_bp(bp[i])->type & HW_BRK_TYPE_EXTRANEOUS_IRQ))
perf_bp_event(bp[i], regs);
}
reset:
for (i = 0; i < nr_wp_slots(); i++) {
- if (!info[i])
+ if (!bp[i])
continue;
- __set_breakpoint(i, info[i]);
+ __set_breakpoint(i, counter_arch_bp(bp[i]));
}
out:
static int single_step_dabr_instruction(struct die_args *args)
{
struct pt_regs *regs = args->regs;
- struct perf_event *bp = NULL;
- struct arch_hw_breakpoint *info;
- int i;
bool found = false;
/*
* Check if we are single-stepping as a result of a
* previous HW Breakpoint exception
*/
- for (i = 0; i < nr_wp_slots(); i++) {
- bp = current->thread.last_hit_ubp[i];
+ for (int i = 0; i < nr_wp_slots(); i++) {
+ struct perf_event *bp;
+ struct arch_hw_breakpoint *info;
+
+ bp = __this_cpu_read(bp_per_reg[i]);
if (!bp)
continue;
- found = true;
info = counter_arch_bp(bp);
+ if (!info->perf_single_step)
+ continue;
+
+ found = true;
+
/*
* We shall invoke the user-defined callback function in the
* single stepping handler to confirm to 'trigger-after-execute'
*/
if (!(info->type & HW_BRK_TYPE_EXTRANEOUS_IRQ))
perf_bp_event(bp, regs);
- current->thread.last_hit_ubp[i] = NULL;
- }
- if (!found)
- return NOTIFY_DONE;
-
- for (i = 0; i < nr_wp_slots(); i++) {
- bp = __this_cpu_read(bp_per_reg[i]);
- if (!bp)
- continue;
-
- info = counter_arch_bp(bp);
- __set_breakpoint(i, info);
+ info->perf_single_step = false;
+ __set_breakpoint(i, counter_arch_bp(bp));
}
/*
* If the process was being single-stepped by ptrace, let the
* other single-step actions occur (e.g. generate SIGTRAP).
*/
- if (test_thread_flag(TIF_SINGLESTEP))
+ if (!found || test_thread_flag(TIF_SINGLESTEP))
return NOTIFY_DONE;
return NOTIFY_STOP;
return 0;
}
-static struct notifier_block fail_iommu_bus_notifier = {
+/*
+ * PCI and VIO buses need separate notifier_block structs, since they're linked
+ * list nodes. Sharing a notifier_block would mean that any notifiers later
+ * registered for PCI buses would also get called by VIO buses and vice versa.
+ */
+static struct notifier_block fail_iommu_pci_bus_notifier = {
.notifier_call = fail_iommu_bus_notify
};
+#ifdef CONFIG_IBMVIO
+static struct notifier_block fail_iommu_vio_bus_notifier = {
+ .notifier_call = fail_iommu_bus_notify
+};
+#endif
+
static int __init fail_iommu_setup(void)
{
#ifdef CONFIG_PCI
- bus_register_notifier(&pci_bus_type, &fail_iommu_bus_notifier);
+ bus_register_notifier(&pci_bus_type, &fail_iommu_pci_bus_notifier);
#endif
#ifdef CONFIG_IBMVIO
- bus_register_notifier(&vio_bus_type, &fail_iommu_bus_notifier);
+ bus_register_notifier(&vio_bus_type, &fail_iommu_vio_bus_notifier);
#endif
return 0;
#include <linux/serial_core.h>
#include <linux/console.h>
#include <linux/pci.h>
+#include <linux/of.h>
#include <linux/of_address.h>
-#include <linux/of_device.h>
#include <linux/of_irq.h>
#include <linux/serial_reg.h>
#include <asm/io.h>
*
* setjmp/longjmp code by Paul Mackerras.
*/
+#include <linux/export.h>
#include <asm/ppc_asm.h>
#include <asm/unistd.h>
#include <asm/asm-compat.h>
#include <asm/asm-offsets.h>
-#include <asm/export.h>
.text
*
*/
+#include <linux/export.h>
#include <linux/sys.h>
#include <asm/unistd.h>
#include <asm/errno.h>
#include <asm/processor.h>
#include <asm/bug.h>
#include <asm/ptrace.h>
-#include <asm/export.h>
#include <asm/feature-fixups.h>
.text
* PPC64 updates by Dave Engebretsen (engebret@us.ibm.com)
*/
+#include <linux/export.h>
#include <linux/linkage.h>
#include <linux/sys.h>
#include <asm/unistd.h>
#include <asm/kexec.h>
#include <asm/ptrace.h>
#include <asm/mmu.h>
-#include <asm/export.h>
#include <asm/feature-fixups.h>
.text
return 0;
}
-#ifdef CONFIG_MPROFILE_KERNEL
+#if defined(CONFIG_MPROFILE_KERNEL) || defined(CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY)
static u32 stub_insns[] = {
#ifdef CONFIG_PPC_KERNEL_PCREL
#include <linux/export.h>
#include <linux/mod_devicetable.h>
#include <linux/pci.h>
-#include <linux/of.h>
-#include <linux/of_device.h>
-#include <linux/of_platform.h>
+#include <linux/platform_device.h>
#include <linux/atomic.h>
#include <asm/errno.h>
{
struct pci_controller *phb;
- phb = zalloc_maybe_bootmem(sizeof(struct pci_controller), GFP_KERNEL);
+ phb = kzalloc(sizeof(struct pci_controller), GFP_KERNEL);
if (phb == NULL)
return NULL;
}
EXPORT_SYMBOL_GPL(release_pmc_hardware);
-#ifdef CONFIG_PPC64
+#ifdef CONFIG_PPC_BOOK3S_64
void power4_enable_pmcs(void)
{
unsigned long hid0;
return membuf_zero(&to, (ELF_NGREG - PT_REGS_COUNT) * sizeof(u32));
}
-int gpr32_set_common(struct task_struct *target,
- const struct user_regset *regset,
- unsigned int pos, unsigned int count,
- const void *kbuf, const void __user *ubuf,
- unsigned long *regs)
+static int gpr32_set_common_kernel(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, unsigned long *regs)
{
const compat_ulong_t *k = kbuf;
+
+ pos /= sizeof(compat_ulong_t);
+ count /= sizeof(compat_ulong_t);
+
+ for (; count > 0 && pos < PT_MSR; --count)
+ regs[pos++] = *k++;
+
+ if (count > 0 && pos == PT_MSR) {
+ set_user_msr(target, *k++);
+ ++pos;
+ --count;
+ }
+
+ for (; count > 0 && pos <= PT_MAX_PUT_REG; --count)
+ regs[pos++] = *k++;
+ for (; count > 0 && pos < PT_TRAP; --count, ++pos)
+ ++k;
+
+ if (count > 0 && pos == PT_TRAP) {
+ set_user_trap(target, *k++);
+ ++pos;
+ --count;
+ }
+
+ kbuf = k;
+ pos *= sizeof(compat_ulong_t);
+ count *= sizeof(compat_ulong_t);
+ user_regset_copyin_ignore(&pos, &count, &kbuf, NULL,
+ (PT_TRAP + 1) * sizeof(compat_ulong_t), -1);
+ return 0;
+}
+
+static int gpr32_set_common_user(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void __user *ubuf, unsigned long *regs)
+{
const compat_ulong_t __user *u = ubuf;
+ const void *kbuf = NULL;
compat_ulong_t reg;
- if (!kbuf && !user_read_access_begin(u, count))
+ if (!user_read_access_begin(u, count))
return -EFAULT;
pos /= sizeof(reg);
count /= sizeof(reg);
- if (kbuf)
- for (; count > 0 && pos < PT_MSR; --count)
- regs[pos++] = *k++;
- else
- for (; count > 0 && pos < PT_MSR; --count) {
- unsafe_get_user(reg, u++, Efault);
- regs[pos++] = reg;
- }
-
+ for (; count > 0 && pos < PT_MSR; --count) {
+ unsafe_get_user(reg, u++, Efault);
+ regs[pos++] = reg;
+ }
if (count > 0 && pos == PT_MSR) {
- if (kbuf)
- reg = *k++;
- else
- unsafe_get_user(reg, u++, Efault);
+ unsafe_get_user(reg, u++, Efault);
set_user_msr(target, reg);
++pos;
--count;
}
- if (kbuf) {
- for (; count > 0 && pos <= PT_MAX_PUT_REG; --count)
- regs[pos++] = *k++;
- for (; count > 0 && pos < PT_TRAP; --count, ++pos)
- ++k;
- } else {
- for (; count > 0 && pos <= PT_MAX_PUT_REG; --count) {
- unsafe_get_user(reg, u++, Efault);
- regs[pos++] = reg;
- }
- for (; count > 0 && pos < PT_TRAP; --count, ++pos)
- unsafe_get_user(reg, u++, Efault);
+ for (; count > 0 && pos <= PT_MAX_PUT_REG; --count) {
+ unsafe_get_user(reg, u++, Efault);
+ regs[pos++] = reg;
}
+ for (; count > 0 && pos < PT_TRAP; --count, ++pos)
+ unsafe_get_user(reg, u++, Efault);
if (count > 0 && pos == PT_TRAP) {
- if (kbuf)
- reg = *k++;
- else
- unsafe_get_user(reg, u++, Efault);
+ unsafe_get_user(reg, u++, Efault);
set_user_trap(target, reg);
++pos;
--count;
}
- if (!kbuf)
- user_read_access_end();
+ user_read_access_end();
- kbuf = k;
ubuf = u;
pos *= sizeof(reg);
count *= sizeof(reg);
return -EFAULT;
}
+int gpr32_set_common(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf,
+ unsigned long *regs)
+{
+ if (kbuf)
+ return gpr32_set_common_kernel(target, regset, pos, count, kbuf, regs);
+ else
+ return gpr32_set_common_user(target, regset, pos, count, ubuf, regs);
+}
+
static int gpr32_get(struct task_struct *target,
const struct user_regset *regset,
struct membuf to)
}
EXPORT_SYMBOL_GPL(rtas_busy_delay);
-static int rtas_error_rc(int rtas_rc)
+int rtas_error_rc(int rtas_rc)
{
int rc;
switch (rtas_rc) {
- case -1: /* Hardware Error */
- rc = -EIO;
- break;
- case -3: /* Bad indicator/domain/etc */
- rc = -EINVAL;
- break;
- case -9000: /* Isolation error */
- rc = -EFAULT;
- break;
- case -9001: /* Outstanding TCE/PTE */
- rc = -EEXIST;
- break;
- case -9002: /* No usable slot */
- rc = -ENODEV;
- break;
- default:
- pr_err("%s: unexpected error %d\n", __func__, rtas_rc);
- rc = -ERANGE;
- break;
+ case RTAS_HARDWARE_ERROR: /* Hardware Error */
+ rc = -EIO;
+ break;
+ case RTAS_INVALID_PARAMETER: /* Bad indicator/domain/etc */
+ rc = -EINVAL;
+ break;
+ case -9000: /* Isolation error */
+ rc = -EFAULT;
+ break;
+ case -9001: /* Outstanding TCE/PTE */
+ rc = -EEXIST;
+ break;
+ case -9002: /* No usable slot */
+ rc = -ENODEV;
+ break;
+ default:
+ pr_err("%s: unexpected error %d\n", __func__, rtas_rc);
+ rc = -ERANGE;
+ break;
}
return rc;
}
+EXPORT_SYMBOL_GPL(rtas_error_rc);
int rtas_get_power_level(int powerdomain, int *level)
{
void rtas_os_term(char *str)
{
s32 token = rtas_function_token(RTAS_FN_IBM_OS_TERM);
+ static struct rtas_args args;
int status;
/*
* schedules.
*/
do {
- status = rtas_call(token, 1, 1, NULL, __pa(rtas_os_term_buf));
+ rtas_call_unlocked(&args, token, 1, 1, NULL, __pa(rtas_os_term_buf));
+ status = be32_to_cpu(args.rets[0]);
} while (rtas_busy_delay_time(status));
if (status != 0)
#include <linux/serial_8250.h>
#include <linux/percpu.h>
#include <linux/memblock.h>
-#include <linux/of_irq.h>
+#include <linux/of.h>
#include <linux/of_fdt.h>
-#include <linux/of_platform.h>
+#include <linux/of_irq.h>
#include <linux/hugetlb.h>
#include <linux/pgtable.h>
#include <asm/io.h>
klp_init_thread_info(&init_task);
setup_initial_init_mm(_stext, _etext, _edata, _end);
-
+ /* sched_init() does the mmgrab(&init_mm) for the primary CPU */
+ VM_WARN_ON(cpumask_test_cpu(smp_processor_id(), mm_cpumask(&init_mm)));
+ cpumask_set_cpu(smp_processor_id(), mm_cpumask(&init_mm));
+ inc_mm_active_cpus(&init_mm);
mm_iommu_init(&init_mm);
+
irqstack_early_init();
exc_lvl_early_init();
emergency_stack_init();
#include <asm/smp.h>
#include <asm/time.h>
#include <asm/machdep.h>
+#include <asm/mmu_context.h>
#include <asm/cputhreads.h>
#include <asm/cputable.h>
#include <asm/mpic.h>
void __init smp_prepare_cpus(unsigned int max_cpus)
{
- unsigned int cpu;
+ unsigned int cpu, num_threads;
DBG("smp_prepare_cpus\n");
if (smp_ops && smp_ops->probe)
smp_ops->probe();
+
+ // Initalise the generic SMT topology support
+ num_threads = 1;
+ if (smt_enabled_at_boot)
+ num_threads = smt_enabled_at_boot;
+ cpu_smt_set_num_threads(num_threads, threads_per_core);
}
void smp_prepare_boot_cpu(void)
mmgrab_lazy_tlb(&init_mm);
current->active_mm = &init_mm;
+ VM_WARN_ON(cpumask_test_cpu(smp_processor_id(), mm_cpumask(&init_mm)));
+ cpumask_set_cpu(cpu, mm_cpumask(&init_mm));
+ inc_mm_active_cpus(&init_mm);
smp_store_cpu_info(cpu);
set_dec(tb_ticks_per_jiffy);
void __cpu_die(unsigned int cpu)
{
+ /*
+ * This could perhaps be a generic call in idlea_task_dead(), but
+ * that requires testing from all archs, so first put it here to
+ */
+ VM_WARN_ON_ONCE(!cpumask_test_cpu(cpu, mm_cpumask(&init_mm)));
+ dec_mm_active_cpus(&init_mm);
+ cpumask_clear_cpu(cpu, mm_cpumask(&init_mm));
+
if (smp_ops->cpu_die)
smp_ops->cpu_die(cpu);
}
iamr = mfspr(SPRN_IAMR);
regs->amr = amr;
regs->iamr = iamr;
- if (mmu_has_feature(MMU_FTR_BOOK3S_KUAP)) {
+ if (mmu_has_feature(MMU_FTR_KUAP)) {
mtspr(SPRN_AMR, AMR_KUAP_BLOCKED);
flush_needed = true;
}
* Copyright 2012 Matt Evans & Michael Neuling, IBM Corporation.
*/
+#include <linux/export.h>
#include <asm/asm-offsets.h>
#include <asm/ppc_asm.h>
#include <asm/ppc-opcode.h>
#include <asm/ptrace.h>
#include <asm/reg.h>
#include <asm/bug.h>
-#include <asm/export.h>
#include <asm/feature-fixups.h>
#ifdef CONFIG_VSX
ifdef CONFIG_FUNCTION_TRACER
# do not trace tracer code
CFLAGS_REMOVE_ftrace.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_ftrace_64_pg.o = $(CC_FLAGS_FTRACE)
endif
-obj32-$(CONFIG_FUNCTION_TRACER) += ftrace_mprofile.o
+obj32-$(CONFIG_FUNCTION_TRACER) += ftrace.o ftrace_entry.o
ifdef CONFIG_MPROFILE_KERNEL
-obj64-$(CONFIG_FUNCTION_TRACER) += ftrace_mprofile.o
+obj64-$(CONFIG_FUNCTION_TRACER) += ftrace.o ftrace_entry.o
else
-obj64-$(CONFIG_FUNCTION_TRACER) += ftrace_64_pg.o
+obj64-$(CONFIG_FUNCTION_TRACER) += ftrace_64_pg.o ftrace_64_pg_entry.o
endif
-obj-$(CONFIG_FUNCTION_TRACER) += ftrace_low.o ftrace.o
obj-$(CONFIG_TRACING) += trace_clock.o
obj-$(CONFIG_PPC64) += $(obj64-y)
KCOV_INSTRUMENT_ftrace.o := n
KCSAN_SANITIZE_ftrace.o := n
UBSAN_SANITIZE_ftrace.o := n
+GCOV_PROFILE_ftrace_64_pg.o := n
+KCOV_INSTRUMENT_ftrace_64_pg.o := n
+KCSAN_SANITIZE_ftrace_64_pg.o := n
+UBSAN_SANITIZE_ftrace_64_pg.o := n
#include <asm/syscall.h>
#include <asm/inst.h>
-/*
- * We generally only have a single long_branch tramp and at most 2 or 3 plt
- * tramps generated. But, we don't use the plt tramps currently. We also allot
- * 2 tramps after .text and .init.text. So, we only end up with around 3 usable
- * tramps in total. Set aside 8 just to be sure.
- */
-#define NUM_FTRACE_TRAMPS 8
+#define NUM_FTRACE_TRAMPS 2
static unsigned long ftrace_tramps[NUM_FTRACE_TRAMPS];
-static ppc_inst_t
-ftrace_call_replace(unsigned long ip, unsigned long addr, int link)
+static ppc_inst_t ftrace_create_branch_inst(unsigned long ip, unsigned long addr, int link)
{
ppc_inst_t op;
- addr = ppc_function_entry((void *)addr);
-
- /* if (link) set op to 'bl' else 'b' */
+ WARN_ON(!is_offset_in_branch_range(addr - ip));
create_branch(&op, (u32 *)ip, addr, link ? BRANCH_SET_LINK : 0);
return op;
}
-static inline int
-ftrace_modify_code(unsigned long ip, ppc_inst_t old, ppc_inst_t new)
+static inline int ftrace_read_inst(unsigned long ip, ppc_inst_t *op)
{
- ppc_inst_t replaced;
-
- /*
- * Note:
- * We are paranoid about modifying text, as if a bug was to happen, it
- * could cause us to read or write to someplace that could cause harm.
- * Carefully read and modify the code with probe_kernel_*(), and make
- * sure what we read is what we expected it to be before modifying it.
- */
-
- /* read the text we want to modify */
- if (copy_inst_from_kernel_nofault(&replaced, (void *)ip))
+ if (copy_inst_from_kernel_nofault(op, (void *)ip)) {
+ pr_err("0x%lx: fetching instruction failed\n", ip);
return -EFAULT;
-
- /* Make sure it is what we expect it to be */
- if (!ppc_inst_equal(replaced, old)) {
- pr_err("%p: replaced (%08lx) != old (%08lx)", (void *)ip,
- ppc_inst_as_ulong(replaced), ppc_inst_as_ulong(old));
- return -EINVAL;
}
- /* replace the text with the new text */
- return patch_instruction((u32 *)ip, new);
+ return 0;
}
-/*
- * Helper functions that are the same for both PPC64 and PPC32.
- */
-static int test_24bit_addr(unsigned long ip, unsigned long addr)
+static inline int ftrace_validate_inst(unsigned long ip, ppc_inst_t inst)
{
- addr = ppc_function_entry((void *)addr);
-
- return is_offset_in_branch_range(addr - ip);
-}
+ ppc_inst_t op;
+ int ret;
-static int is_bl_op(ppc_inst_t op)
-{
- return (ppc_inst_val(op) & ~PPC_LI_MASK) == PPC_RAW_BL(0);
-}
+ ret = ftrace_read_inst(ip, &op);
+ if (!ret && !ppc_inst_equal(op, inst)) {
+ pr_err("0x%lx: expected (%08lx) != found (%08lx)\n",
+ ip, ppc_inst_as_ulong(inst), ppc_inst_as_ulong(op));
+ ret = -EINVAL;
+ }
-static int is_b_op(ppc_inst_t op)
-{
- return (ppc_inst_val(op) & ~PPC_LI_MASK) == PPC_RAW_BRANCH(0);
+ return ret;
}
-static unsigned long find_bl_target(unsigned long ip, ppc_inst_t op)
+static inline int ftrace_modify_code(unsigned long ip, ppc_inst_t old, ppc_inst_t new)
{
- int offset;
+ int ret = ftrace_validate_inst(ip, old);
- offset = PPC_LI(ppc_inst_val(op));
- /* make it signed */
- if (offset & 0x02000000)
- offset |= 0xfe000000;
+ if (!ret)
+ ret = patch_instruction((u32 *)ip, new);
- return ip + (long)offset;
+ return ret;
}
-#ifdef CONFIG_MODULES
-static int
-__ftrace_make_nop(struct module *mod,
- struct dyn_ftrace *rec, unsigned long addr)
-{
- unsigned long entry, ptr, tramp;
- unsigned long ip = rec->ip;
- ppc_inst_t op, pop;
-
- /* read where this goes */
- if (copy_inst_from_kernel_nofault(&op, (void *)ip)) {
- pr_err("Fetching opcode failed.\n");
- return -EFAULT;
- }
-
- /* Make sure that this is still a 24bit jump */
- if (!is_bl_op(op)) {
- pr_err("Not expected bl: opcode is %08lx\n", ppc_inst_as_ulong(op));
- return -EINVAL;
- }
-
- /* lets find where the pointer goes */
- tramp = find_bl_target(ip, op);
-
- pr_devel("ip:%lx jumps to %lx", ip, tramp);
-
- if (module_trampoline_target(mod, tramp, &ptr)) {
- pr_err("Failed to get trampoline target\n");
- return -EFAULT;
- }
-
- pr_devel("trampoline target %lx", ptr);
-
- entry = ppc_global_function_entry((void *)addr);
- /* This should match what was called */
- if (ptr != entry) {
- pr_err("addr %lx does not match expected %lx\n", ptr, entry);
- return -EINVAL;
- }
-
- if (IS_ENABLED(CONFIG_MPROFILE_KERNEL)) {
- if (copy_inst_from_kernel_nofault(&op, (void *)(ip - 4))) {
- pr_err("Fetching instruction at %lx failed.\n", ip - 4);
- return -EFAULT;
- }
-
- /* We expect either a mflr r0, or a std r0, LRSAVE(r1) */
- if (!ppc_inst_equal(op, ppc_inst(PPC_RAW_MFLR(_R0))) &&
- !ppc_inst_equal(op, ppc_inst(PPC_INST_STD_LR))) {
- pr_err("Unexpected instruction %08lx around bl _mcount\n",
- ppc_inst_as_ulong(op));
- return -EINVAL;
- }
- } else if (IS_ENABLED(CONFIG_PPC64)) {
- /*
- * Check what is in the next instruction. We can see ld r2,40(r1), but
- * on first pass after boot we will see mflr r0.
- */
- if (copy_inst_from_kernel_nofault(&op, (void *)(ip + 4))) {
- pr_err("Fetching op failed.\n");
- return -EFAULT;
- }
-
- if (!ppc_inst_equal(op, ppc_inst(PPC_INST_LD_TOC))) {
- pr_err("Expected %08lx found %08lx\n", PPC_INST_LD_TOC,
- ppc_inst_as_ulong(op));
- return -EINVAL;
- }
- }
-
- /*
- * When using -mprofile-kernel or PPC32 there is no load to jump over.
- *
- * Otherwise our original call site looks like:
- *
- * bl <tramp>
- * ld r2,XX(r1)
- *
- * Milton Miller pointed out that we can not simply nop the branch.
- * If a task was preempted when calling a trace function, the nops
- * will remove the way to restore the TOC in r2 and the r2 TOC will
- * get corrupted.
- *
- * Use a b +8 to jump over the load.
- * XXX: could make PCREL depend on MPROFILE_KERNEL
- * XXX: check PCREL && MPROFILE_KERNEL calling sequence
- */
- if (IS_ENABLED(CONFIG_MPROFILE_KERNEL) || IS_ENABLED(CONFIG_PPC32))
- pop = ppc_inst(PPC_RAW_NOP());
- else
- pop = ppc_inst(PPC_RAW_BRANCH(8)); /* b +8 */
-
- if (patch_instruction((u32 *)ip, pop)) {
- pr_err("Patching NOP failed.\n");
- return -EPERM;
- }
-
- return 0;
-}
-#else
-static int __ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, unsigned long addr)
+static int is_bl_op(ppc_inst_t op)
{
- return 0;
+ return (ppc_inst_val(op) & ~PPC_LI_MASK) == PPC_RAW_BL(0);
}
-#endif /* CONFIG_MODULES */
static unsigned long find_ftrace_tramp(unsigned long ip)
{
int i;
- /*
- * We have the compiler generated long_branch tramps at the end
- * and we prefer those
- */
- for (i = NUM_FTRACE_TRAMPS - 1; i >= 0; i--)
+ for (i = 0; i < NUM_FTRACE_TRAMPS; i++)
if (!ftrace_tramps[i])
continue;
else if (is_offset_in_branch_range(ftrace_tramps[i] - ip))
return 0;
}
-static int add_ftrace_tramp(unsigned long tramp)
+static int ftrace_get_call_inst(struct dyn_ftrace *rec, unsigned long addr, ppc_inst_t *call_inst)
{
- int i;
+ unsigned long ip = rec->ip;
+ unsigned long stub;
- for (i = 0; i < NUM_FTRACE_TRAMPS; i++)
- if (!ftrace_tramps[i]) {
- ftrace_tramps[i] = tramp;
- return 0;
+ if (is_offset_in_branch_range(addr - ip)) {
+ /* Within range */
+ stub = addr;
+#ifdef CONFIG_MODULES
+ } else if (rec->arch.mod) {
+ /* Module code would be going to one of the module stubs */
+ stub = (addr == (unsigned long)ftrace_caller ? rec->arch.mod->arch.tramp :
+ rec->arch.mod->arch.tramp_regs);
+#endif
+ } else if (core_kernel_text(ip)) {
+ /* We would be branching to one of our ftrace stubs */
+ stub = find_ftrace_tramp(ip);
+ if (!stub) {
+ pr_err("0x%lx: No ftrace stubs reachable\n", ip);
+ return -EINVAL;
}
-
- return -1;
-}
-
-/*
- * If this is a compiler generated long_branch trampoline (essentially, a
- * trampoline that has a branch to _mcount()), we re-write the branch to
- * instead go to ftrace_[regs_]caller() and note down the location of this
- * trampoline.
- */
-static int setup_mcount_compiler_tramp(unsigned long tramp)
-{
- int i;
- ppc_inst_t op;
- unsigned long ptr;
-
- /* Is this a known long jump tramp? */
- for (i = 0; i < NUM_FTRACE_TRAMPS; i++)
- if (ftrace_tramps[i] == tramp)
- return 0;
-
- /* New trampoline -- read where this goes */
- if (copy_inst_from_kernel_nofault(&op, (void *)tramp)) {
- pr_debug("Fetching opcode failed.\n");
- return -1;
- }
-
- /* Is this a 24 bit branch? */
- if (!is_b_op(op)) {
- pr_debug("Trampoline is not a long branch tramp.\n");
- return -1;
- }
-
- /* lets find where the pointer goes */
- ptr = find_bl_target(tramp, op);
-
- if (ptr != ppc_global_function_entry((void *)_mcount)) {
- pr_debug("Trampoline target %p is not _mcount\n", (void *)ptr);
- return -1;
- }
-
- /* Let's re-write the tramp to go to ftrace_[regs_]caller */
- if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
- ptr = ppc_global_function_entry((void *)ftrace_regs_caller);
- else
- ptr = ppc_global_function_entry((void *)ftrace_caller);
-
- if (patch_branch((u32 *)tramp, ptr, 0)) {
- pr_debug("REL24 out of range!\n");
- return -1;
- }
-
- if (add_ftrace_tramp(tramp)) {
- pr_debug("No tramp locations left\n");
- return -1;
+ } else {
+ return -EINVAL;
}
+ *call_inst = ftrace_create_branch_inst(ip, stub, 1);
return 0;
}
-static int __ftrace_make_nop_kernel(struct dyn_ftrace *rec, unsigned long addr)
+#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
+int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr, unsigned long addr)
{
- unsigned long tramp, ip = rec->ip;
- ppc_inst_t op;
-
- /* Read where this goes */
- if (copy_inst_from_kernel_nofault(&op, (void *)ip)) {
- pr_err("Fetching opcode failed.\n");
- return -EFAULT;
- }
-
- /* Make sure that this is still a 24bit jump */
- if (!is_bl_op(op)) {
- pr_err("Not expected bl: opcode is %08lx\n", ppc_inst_as_ulong(op));
- return -EINVAL;
- }
-
- /* Let's find where the pointer goes */
- tramp = find_bl_target(ip, op);
-
- pr_devel("ip:%lx jumps to %lx", ip, tramp);
-
- if (setup_mcount_compiler_tramp(tramp)) {
- /* Are other trampolines reachable? */
- if (!find_ftrace_tramp(ip)) {
- pr_err("No ftrace trampolines reachable from %ps\n",
- (void *)ip);
- return -EINVAL;
- }
- }
-
- if (patch_instruction((u32 *)ip, ppc_inst(PPC_RAW_NOP()))) {
- pr_err("Patching NOP failed.\n");
- return -EPERM;
- }
-
- return 0;
+ /* This should never be called since we override ftrace_replace_code() */
+ WARN_ON(1);
+ return -EINVAL;
}
+#endif
-int ftrace_make_nop(struct module *mod,
- struct dyn_ftrace *rec, unsigned long addr)
+int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
{
- unsigned long ip = rec->ip;
ppc_inst_t old, new;
+ int ret;
- /*
- * If the calling address is more that 24 bits away,
- * then we had to use a trampoline to make the call.
- * Otherwise just update the call site.
- */
- if (test_24bit_addr(ip, addr)) {
- /* within range */
- old = ftrace_call_replace(ip, addr, 1);
- new = ppc_inst(PPC_RAW_NOP());
- return ftrace_modify_code(ip, old, new);
- } else if (core_kernel_text(ip)) {
- return __ftrace_make_nop_kernel(rec, addr);
- } else if (!IS_ENABLED(CONFIG_MODULES)) {
+ /* This can only ever be called during module load */
+ if (WARN_ON(!IS_ENABLED(CONFIG_MODULES) || core_kernel_text(rec->ip)))
return -EINVAL;
- }
- /*
- * Out of range jumps are called from modules.
- * We should either already have a pointer to the module
- * or it has been passed in.
- */
- if (!rec->arch.mod) {
- if (!mod) {
- pr_err("No module loaded addr=%lx\n", addr);
- return -EFAULT;
- }
- rec->arch.mod = mod;
- } else if (mod) {
- if (mod != rec->arch.mod) {
- pr_err("Record mod %p not equal to passed in mod %p\n",
- rec->arch.mod, mod);
- return -EINVAL;
- }
- /* nothing to do if mod == rec->arch.mod */
- } else
- mod = rec->arch.mod;
+ old = ppc_inst(PPC_RAW_NOP());
+ ret = ftrace_get_call_inst(rec, addr, &new);
+ if (ret)
+ return ret;
- return __ftrace_make_nop(mod, rec, addr);
+ return ftrace_modify_code(rec->ip, old, new);
}
-#ifdef CONFIG_MODULES
-/*
- * Examine the existing instructions for __ftrace_make_call.
- * They should effectively be a NOP, and follow formal constraints,
- * depending on the ABI. Return false if they don't.
- */
-static bool expected_nop_sequence(void *ip, ppc_inst_t op0, ppc_inst_t op1)
-{
- if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
- return ppc_inst_equal(op0, ppc_inst(PPC_RAW_NOP()));
- else
- return ppc_inst_equal(op0, ppc_inst(PPC_RAW_BRANCH(8))) &&
- ppc_inst_equal(op1, ppc_inst(PPC_INST_LD_TOC));
-}
-
-static int
-__ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
-{
- ppc_inst_t op[2];
- void *ip = (void *)rec->ip;
- unsigned long entry, ptr, tramp;
- struct module *mod = rec->arch.mod;
-
- /* read where this goes */
- if (copy_inst_from_kernel_nofault(op, ip))
- return -EFAULT;
-
- if (!IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS) &&
- copy_inst_from_kernel_nofault(op + 1, ip + 4))
- return -EFAULT;
-
- if (!expected_nop_sequence(ip, op[0], op[1])) {
- pr_err("Unexpected call sequence at %p: %08lx %08lx\n", ip,
- ppc_inst_as_ulong(op[0]), ppc_inst_as_ulong(op[1]));
- return -EINVAL;
- }
-
- /* If we never set up ftrace trampoline(s), then bail */
- if (!mod->arch.tramp ||
- (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS) && !mod->arch.tramp_regs)) {
- pr_err("No ftrace trampoline\n");
- return -EINVAL;
- }
-
- if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS) && rec->flags & FTRACE_FL_REGS)
- tramp = mod->arch.tramp_regs;
- else
- tramp = mod->arch.tramp;
-
- if (module_trampoline_target(mod, tramp, &ptr)) {
- pr_err("Failed to get trampoline target\n");
- return -EFAULT;
- }
-
- pr_devel("trampoline target %lx", ptr);
-
- entry = ppc_global_function_entry((void *)addr);
- /* This should match what was called */
- if (ptr != entry) {
- pr_err("addr %lx does not match expected %lx\n", ptr, entry);
- return -EINVAL;
- }
-
- if (patch_branch(ip, tramp, BRANCH_SET_LINK)) {
- pr_err("REL24 out of range!\n");
- return -EINVAL;
- }
-
- return 0;
-}
-#else
-static int __ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
+int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, unsigned long addr)
{
- return 0;
+ /*
+ * This should never be called since we override ftrace_replace_code(),
+ * as well as ftrace_init_nop()
+ */
+ WARN_ON(1);
+ return -EINVAL;
}
-#endif /* CONFIG_MODULES */
-static int __ftrace_make_call_kernel(struct dyn_ftrace *rec, unsigned long addr)
+void ftrace_replace_code(int enable)
{
- ppc_inst_t op;
- void *ip = (void *)rec->ip;
- unsigned long tramp, entry, ptr;
+ ppc_inst_t old, new, call_inst, new_call_inst;
+ ppc_inst_t nop_inst = ppc_inst(PPC_RAW_NOP());
+ unsigned long ip, new_addr, addr;
+ struct ftrace_rec_iter *iter;
+ struct dyn_ftrace *rec;
+ int ret = 0, update;
- /* Make sure we're being asked to patch branch to a known ftrace addr */
- entry = ppc_global_function_entry((void *)ftrace_caller);
- ptr = ppc_global_function_entry((void *)addr);
+ for_ftrace_rec_iter(iter) {
+ rec = ftrace_rec_iter_record(iter);
+ ip = rec->ip;
- if (ptr != entry && IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
- entry = ppc_global_function_entry((void *)ftrace_regs_caller);
-
- if (ptr != entry) {
- pr_err("Unknown ftrace addr to patch: %ps\n", (void *)ptr);
- return -EINVAL;
- }
-
- /* Make sure we have a nop */
- if (copy_inst_from_kernel_nofault(&op, ip)) {
- pr_err("Unable to read ftrace location %p\n", ip);
- return -EFAULT;
- }
+ if (rec->flags & FTRACE_FL_DISABLED && !(rec->flags & FTRACE_FL_ENABLED))
+ continue;
- if (!ppc_inst_equal(op, ppc_inst(PPC_RAW_NOP()))) {
- pr_err("Unexpected call sequence at %p: %08lx\n",
- ip, ppc_inst_as_ulong(op));
- return -EINVAL;
- }
+ addr = ftrace_get_addr_curr(rec);
+ new_addr = ftrace_get_addr_new(rec);
+ update = ftrace_update_record(rec, enable);
- tramp = find_ftrace_tramp((unsigned long)ip);
- if (!tramp) {
- pr_err("No ftrace trampolines reachable from %ps\n", ip);
- return -EINVAL;
- }
+ switch (update) {
+ case FTRACE_UPDATE_IGNORE:
+ default:
+ continue;
+ case FTRACE_UPDATE_MODIFY_CALL:
+ ret = ftrace_get_call_inst(rec, new_addr, &new_call_inst);
+ ret |= ftrace_get_call_inst(rec, addr, &call_inst);
+ old = call_inst;
+ new = new_call_inst;
+ break;
+ case FTRACE_UPDATE_MAKE_NOP:
+ ret = ftrace_get_call_inst(rec, addr, &call_inst);
+ old = call_inst;
+ new = nop_inst;
+ break;
+ case FTRACE_UPDATE_MAKE_CALL:
+ ret = ftrace_get_call_inst(rec, new_addr, &call_inst);
+ old = nop_inst;
+ new = call_inst;
+ break;
+ }
- if (patch_branch(ip, tramp, BRANCH_SET_LINK)) {
- pr_err("Error patching branch to ftrace tramp!\n");
- return -EINVAL;
+ if (!ret)
+ ret = ftrace_modify_code(ip, old, new);
+ if (ret)
+ goto out;
}
- return 0;
+out:
+ if (ret)
+ ftrace_bug(ret, rec);
+ return;
}
-int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
+int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec)
{
- unsigned long ip = rec->ip;
+ unsigned long addr, ip = rec->ip;
ppc_inst_t old, new;
-
- /*
- * If the calling address is more that 24 bits away,
- * then we had to use a trampoline to make the call.
- * Otherwise just update the call site.
- */
- if (test_24bit_addr(ip, addr)) {
- /* within range */
- old = ppc_inst(PPC_RAW_NOP());
- new = ftrace_call_replace(ip, addr, 1);
- return ftrace_modify_code(ip, old, new);
- } else if (core_kernel_text(ip)) {
- return __ftrace_make_call_kernel(rec, addr);
- } else if (!IS_ENABLED(CONFIG_MODULES)) {
- /* We should not get here without modules */
- return -EINVAL;
- }
-
- /*
- * Out of range jumps are called from modules.
- * Being that we are converting from nop, it had better
- * already have a module defined.
- */
- if (!rec->arch.mod) {
- pr_err("No module loaded\n");
- return -EINVAL;
- }
-
- return __ftrace_make_call(rec, addr);
-}
-
-#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
-#ifdef CONFIG_MODULES
-static int
-__ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
- unsigned long addr)
-{
- ppc_inst_t op;
- unsigned long ip = rec->ip;
- unsigned long entry, ptr, tramp;
- struct module *mod = rec->arch.mod;
-
- /* If we never set up ftrace trampolines, then bail */
- if (!mod->arch.tramp || !mod->arch.tramp_regs) {
- pr_err("No ftrace trampoline\n");
- return -EINVAL;
- }
-
- /* read where this goes */
- if (copy_inst_from_kernel_nofault(&op, (void *)ip)) {
- pr_err("Fetching opcode failed.\n");
- return -EFAULT;
- }
-
- /* Make sure that this is still a 24bit jump */
- if (!is_bl_op(op)) {
- pr_err("Not expected bl: opcode is %08lx\n", ppc_inst_as_ulong(op));
+ int ret = 0;
+
+ /* Verify instructions surrounding the ftrace location */
+ if (IS_ENABLED(CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY)) {
+ /* Expect nops */
+ ret = ftrace_validate_inst(ip - 4, ppc_inst(PPC_RAW_NOP()));
+ if (!ret)
+ ret = ftrace_validate_inst(ip, ppc_inst(PPC_RAW_NOP()));
+ } else if (IS_ENABLED(CONFIG_PPC32)) {
+ /* Expected sequence: 'mflr r0', 'stw r0,4(r1)', 'bl _mcount' */
+ ret = ftrace_validate_inst(ip - 8, ppc_inst(PPC_RAW_MFLR(_R0)));
+ if (!ret)
+ ret = ftrace_validate_inst(ip - 4, ppc_inst(PPC_RAW_STW(_R0, _R1, 4)));
+ } else if (IS_ENABLED(CONFIG_MPROFILE_KERNEL)) {
+ /* Expected sequence: 'mflr r0', ['std r0,16(r1)'], 'bl _mcount' */
+ ret = ftrace_read_inst(ip - 4, &old);
+ if (!ret && !ppc_inst_equal(old, ppc_inst(PPC_RAW_MFLR(_R0)))) {
+ ret = ftrace_validate_inst(ip - 8, ppc_inst(PPC_RAW_MFLR(_R0)));
+ ret |= ftrace_validate_inst(ip - 4, ppc_inst(PPC_RAW_STD(_R0, _R1, 16)));
+ }
+ } else {
return -EINVAL;
}
- /* lets find where the pointer goes */
- tramp = find_bl_target(ip, op);
- entry = ppc_global_function_entry((void *)old_addr);
-
- pr_devel("ip:%lx jumps to %lx", ip, tramp);
+ if (ret)
+ return ret;
- if (tramp != entry) {
- /* old_addr is not within range, so we must have used a trampoline */
- if (module_trampoline_target(mod, tramp, &ptr)) {
- pr_err("Failed to get trampoline target\n");
+ if (!core_kernel_text(ip)) {
+ if (!mod) {
+ pr_err("0x%lx: No module provided for non-kernel address\n", ip);
return -EFAULT;
}
-
- pr_devel("trampoline target %lx", ptr);
-
- /* This should match what was called */
- if (ptr != entry) {
- pr_err("addr %lx does not match expected %lx\n", ptr, entry);
- return -EINVAL;
- }
- }
-
- /* The new target may be within range */
- if (test_24bit_addr(ip, addr)) {
- /* within range */
- if (patch_branch((u32 *)ip, addr, BRANCH_SET_LINK)) {
- pr_err("REL24 out of range!\n");
- return -EINVAL;
- }
-
- return 0;
- }
-
- if (rec->flags & FTRACE_FL_REGS)
- tramp = mod->arch.tramp_regs;
- else
- tramp = mod->arch.tramp;
-
- if (module_trampoline_target(mod, tramp, &ptr)) {
- pr_err("Failed to get trampoline target\n");
- return -EFAULT;
- }
-
- pr_devel("trampoline target %lx", ptr);
-
- entry = ppc_global_function_entry((void *)addr);
- /* This should match what was called */
- if (ptr != entry) {
- pr_err("addr %lx does not match expected %lx\n", ptr, entry);
- return -EINVAL;
- }
-
- if (patch_branch((u32 *)ip, tramp, BRANCH_SET_LINK)) {
- pr_err("REL24 out of range!\n");
- return -EINVAL;
+ rec->arch.mod = mod;
}
- return 0;
-}
-#else
-static int __ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr, unsigned long addr)
-{
- return 0;
-}
-#endif
-
-int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
- unsigned long addr)
-{
- unsigned long ip = rec->ip;
- ppc_inst_t old, new;
-
- /*
- * If the calling address is more that 24 bits away,
- * then we had to use a trampoline to make the call.
- * Otherwise just update the call site.
- */
- if (test_24bit_addr(ip, addr) && test_24bit_addr(ip, old_addr)) {
- /* within range */
- old = ftrace_call_replace(ip, old_addr, 1);
- new = ftrace_call_replace(ip, addr, 1);
- return ftrace_modify_code(ip, old, new);
- } else if (core_kernel_text(ip)) {
+ /* Nop-out the ftrace location */
+ new = ppc_inst(PPC_RAW_NOP());
+ addr = MCOUNT_ADDR;
+ if (IS_ENABLED(CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY)) {
+ /* we instead patch-in the 'mflr r0' */
+ old = ppc_inst(PPC_RAW_NOP());
+ new = ppc_inst(PPC_RAW_MFLR(_R0));
+ ret = ftrace_modify_code(ip - 4, old, new);
+ } else if (is_offset_in_branch_range(addr - ip)) {
+ /* Within range */
+ old = ftrace_create_branch_inst(ip, addr, 1);
+ ret = ftrace_modify_code(ip, old, new);
+ } else if (core_kernel_text(ip) || (IS_ENABLED(CONFIG_MODULES) && mod)) {
/*
- * We always patch out of range locations to go to the regs
- * variant, so there is nothing to do here
+ * We would be branching to a linker-generated stub, or to the module _mcount
+ * stub. Let's just confirm we have a 'bl' here.
*/
- return 0;
- } else if (!IS_ENABLED(CONFIG_MODULES)) {
- /* We should not get here without modules */
- return -EINVAL;
- }
-
- /*
- * Out of range jumps are called from modules.
- */
- if (!rec->arch.mod) {
- pr_err("No module loaded\n");
+ ret = ftrace_read_inst(ip, &old);
+ if (ret)
+ return ret;
+ if (!is_bl_op(old)) {
+ pr_err("0x%lx: expected (bl) != found (%08lx)\n", ip, ppc_inst_as_ulong(old));
+ return -EINVAL;
+ }
+ ret = patch_instruction((u32 *)ip, new);
+ } else {
return -EINVAL;
}
- return __ftrace_modify_call(rec, old_addr, addr);
+ return ret;
}
-#endif
int ftrace_update_ftrace_func(ftrace_func_t func)
{
int ret;
old = ppc_inst_read((u32 *)&ftrace_call);
- new = ftrace_call_replace(ip, (unsigned long)func, 1);
+ new = ftrace_create_branch_inst(ip, ppc_function_entry(func), 1);
ret = ftrace_modify_code(ip, old, new);
/* Also update the regs callback function */
if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS) && !ret) {
ip = (unsigned long)(&ftrace_regs_call);
old = ppc_inst_read((u32 *)&ftrace_regs_call);
- new = ftrace_call_replace(ip, (unsigned long)func, 1);
+ new = ftrace_create_branch_inst(ip, ppc_function_entry(func), 1);
ret = ftrace_modify_code(ip, old, new);
}
ftrace_modify_all_code(command);
}
-#ifdef CONFIG_PPC64
-#define PACATOC offsetof(struct paca_struct, kernel_toc)
-
-extern unsigned int ftrace_tramp_text[], ftrace_tramp_init[];
-
void ftrace_free_init_tramp(void)
{
int i;
}
}
-int __init ftrace_dyn_arch_init(void)
+static void __init add_ftrace_tramp(unsigned long tramp)
{
int i;
+
+ for (i = 0; i < NUM_FTRACE_TRAMPS; i++)
+ if (!ftrace_tramps[i]) {
+ ftrace_tramps[i] = tramp;
+ return;
+ }
+}
+
+int __init ftrace_dyn_arch_init(void)
+{
unsigned int *tramp[] = { ftrace_tramp_text, ftrace_tramp_init };
-#ifdef CONFIG_PPC_KERNEL_PCREL
+ unsigned long addr = FTRACE_REGS_ADDR;
+ long reladdr;
+ int i;
u32 stub_insns[] = {
+#ifdef CONFIG_PPC_KERNEL_PCREL
/* pla r12,addr */
PPC_PREFIX_MLS | __PPC_PRFX_R(1),
PPC_INST_PADDI | ___PPC_RT(_R12),
PPC_RAW_MTCTR(_R12),
PPC_RAW_BCTR()
- };
-#else
- u32 stub_insns[] = {
- PPC_RAW_LD(_R12, _R13, PACATOC),
+#elif defined(CONFIG_PPC64)
+ PPC_RAW_LD(_R12, _R13, offsetof(struct paca_struct, kernel_toc)),
PPC_RAW_ADDIS(_R12, _R12, 0),
PPC_RAW_ADDI(_R12, _R12, 0),
PPC_RAW_MTCTR(_R12),
PPC_RAW_BCTR()
- };
+#else
+ PPC_RAW_LIS(_R12, 0),
+ PPC_RAW_ADDI(_R12, _R12, 0),
+ PPC_RAW_MTCTR(_R12),
+ PPC_RAW_BCTR()
#endif
-
- unsigned long addr;
- long reladdr;
-
- if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
- addr = ppc_global_function_entry((void *)ftrace_regs_caller);
- else
- addr = ppc_global_function_entry((void *)ftrace_caller);
+ };
if (IS_ENABLED(CONFIG_PPC_KERNEL_PCREL)) {
for (i = 0; i < 2; i++) {
tramp[i][1] |= IMM_L(reladdr);
add_ftrace_tramp((unsigned long)tramp[i]);
}
- } else {
+ } else if (IS_ENABLED(CONFIG_PPC64)) {
reladdr = addr - kernel_toc_addr();
- if (reladdr >= (long)SZ_2G || reladdr < -(long)SZ_2G) {
+ if (reladdr >= (long)SZ_2G || reladdr < -(long long)SZ_2G) {
pr_err("Address of %ps out of range of kernel_toc.\n",
(void *)addr);
return -1;
tramp[i][2] |= PPC_LO(reladdr);
add_ftrace_tramp((unsigned long)tramp[i]);
}
+ } else {
+ for (i = 0; i < 2; i++) {
+ memcpy(tramp[i], stub_insns, sizeof(stub_insns));
+ tramp[i][0] |= PPC_HA(addr);
+ tramp[i][1] |= PPC_LO(addr);
+ add_ftrace_tramp((unsigned long)tramp[i]);
+ }
}
return 0;
}
-#endif
#ifdef CONFIG_FUNCTION_GRAPH_TRACER
-
-extern void ftrace_graph_call(void);
-extern void ftrace_graph_stub(void);
-
-static int ftrace_modify_ftrace_graph_caller(bool enable)
-{
- unsigned long ip = (unsigned long)(&ftrace_graph_call);
- unsigned long addr = (unsigned long)(&ftrace_graph_caller);
- unsigned long stub = (unsigned long)(&ftrace_graph_stub);
- ppc_inst_t old, new;
-
- if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_ARGS))
- return 0;
-
- old = ftrace_call_replace(ip, enable ? stub : addr, 0);
- new = ftrace_call_replace(ip, enable ? addr : stub, 0);
-
- return ftrace_modify_code(ip, old, new);
-}
-
-int ftrace_enable_ftrace_graph_caller(void)
-{
- return ftrace_modify_ftrace_graph_caller(true);
-}
-
-int ftrace_disable_ftrace_graph_caller(void)
-{
- return ftrace_modify_ftrace_graph_caller(false);
-}
-
-/*
- * Hook the return address and push it in the stack of return addrs
- * in current thread info. Return the address we want to divert to.
- */
-static unsigned long
-__prepare_ftrace_return(unsigned long parent, unsigned long ip, unsigned long sp)
+void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *op, struct ftrace_regs *fregs)
{
- unsigned long return_hooker;
+ unsigned long sp = fregs->regs.gpr[1];
int bit;
if (unlikely(ftrace_graph_is_dead()))
if (unlikely(atomic_read(¤t->tracing_graph_pause)))
goto out;
- bit = ftrace_test_recursion_trylock(ip, parent);
+ bit = ftrace_test_recursion_trylock(ip, parent_ip);
if (bit < 0)
goto out;
- return_hooker = ppc_function_entry(return_to_handler);
-
- if (!function_graph_enter(parent, ip, 0, (unsigned long *)sp))
- parent = return_hooker;
+ if (!function_graph_enter(parent_ip, ip, 0, (unsigned long *)sp))
+ parent_ip = ppc_function_entry(return_to_handler);
ftrace_test_recursion_unlock(bit);
out:
- return parent;
+ fregs->regs.link = parent_ip;
}
-
-#ifdef CONFIG_DYNAMIC_FTRACE_WITH_ARGS
-void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
- struct ftrace_ops *op, struct ftrace_regs *fregs)
-{
- fregs->regs.link = __prepare_ftrace_return(parent_ip, ip, fregs->regs.gpr[1]);
-}
-#else
-unsigned long prepare_ftrace_return(unsigned long parent, unsigned long ip,
- unsigned long sp)
-{
- return __prepare_ftrace_return(parent, ip, sp);
-}
-#endif
#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
-
-#ifdef CONFIG_PPC64_ELF_ABI_V1
-char *arch_ftrace_match_adjust(char *str, const char *search)
-{
- if (str[0] == '.' && search[0] != '.')
- return str + 1;
- else
- return str;
-}
-#endif /* CONFIG_PPC64_ELF_ABI_V1 */
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0-or-later */
-/*
- * Split from ftrace_64.S
- */
-
-#include <linux/magic.h>
-#include <asm/ppc_asm.h>
-#include <asm/asm-offsets.h>
-#include <asm/ftrace.h>
-#include <asm/ppc-opcode.h>
-#include <asm/export.h>
-
-_GLOBAL_TOC(ftrace_caller)
- lbz r3, PACA_FTRACE_ENABLED(r13)
- cmpdi r3, 0
- beqlr
-
- /* Taken from output of objdump from lib64/glibc */
- mflr r3
- ld r11, 0(r1)
- stdu r1, -112(r1)
- std r3, 128(r1)
- ld r4, 16(r11)
- subi r3, r3, MCOUNT_INSN_SIZE
-.globl ftrace_call
-ftrace_call:
- bl ftrace_stub
- nop
-#ifdef CONFIG_FUNCTION_GRAPH_TRACER
-.globl ftrace_graph_call
-ftrace_graph_call:
- b ftrace_graph_stub
-_GLOBAL(ftrace_graph_stub)
-#endif
- ld r0, 128(r1)
- mtlr r0
- addi r1, r1, 112
-
-_GLOBAL(ftrace_stub)
- blr
-
-#ifdef CONFIG_FUNCTION_GRAPH_TRACER
-_GLOBAL(ftrace_graph_caller)
- addi r5, r1, 112
- /* load r4 with local address */
- ld r4, 128(r1)
- subi r4, r4, MCOUNT_INSN_SIZE
-
- /* Grab the LR out of the caller stack frame */
- ld r11, 112(r1)
- ld r3, 16(r11)
-
- bl prepare_ftrace_return
- nop
-
- /*
- * prepare_ftrace_return gives us the address we divert to.
- * Change the LR in the callers stack frame to this.
- */
- ld r11, 112(r1)
- std r3, 16(r11)
-
- ld r0, 128(r1)
- mtlr r0
- addi r1, r1, 112
- blr
-#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Code for replacing ftrace calls with jumps.
+ *
+ * Copyright (C) 2007-2008 Steven Rostedt <srostedt@redhat.com>
+ *
+ * Thanks goes out to P.A. Semi, Inc for supplying me with a PPC64 box.
+ *
+ * Added function graph tracer code, taken from x86 that was written
+ * by Frederic Weisbecker, and ported to PPC by Steven Rostedt.
+ *
+ */
+
+#define pr_fmt(fmt) "ftrace-powerpc: " fmt
+
+#include <linux/spinlock.h>
+#include <linux/hardirq.h>
+#include <linux/uaccess.h>
+#include <linux/module.h>
+#include <linux/ftrace.h>
+#include <linux/percpu.h>
+#include <linux/init.h>
+#include <linux/list.h>
+
+#include <asm/cacheflush.h>
+#include <asm/code-patching.h>
+#include <asm/ftrace.h>
+#include <asm/syscall.h>
+#include <asm/inst.h>
+
+/*
+ * We generally only have a single long_branch tramp and at most 2 or 3 plt
+ * tramps generated. But, we don't use the plt tramps currently. We also allot
+ * 2 tramps after .text and .init.text. So, we only end up with around 3 usable
+ * tramps in total. Set aside 8 just to be sure.
+ */
+#define NUM_FTRACE_TRAMPS 8
+static unsigned long ftrace_tramps[NUM_FTRACE_TRAMPS];
+
+static ppc_inst_t
+ftrace_call_replace(unsigned long ip, unsigned long addr, int link)
+{
+ ppc_inst_t op;
+
+ addr = ppc_function_entry((void *)addr);
+
+ /* if (link) set op to 'bl' else 'b' */
+ create_branch(&op, (u32 *)ip, addr, link ? BRANCH_SET_LINK : 0);
+
+ return op;
+}
+
+static inline int
+ftrace_modify_code(unsigned long ip, ppc_inst_t old, ppc_inst_t new)
+{
+ ppc_inst_t replaced;
+
+ /*
+ * Note:
+ * We are paranoid about modifying text, as if a bug was to happen, it
+ * could cause us to read or write to someplace that could cause harm.
+ * Carefully read and modify the code with probe_kernel_*(), and make
+ * sure what we read is what we expected it to be before modifying it.
+ */
+
+ /* read the text we want to modify */
+ if (copy_inst_from_kernel_nofault(&replaced, (void *)ip))
+ return -EFAULT;
+
+ /* Make sure it is what we expect it to be */
+ if (!ppc_inst_equal(replaced, old)) {
+ pr_err("%p: replaced (%08lx) != old (%08lx)", (void *)ip,
+ ppc_inst_as_ulong(replaced), ppc_inst_as_ulong(old));
+ return -EINVAL;
+ }
+
+ /* replace the text with the new text */
+ return patch_instruction((u32 *)ip, new);
+}
+
+/*
+ * Helper functions that are the same for both PPC64 and PPC32.
+ */
+static int test_24bit_addr(unsigned long ip, unsigned long addr)
+{
+ addr = ppc_function_entry((void *)addr);
+
+ return is_offset_in_branch_range(addr - ip);
+}
+
+static int is_bl_op(ppc_inst_t op)
+{
+ return (ppc_inst_val(op) & ~PPC_LI_MASK) == PPC_RAW_BL(0);
+}
+
+static int is_b_op(ppc_inst_t op)
+{
+ return (ppc_inst_val(op) & ~PPC_LI_MASK) == PPC_RAW_BRANCH(0);
+}
+
+static unsigned long find_bl_target(unsigned long ip, ppc_inst_t op)
+{
+ int offset;
+
+ offset = PPC_LI(ppc_inst_val(op));
+ /* make it signed */
+ if (offset & 0x02000000)
+ offset |= 0xfe000000;
+
+ return ip + (long)offset;
+}
+
+#ifdef CONFIG_MODULES
+static int
+__ftrace_make_nop(struct module *mod,
+ struct dyn_ftrace *rec, unsigned long addr)
+{
+ unsigned long entry, ptr, tramp;
+ unsigned long ip = rec->ip;
+ ppc_inst_t op, pop;
+
+ /* read where this goes */
+ if (copy_inst_from_kernel_nofault(&op, (void *)ip)) {
+ pr_err("Fetching opcode failed.\n");
+ return -EFAULT;
+ }
+
+ /* Make sure that this is still a 24bit jump */
+ if (!is_bl_op(op)) {
+ pr_err("Not expected bl: opcode is %08lx\n", ppc_inst_as_ulong(op));
+ return -EINVAL;
+ }
+
+ /* lets find where the pointer goes */
+ tramp = find_bl_target(ip, op);
+
+ pr_devel("ip:%lx jumps to %lx", ip, tramp);
+
+ if (module_trampoline_target(mod, tramp, &ptr)) {
+ pr_err("Failed to get trampoline target\n");
+ return -EFAULT;
+ }
+
+ pr_devel("trampoline target %lx", ptr);
+
+ entry = ppc_global_function_entry((void *)addr);
+ /* This should match what was called */
+ if (ptr != entry) {
+ pr_err("addr %lx does not match expected %lx\n", ptr, entry);
+ return -EINVAL;
+ }
+
+ if (IS_ENABLED(CONFIG_MPROFILE_KERNEL)) {
+ if (copy_inst_from_kernel_nofault(&op, (void *)(ip - 4))) {
+ pr_err("Fetching instruction at %lx failed.\n", ip - 4);
+ return -EFAULT;
+ }
+
+ /* We expect either a mflr r0, or a std r0, LRSAVE(r1) */
+ if (!ppc_inst_equal(op, ppc_inst(PPC_RAW_MFLR(_R0))) &&
+ !ppc_inst_equal(op, ppc_inst(PPC_INST_STD_LR))) {
+ pr_err("Unexpected instruction %08lx around bl _mcount\n",
+ ppc_inst_as_ulong(op));
+ return -EINVAL;
+ }
+ } else if (IS_ENABLED(CONFIG_PPC64)) {
+ /*
+ * Check what is in the next instruction. We can see ld r2,40(r1), but
+ * on first pass after boot we will see mflr r0.
+ */
+ if (copy_inst_from_kernel_nofault(&op, (void *)(ip + 4))) {
+ pr_err("Fetching op failed.\n");
+ return -EFAULT;
+ }
+
+ if (!ppc_inst_equal(op, ppc_inst(PPC_INST_LD_TOC))) {
+ pr_err("Expected %08lx found %08lx\n", PPC_INST_LD_TOC,
+ ppc_inst_as_ulong(op));
+ return -EINVAL;
+ }
+ }
+
+ /*
+ * When using -mprofile-kernel or PPC32 there is no load to jump over.
+ *
+ * Otherwise our original call site looks like:
+ *
+ * bl <tramp>
+ * ld r2,XX(r1)
+ *
+ * Milton Miller pointed out that we can not simply nop the branch.
+ * If a task was preempted when calling a trace function, the nops
+ * will remove the way to restore the TOC in r2 and the r2 TOC will
+ * get corrupted.
+ *
+ * Use a b +8 to jump over the load.
+ */
+ if (IS_ENABLED(CONFIG_MPROFILE_KERNEL) || IS_ENABLED(CONFIG_PPC32))
+ pop = ppc_inst(PPC_RAW_NOP());
+ else
+ pop = ppc_inst(PPC_RAW_BRANCH(8)); /* b +8 */
+
+ if (patch_instruction((u32 *)ip, pop)) {
+ pr_err("Patching NOP failed.\n");
+ return -EPERM;
+ }
+
+ return 0;
+}
+#else
+static int __ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, unsigned long addr)
+{
+ return 0;
+}
+#endif /* CONFIG_MODULES */
+
+static unsigned long find_ftrace_tramp(unsigned long ip)
+{
+ int i;
+
+ /*
+ * We have the compiler generated long_branch tramps at the end
+ * and we prefer those
+ */
+ for (i = NUM_FTRACE_TRAMPS - 1; i >= 0; i--)
+ if (!ftrace_tramps[i])
+ continue;
+ else if (is_offset_in_branch_range(ftrace_tramps[i] - ip))
+ return ftrace_tramps[i];
+
+ return 0;
+}
+
+static int add_ftrace_tramp(unsigned long tramp)
+{
+ int i;
+
+ for (i = 0; i < NUM_FTRACE_TRAMPS; i++)
+ if (!ftrace_tramps[i]) {
+ ftrace_tramps[i] = tramp;
+ return 0;
+ }
+
+ return -1;
+}
+
+/*
+ * If this is a compiler generated long_branch trampoline (essentially, a
+ * trampoline that has a branch to _mcount()), we re-write the branch to
+ * instead go to ftrace_[regs_]caller() and note down the location of this
+ * trampoline.
+ */
+static int setup_mcount_compiler_tramp(unsigned long tramp)
+{
+ int i;
+ ppc_inst_t op;
+ unsigned long ptr;
+
+ /* Is this a known long jump tramp? */
+ for (i = 0; i < NUM_FTRACE_TRAMPS; i++)
+ if (ftrace_tramps[i] == tramp)
+ return 0;
+
+ /* New trampoline -- read where this goes */
+ if (copy_inst_from_kernel_nofault(&op, (void *)tramp)) {
+ pr_debug("Fetching opcode failed.\n");
+ return -1;
+ }
+
+ /* Is this a 24 bit branch? */
+ if (!is_b_op(op)) {
+ pr_debug("Trampoline is not a long branch tramp.\n");
+ return -1;
+ }
+
+ /* lets find where the pointer goes */
+ ptr = find_bl_target(tramp, op);
+
+ if (ptr != ppc_global_function_entry((void *)_mcount)) {
+ pr_debug("Trampoline target %p is not _mcount\n", (void *)ptr);
+ return -1;
+ }
+
+ /* Let's re-write the tramp to go to ftrace_[regs_]caller */
+ if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
+ ptr = ppc_global_function_entry((void *)ftrace_regs_caller);
+ else
+ ptr = ppc_global_function_entry((void *)ftrace_caller);
+
+ if (patch_branch((u32 *)tramp, ptr, 0)) {
+ pr_debug("REL24 out of range!\n");
+ return -1;
+ }
+
+ if (add_ftrace_tramp(tramp)) {
+ pr_debug("No tramp locations left\n");
+ return -1;
+ }
+
+ return 0;
+}
+
+static int __ftrace_make_nop_kernel(struct dyn_ftrace *rec, unsigned long addr)
+{
+ unsigned long tramp, ip = rec->ip;
+ ppc_inst_t op;
+
+ /* Read where this goes */
+ if (copy_inst_from_kernel_nofault(&op, (void *)ip)) {
+ pr_err("Fetching opcode failed.\n");
+ return -EFAULT;
+ }
+
+ /* Make sure that this is still a 24bit jump */
+ if (!is_bl_op(op)) {
+ pr_err("Not expected bl: opcode is %08lx\n", ppc_inst_as_ulong(op));
+ return -EINVAL;
+ }
+
+ /* Let's find where the pointer goes */
+ tramp = find_bl_target(ip, op);
+
+ pr_devel("ip:%lx jumps to %lx", ip, tramp);
+
+ if (setup_mcount_compiler_tramp(tramp)) {
+ /* Are other trampolines reachable? */
+ if (!find_ftrace_tramp(ip)) {
+ pr_err("No ftrace trampolines reachable from %ps\n",
+ (void *)ip);
+ return -EINVAL;
+ }
+ }
+
+ if (patch_instruction((u32 *)ip, ppc_inst(PPC_RAW_NOP()))) {
+ pr_err("Patching NOP failed.\n");
+ return -EPERM;
+ }
+
+ return 0;
+}
+
+int ftrace_make_nop(struct module *mod,
+ struct dyn_ftrace *rec, unsigned long addr)
+{
+ unsigned long ip = rec->ip;
+ ppc_inst_t old, new;
+
+ /*
+ * If the calling address is more that 24 bits away,
+ * then we had to use a trampoline to make the call.
+ * Otherwise just update the call site.
+ */
+ if (test_24bit_addr(ip, addr)) {
+ /* within range */
+ old = ftrace_call_replace(ip, addr, 1);
+ new = ppc_inst(PPC_RAW_NOP());
+ return ftrace_modify_code(ip, old, new);
+ } else if (core_kernel_text(ip)) {
+ return __ftrace_make_nop_kernel(rec, addr);
+ } else if (!IS_ENABLED(CONFIG_MODULES)) {
+ return -EINVAL;
+ }
+
+ /*
+ * Out of range jumps are called from modules.
+ * We should either already have a pointer to the module
+ * or it has been passed in.
+ */
+ if (!rec->arch.mod) {
+ if (!mod) {
+ pr_err("No module loaded addr=%lx\n", addr);
+ return -EFAULT;
+ }
+ rec->arch.mod = mod;
+ } else if (mod) {
+ if (mod != rec->arch.mod) {
+ pr_err("Record mod %p not equal to passed in mod %p\n",
+ rec->arch.mod, mod);
+ return -EINVAL;
+ }
+ /* nothing to do if mod == rec->arch.mod */
+ } else
+ mod = rec->arch.mod;
+
+ return __ftrace_make_nop(mod, rec, addr);
+}
+
+#ifdef CONFIG_MODULES
+/*
+ * Examine the existing instructions for __ftrace_make_call.
+ * They should effectively be a NOP, and follow formal constraints,
+ * depending on the ABI. Return false if they don't.
+ */
+static bool expected_nop_sequence(void *ip, ppc_inst_t op0, ppc_inst_t op1)
+{
+ if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
+ return ppc_inst_equal(op0, ppc_inst(PPC_RAW_NOP()));
+ else
+ return ppc_inst_equal(op0, ppc_inst(PPC_RAW_BRANCH(8))) &&
+ ppc_inst_equal(op1, ppc_inst(PPC_INST_LD_TOC));
+}
+
+static int
+__ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
+{
+ ppc_inst_t op[2];
+ void *ip = (void *)rec->ip;
+ unsigned long entry, ptr, tramp;
+ struct module *mod = rec->arch.mod;
+
+ /* read where this goes */
+ if (copy_inst_from_kernel_nofault(op, ip))
+ return -EFAULT;
+
+ if (!IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS) &&
+ copy_inst_from_kernel_nofault(op + 1, ip + 4))
+ return -EFAULT;
+
+ if (!expected_nop_sequence(ip, op[0], op[1])) {
+ pr_err("Unexpected call sequence at %p: %08lx %08lx\n", ip,
+ ppc_inst_as_ulong(op[0]), ppc_inst_as_ulong(op[1]));
+ return -EINVAL;
+ }
+
+ /* If we never set up ftrace trampoline(s), then bail */
+ if (!mod->arch.tramp ||
+ (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS) && !mod->arch.tramp_regs)) {
+ pr_err("No ftrace trampoline\n");
+ return -EINVAL;
+ }
+
+ if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS) && rec->flags & FTRACE_FL_REGS)
+ tramp = mod->arch.tramp_regs;
+ else
+ tramp = mod->arch.tramp;
+
+ if (module_trampoline_target(mod, tramp, &ptr)) {
+ pr_err("Failed to get trampoline target\n");
+ return -EFAULT;
+ }
+
+ pr_devel("trampoline target %lx", ptr);
+
+ entry = ppc_global_function_entry((void *)addr);
+ /* This should match what was called */
+ if (ptr != entry) {
+ pr_err("addr %lx does not match expected %lx\n", ptr, entry);
+ return -EINVAL;
+ }
+
+ if (patch_branch(ip, tramp, BRANCH_SET_LINK)) {
+ pr_err("REL24 out of range!\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+#else
+static int __ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
+{
+ return 0;
+}
+#endif /* CONFIG_MODULES */
+
+static int __ftrace_make_call_kernel(struct dyn_ftrace *rec, unsigned long addr)
+{
+ ppc_inst_t op;
+ void *ip = (void *)rec->ip;
+ unsigned long tramp, entry, ptr;
+
+ /* Make sure we're being asked to patch branch to a known ftrace addr */
+ entry = ppc_global_function_entry((void *)ftrace_caller);
+ ptr = ppc_global_function_entry((void *)addr);
+
+ if (ptr != entry && IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
+ entry = ppc_global_function_entry((void *)ftrace_regs_caller);
+
+ if (ptr != entry) {
+ pr_err("Unknown ftrace addr to patch: %ps\n", (void *)ptr);
+ return -EINVAL;
+ }
+
+ /* Make sure we have a nop */
+ if (copy_inst_from_kernel_nofault(&op, ip)) {
+ pr_err("Unable to read ftrace location %p\n", ip);
+ return -EFAULT;
+ }
+
+ if (!ppc_inst_equal(op, ppc_inst(PPC_RAW_NOP()))) {
+ pr_err("Unexpected call sequence at %p: %08lx\n",
+ ip, ppc_inst_as_ulong(op));
+ return -EINVAL;
+ }
+
+ tramp = find_ftrace_tramp((unsigned long)ip);
+ if (!tramp) {
+ pr_err("No ftrace trampolines reachable from %ps\n", ip);
+ return -EINVAL;
+ }
+
+ if (patch_branch(ip, tramp, BRANCH_SET_LINK)) {
+ pr_err("Error patching branch to ftrace tramp!\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
+{
+ unsigned long ip = rec->ip;
+ ppc_inst_t old, new;
+
+ /*
+ * If the calling address is more that 24 bits away,
+ * then we had to use a trampoline to make the call.
+ * Otherwise just update the call site.
+ */
+ if (test_24bit_addr(ip, addr)) {
+ /* within range */
+ old = ppc_inst(PPC_RAW_NOP());
+ new = ftrace_call_replace(ip, addr, 1);
+ return ftrace_modify_code(ip, old, new);
+ } else if (core_kernel_text(ip)) {
+ return __ftrace_make_call_kernel(rec, addr);
+ } else if (!IS_ENABLED(CONFIG_MODULES)) {
+ /* We should not get here without modules */
+ return -EINVAL;
+ }
+
+ /*
+ * Out of range jumps are called from modules.
+ * Being that we are converting from nop, it had better
+ * already have a module defined.
+ */
+ if (!rec->arch.mod) {
+ pr_err("No module loaded\n");
+ return -EINVAL;
+ }
+
+ return __ftrace_make_call(rec, addr);
+}
+
+#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
+#ifdef CONFIG_MODULES
+static int
+__ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
+ unsigned long addr)
+{
+ ppc_inst_t op;
+ unsigned long ip = rec->ip;
+ unsigned long entry, ptr, tramp;
+ struct module *mod = rec->arch.mod;
+
+ /* If we never set up ftrace trampolines, then bail */
+ if (!mod->arch.tramp || !mod->arch.tramp_regs) {
+ pr_err("No ftrace trampoline\n");
+ return -EINVAL;
+ }
+
+ /* read where this goes */
+ if (copy_inst_from_kernel_nofault(&op, (void *)ip)) {
+ pr_err("Fetching opcode failed.\n");
+ return -EFAULT;
+ }
+
+ /* Make sure that this is still a 24bit jump */
+ if (!is_bl_op(op)) {
+ pr_err("Not expected bl: opcode is %08lx\n", ppc_inst_as_ulong(op));
+ return -EINVAL;
+ }
+
+ /* lets find where the pointer goes */
+ tramp = find_bl_target(ip, op);
+ entry = ppc_global_function_entry((void *)old_addr);
+
+ pr_devel("ip:%lx jumps to %lx", ip, tramp);
+
+ if (tramp != entry) {
+ /* old_addr is not within range, so we must have used a trampoline */
+ if (module_trampoline_target(mod, tramp, &ptr)) {
+ pr_err("Failed to get trampoline target\n");
+ return -EFAULT;
+ }
+
+ pr_devel("trampoline target %lx", ptr);
+
+ /* This should match what was called */
+ if (ptr != entry) {
+ pr_err("addr %lx does not match expected %lx\n", ptr, entry);
+ return -EINVAL;
+ }
+ }
+
+ /* The new target may be within range */
+ if (test_24bit_addr(ip, addr)) {
+ /* within range */
+ if (patch_branch((u32 *)ip, addr, BRANCH_SET_LINK)) {
+ pr_err("REL24 out of range!\n");
+ return -EINVAL;
+ }
+
+ return 0;
+ }
+
+ if (rec->flags & FTRACE_FL_REGS)
+ tramp = mod->arch.tramp_regs;
+ else
+ tramp = mod->arch.tramp;
+
+ if (module_trampoline_target(mod, tramp, &ptr)) {
+ pr_err("Failed to get trampoline target\n");
+ return -EFAULT;
+ }
+
+ pr_devel("trampoline target %lx", ptr);
+
+ entry = ppc_global_function_entry((void *)addr);
+ /* This should match what was called */
+ if (ptr != entry) {
+ pr_err("addr %lx does not match expected %lx\n", ptr, entry);
+ return -EINVAL;
+ }
+
+ if (patch_branch((u32 *)ip, tramp, BRANCH_SET_LINK)) {
+ pr_err("REL24 out of range!\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+#else
+static int __ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr, unsigned long addr)
+{
+ return 0;
+}
+#endif
+
+int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
+ unsigned long addr)
+{
+ unsigned long ip = rec->ip;
+ ppc_inst_t old, new;
+
+ /*
+ * If the calling address is more that 24 bits away,
+ * then we had to use a trampoline to make the call.
+ * Otherwise just update the call site.
+ */
+ if (test_24bit_addr(ip, addr) && test_24bit_addr(ip, old_addr)) {
+ /* within range */
+ old = ftrace_call_replace(ip, old_addr, 1);
+ new = ftrace_call_replace(ip, addr, 1);
+ return ftrace_modify_code(ip, old, new);
+ } else if (core_kernel_text(ip)) {
+ /*
+ * We always patch out of range locations to go to the regs
+ * variant, so there is nothing to do here
+ */
+ return 0;
+ } else if (!IS_ENABLED(CONFIG_MODULES)) {
+ /* We should not get here without modules */
+ return -EINVAL;
+ }
+
+ /*
+ * Out of range jumps are called from modules.
+ */
+ if (!rec->arch.mod) {
+ pr_err("No module loaded\n");
+ return -EINVAL;
+ }
+
+ return __ftrace_modify_call(rec, old_addr, addr);
+}
+#endif
+
+int ftrace_update_ftrace_func(ftrace_func_t func)
+{
+ unsigned long ip = (unsigned long)(&ftrace_call);
+ ppc_inst_t old, new;
+ int ret;
+
+ old = ppc_inst_read((u32 *)&ftrace_call);
+ new = ftrace_call_replace(ip, (unsigned long)func, 1);
+ ret = ftrace_modify_code(ip, old, new);
+
+ /* Also update the regs callback function */
+ if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS) && !ret) {
+ ip = (unsigned long)(&ftrace_regs_call);
+ old = ppc_inst_read((u32 *)&ftrace_regs_call);
+ new = ftrace_call_replace(ip, (unsigned long)func, 1);
+ ret = ftrace_modify_code(ip, old, new);
+ }
+
+ return ret;
+}
+
+/*
+ * Use the default ftrace_modify_all_code, but without
+ * stop_machine().
+ */
+void arch_ftrace_update_code(int command)
+{
+ ftrace_modify_all_code(command);
+}
+
+#ifdef CONFIG_PPC64
+#define PACATOC offsetof(struct paca_struct, kernel_toc)
+
+extern unsigned int ftrace_tramp_text[], ftrace_tramp_init[];
+
+void ftrace_free_init_tramp(void)
+{
+ int i;
+
+ for (i = 0; i < NUM_FTRACE_TRAMPS && ftrace_tramps[i]; i++)
+ if (ftrace_tramps[i] == (unsigned long)ftrace_tramp_init) {
+ ftrace_tramps[i] = 0;
+ return;
+ }
+}
+
+int __init ftrace_dyn_arch_init(void)
+{
+ int i;
+ unsigned int *tramp[] = { ftrace_tramp_text, ftrace_tramp_init };
+ u32 stub_insns[] = {
+ PPC_RAW_LD(_R12, _R13, PACATOC),
+ PPC_RAW_ADDIS(_R12, _R12, 0),
+ PPC_RAW_ADDI(_R12, _R12, 0),
+ PPC_RAW_MTCTR(_R12),
+ PPC_RAW_BCTR()
+ };
+ unsigned long addr;
+ long reladdr;
+
+ if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
+ addr = ppc_global_function_entry((void *)ftrace_regs_caller);
+ else
+ addr = ppc_global_function_entry((void *)ftrace_caller);
+
+ reladdr = addr - kernel_toc_addr();
+
+ if (reladdr >= SZ_2G || reladdr < -(long)SZ_2G) {
+ pr_err("Address of %ps out of range of kernel_toc.\n",
+ (void *)addr);
+ return -1;
+ }
+
+ for (i = 0; i < 2; i++) {
+ memcpy(tramp[i], stub_insns, sizeof(stub_insns));
+ tramp[i][1] |= PPC_HA(reladdr);
+ tramp[i][2] |= PPC_LO(reladdr);
+ add_ftrace_tramp((unsigned long)tramp[i]);
+ }
+
+ return 0;
+}
+#endif
+
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+
+extern void ftrace_graph_call(void);
+extern void ftrace_graph_stub(void);
+
+static int ftrace_modify_ftrace_graph_caller(bool enable)
+{
+ unsigned long ip = (unsigned long)(&ftrace_graph_call);
+ unsigned long addr = (unsigned long)(&ftrace_graph_caller);
+ unsigned long stub = (unsigned long)(&ftrace_graph_stub);
+ ppc_inst_t old, new;
+
+ if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_ARGS))
+ return 0;
+
+ old = ftrace_call_replace(ip, enable ? stub : addr, 0);
+ new = ftrace_call_replace(ip, enable ? addr : stub, 0);
+
+ return ftrace_modify_code(ip, old, new);
+}
+
+int ftrace_enable_ftrace_graph_caller(void)
+{
+ return ftrace_modify_ftrace_graph_caller(true);
+}
+
+int ftrace_disable_ftrace_graph_caller(void)
+{
+ return ftrace_modify_ftrace_graph_caller(false);
+}
+
+/*
+ * Hook the return address and push it in the stack of return addrs
+ * in current thread info. Return the address we want to divert to.
+ */
+static unsigned long
+__prepare_ftrace_return(unsigned long parent, unsigned long ip, unsigned long sp)
+{
+ unsigned long return_hooker;
+ int bit;
+
+ if (unlikely(ftrace_graph_is_dead()))
+ goto out;
+
+ if (unlikely(atomic_read(¤t->tracing_graph_pause)))
+ goto out;
+
+ bit = ftrace_test_recursion_trylock(ip, parent);
+ if (bit < 0)
+ goto out;
+
+ return_hooker = ppc_function_entry(return_to_handler);
+
+ if (!function_graph_enter(parent, ip, 0, (unsigned long *)sp))
+ parent = return_hooker;
+
+ ftrace_test_recursion_unlock(bit);
+out:
+ return parent;
+}
+
+#ifdef CONFIG_DYNAMIC_FTRACE_WITH_ARGS
+void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *op, struct ftrace_regs *fregs)
+{
+ fregs->regs.link = __prepare_ftrace_return(parent_ip, ip, fregs->regs.gpr[1]);
+}
+#else
+unsigned long prepare_ftrace_return(unsigned long parent, unsigned long ip,
+ unsigned long sp)
+{
+ return __prepare_ftrace_return(parent, ip, sp);
+}
+#endif
+#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
+
+#ifdef CONFIG_PPC64_ELF_ABI_V1
+char *arch_ftrace_match_adjust(char *str, const char *search)
+{
+ if (str[0] == '.' && search[0] != '.')
+ return str + 1;
+ else
+ return str;
+}
+#endif /* CONFIG_PPC64_ELF_ABI_V1 */
/* SPDX-License-Identifier: GPL-2.0-or-later */
/*
- * Split from entry_64.S
+ * Split from ftrace_64.S
*/
+#include <linux/export.h>
#include <linux/magic.h>
#include <asm/ppc_asm.h>
#include <asm/asm-offsets.h>
#include <asm/ftrace.h>
#include <asm/ppc-opcode.h>
-#include <asm/export.h>
-#ifdef CONFIG_PPC64
+_GLOBAL_TOC(ftrace_caller)
+ lbz r3, PACA_FTRACE_ENABLED(r13)
+ cmpdi r3, 0
+ beqlr
+
+ /* Taken from output of objdump from lib64/glibc */
+ mflr r3
+ ld r11, 0(r1)
+ stdu r1, -112(r1)
+ std r3, 128(r1)
+ ld r4, 16(r11)
+ subi r3, r3, MCOUNT_INSN_SIZE
+.globl ftrace_call
+ftrace_call:
+ bl ftrace_stub
+ nop
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+.globl ftrace_graph_call
+ftrace_graph_call:
+ b ftrace_graph_stub
+_GLOBAL(ftrace_graph_stub)
+#endif
+ ld r0, 128(r1)
+ mtlr r0
+ addi r1, r1, 112
+
+_GLOBAL(ftrace_stub)
+ blr
+
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+_GLOBAL(ftrace_graph_caller)
+ addi r5, r1, 112
+ /* load r4 with local address */
+ ld r4, 128(r1)
+ subi r4, r4, MCOUNT_INSN_SIZE
+
+ /* Grab the LR out of the caller stack frame */
+ ld r11, 112(r1)
+ ld r3, 16(r11)
+
+ bl prepare_ftrace_return
+ nop
+
+ /*
+ * prepare_ftrace_return gives us the address we divert to.
+ * Change the LR in the callers stack frame to this.
+ */
+ ld r11, 112(r1)
+ std r3, 16(r11)
+
+ ld r0, 128(r1)
+ mtlr r0
+ addi r1, r1, 112
+ blr
+#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
+
.pushsection ".tramp.ftrace.text","aw",@progbits;
.globl ftrace_tramp_text
ftrace_tramp_text:
- .space 64
+ .space 32
.popsection
.pushsection ".tramp.ftrace.init","aw",@progbits;
.globl ftrace_tramp_init
ftrace_tramp_init:
- .space 64
+ .space 32
.popsection
-#endif
_GLOBAL(mcount)
_GLOBAL(_mcount)
* Split from ftrace_64.S
*/
+#include <linux/export.h>
#include <linux/magic.h>
#include <asm/ppc_asm.h>
#include <asm/asm-offsets.h>
#include <asm/ftrace.h>
#include <asm/ppc-opcode.h>
-#include <asm/export.h>
#include <asm/thread_info.h>
#include <asm/bug.h>
#include <asm/ptrace.h>
/* Return to original caller of live patched function */
blr
#endif /* CONFIG_LIVEPATCH */
+
+#ifndef CONFIG_ARCH_USING_PATCHABLE_FUNCTION_ENTRY
+_GLOBAL(mcount)
+_GLOBAL(_mcount)
+EXPORT_SYMBOL(_mcount)
+ mflr r12
+ mtctr r12
+ mtlr r0
+ bctr
+#endif
+
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+_GLOBAL(return_to_handler)
+ /* need to save return values */
+#ifdef CONFIG_PPC64
+ std r4, -32(r1)
+ std r3, -24(r1)
+ /* save TOC */
+ std r2, -16(r1)
+ std r31, -8(r1)
+ mr r31, r1
+ stdu r1, -112(r1)
+
+ /*
+ * We might be called from a module.
+ * Switch to our TOC to run inside the core kernel.
+ */
+ LOAD_PACA_TOC()
+#else
+ stwu r1, -16(r1)
+ stw r3, 8(r1)
+ stw r4, 12(r1)
+#endif
+
+ bl ftrace_return_to_handler
+ nop
+
+ /* return value has real return address */
+ mtlr r3
+
+#ifdef CONFIG_PPC64
+ ld r1, 0(r1)
+ ld r4, -32(r1)
+ ld r3, -24(r1)
+ ld r2, -16(r1)
+ ld r31, -8(r1)
+#else
+ lwz r3, 8(r1)
+ lwz r4, 12(r1)
+ addi r1, r1, 16
+#endif
+
+ /* Jump back to real return address */
+ blr
+#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
+
+.pushsection ".tramp.ftrace.text","aw",@progbits;
+.globl ftrace_tramp_text
+ftrace_tramp_text:
+ .space 32
+.popsection
+
+.pushsection ".tramp.ftrace.init","aw",@progbits;
+.globl ftrace_tramp_init
+ftrace_tramp_init:
+ .space 32
+.popsection
* pretend we got a single-step exception. This was pointed out
* by Kumar Gala. -- paulus
*/
-static void emulate_single_step(struct pt_regs *regs)
+void emulate_single_step(struct pt_regs *regs)
{
if (single_stepping(regs))
__single_step_exception(regs);
}
#if defined(CONFIG_BOOKE_WDT) || defined(CONFIG_40x)
-/*
- * Default handler for a Watchdog exception,
- * spins until a reboot occurs
- */
-void __attribute__ ((weak)) WatchdogHandler(struct pt_regs *regs)
-{
- /* Generic WatchdogHandler, implement your own */
- mtspr(SPRN_TCR, mfspr(SPRN_TCR)&(~TCR_WIE));
- return;
-}
-
DEFINE_INTERRUPT_HANDLER_NMI(WatchdogException)
{
printk (KERN_EMERG "PowerPC Book-E Watchdog Exception\n");
- WatchdogHandler(regs);
+ mtspr(SPRN_TCR, mfspr(SPRN_TCR) & ~TCR_WIE);
return 0;
}
#endif
* Copyright 2019, IBM Corporation.
*
*/
+#include <linux/export.h>
#include <asm/ppc_asm.h>
-#include <asm/export.h>
_GLOBAL(ucall_norets)
EXPORT_SYMBOL_GPL(ucall_norets)
/* SPDX-License-Identifier: GPL-2.0 */
+#include <linux/export.h>
#include <linux/linkage.h>
#include <asm/processor.h>
#include <asm/ppc_asm.h>
#include <asm/thread_info.h>
#include <asm/page.h>
#include <asm/ptrace.h>
-#include <asm/export.h>
#include <asm/asm-compat.h>
/*
#endif
/* careful! __ftr_alt_* sections need to be close to .text */
*(.text.hot .text.hot.* TEXT_MAIN .text.fixup .text.unlikely .text.unlikely.* .fixup __ftr_alt_* .ref.text);
-#ifdef CONFIG_PPC64
*(.tramp.ftrace.text);
-#endif
NOINSTR_TEXT
SCHED_TEXT
LOCK_TEXT
*/
. = ALIGN(PAGE_SIZE);
_einittext = .;
-#ifdef CONFIG_PPC64
*(.tramp.ftrace.init);
-#endif
} :text
/* .exit.text is discarded at runtime, not link time,
void default_machine_crash_shutdown(struct pt_regs *regs)
{
- unsigned int i;
+ volatile unsigned int i;
int (*old_handler)(struct pt_regs *regs);
if (TRAP(regs) == INTERRUPT_SYSTEM_RESET)
#include <linux/kexec.h>
#include <linux/of_fdt.h>
#include <linux/libfdt.h>
-#include <linux/of_device.h>
+#include <linux/of.h>
#include <linux/memblock.h>
#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <asm/kexec_ranges.h>
#include <asm/crashdump-ppc64.h>
#include <asm/mmzone.h>
+#include <asm/iommu.h>
#include <asm/prom.h>
#include <asm/plpks.h>
}
/**
- * get_cpu_node_size - Compute the size of a CPU node in the FDT.
- * This should be done only once and the value is stored in
- * a static variable.
+ * cpu_node_size - Compute the size of a CPU node in the FDT.
+ * This should be done only once and the value is stored in
+ * a static variable.
* Returns the max size of a CPU node in the FDT.
*/
static unsigned int cpu_node_size(void)
if (ret < 0)
goto out;
-#define DIRECT64_PROPNAME "linux,direct64-ddr-window-info"
-#define DMA64_PROPNAME "linux,dma64-ddr-window-info"
ret = update_pci_dma_nodes(fdt, DIRECT64_PROPNAME);
if (ret < 0)
goto out;
ret = update_pci_dma_nodes(fdt, DMA64_PROPNAME);
if (ret < 0)
goto out;
-#undef DMA64_PROPNAME
-#undef DIRECT64_PROPNAME
/* Update memory reserve map */
ret = get_reserved_memory_ranges(&rmem);
#include <linux/sort.h>
#include <linux/kexec.h>
-#include <linux/of_device.h>
+#include <linux/of.h>
#include <linux/slab.h>
#include <asm/sections.h>
#include <asm/kexec_ranges.h>
/* SPDX-License-Identifier: GPL-2.0-only */
+#include <linux/export.h>
#include <asm/asm-offsets.h>
#include <asm/cache.h>
#include <asm/code-patching-asm.h>
#include <asm/exception-64s.h>
-#include <asm/export.h>
#include <asm/kvm_asm.h>
#include <asm/kvm_book3s_asm.h>
#include <asm/mmu.h>
vfree(info->rev);
info->rev = NULL;
if (info->cma)
- kvm_free_hpt_cma(virt_to_page(info->virt),
+ kvm_free_hpt_cma(virt_to_page((void *)info->virt),
1 << (info->order - PAGE_SHIFT));
else if (info->virt)
free_pages(info->virt, info->order - PAGE_SHIFT);
#include <linux/kvm.h>
#include <linux/kvm_host.h>
#include <linux/kernel.h>
+#include <asm/lppaca.h>
#include <asm/opal.h>
#include <asm/mce.h>
#include <asm/machdep.h>
* Authors: Alexander Graf <agraf@suse.de>
*/
+#include <linux/export.h>
#include <linux/linkage.h>
#include <linux/objtool.h>
#include <asm/ppc_asm.h>
#include <asm/exception-64s.h>
#include <asm/kvm_book3s_asm.h>
#include <asm/book3s/64/mmu-hash.h>
-#include <asm/export.h>
#include <asm/tm.h>
#include <asm/opal.h>
#include <asm/thread_info.h>
#include <asm/cputable.h>
#include <asm/kvm_ppc.h>
#include <asm/dbell.h>
+#include <asm/ppc-opcode.h>
#include "booke.h"
#include "e500.h"
local_irq_save(flags);
mtspr(SPRN_MAS5, MAS5_SGS | get_lpid(&vcpu_e500->vcpu));
- asm volatile("tlbilxlpid");
+ /*
+ * clang-17 and older could not assemble tlbilxlpid.
+ * https://github.com/ClangBuiltLinux/linux/issues/1891
+ */
+ asm volatile (PPC_TLBILX_LPID);
mtspr(SPRN_MAS5, 0);
local_irq_restore(flags);
}
* Copyright 2011 Paul Mackerras, IBM Corp. <paulus@au1.ibm.com>
*/
+#include <linux/export.h>
#include <asm/reg.h>
#include <asm/ppc_asm.h>
#include <asm/asm-offsets.h>
-#include <asm/export.h>
#include <asm/tm.h>
#include <asm/cputable.h>
CFLAGS_code-patching.o += $(DISABLE_LATENT_ENTROPY_PLUGIN)
CFLAGS_feature-fixups.o += $(DISABLE_LATENT_ENTROPY_PLUGIN)
-obj-y += alloc.o code-patching.o feature-fixups.o pmem.o
+obj-y += code-patching.o feature-fixups.o pmem.o
obj-$(CONFIG_CODE_PATCHING_SELFTEST) += test-code-patching.o
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0
-#include <linux/types.h>
-#include <linux/init.h>
-#include <linux/slab.h>
-#include <linux/memblock.h>
-#include <linux/string.h>
-#include <asm/setup.h>
-
-
-void * __ref zalloc_maybe_bootmem(size_t size, gfp_t mask)
-{
- void *p;
-
- if (slab_is_available())
- p = kzalloc(size, mask);
- else {
- p = memblock_alloc(size, SMP_CACHE_BYTES);
- if (!p)
- panic("%s: Failed to allocate %zu bytes\n", __func__,
- size);
- }
- return p;
-}
* Severely hacked about by Paul Mackerras (paulus@cs.anu.edu.au).
*/
+#include <linux/export.h>
#include <linux/sys.h>
#include <asm/processor.h>
#include <asm/cache.h>
#include <asm/errno.h>
#include <asm/ppc_asm.h>
-#include <asm/export.h>
.text
* Severely hacked about by Paul Mackerras (paulus@cs.anu.edu.au).
*/
+#include <linux/export.h>
#include <linux/sys.h>
#include <asm/processor.h>
#include <asm/errno.h>
#include <asm/ppc_asm.h>
-#include <asm/export.h>
/*
* Computes the checksum of a memory block at buff, length len,
*
* Copyright (C) 1996-2005 Paul Mackerras.
*/
+#include <linux/export.h>
#include <asm/processor.h>
#include <asm/cache.h>
#include <asm/errno.h>
#include <asm/ppc_asm.h>
-#include <asm/export.h>
#include <asm/code-patching-asm.h>
#include <asm/kasan.h>
* Derived from copyuser_power7.s by Anton Blanchard <anton@au.ibm.com>
* Author - Balbir Singh <bsingharora@gmail.com>
*/
+#include <linux/export.h>
#include <asm/ppc_asm.h>
#include <asm/errno.h>
-#include <asm/export.h>
.macro err1
100:
/*
* Copyright (C) 2008 Mark Nelson, IBM Corp.
*/
+#include <linux/export.h>
#include <asm/page.h>
#include <asm/processor.h>
#include <asm/ppc_asm.h>
#include <asm/asm-offsets.h>
-#include <asm/export.h>
#include <asm/feature-fixups.h>
_GLOBAL_TOC(copy_page)
/*
* Copyright (C) 2002 Paul Mackerras, IBM Corp.
*/
+#include <linux/export.h>
#include <asm/processor.h>
#include <asm/ppc_asm.h>
-#include <asm/export.h>
#include <asm/asm-compat.h>
#include <asm/feature-fixups.h>
return 0;
}
-static int patch_feature_section(unsigned long value, struct fixup_entry *fcur)
+static int patch_feature_section_mask(unsigned long value, unsigned long mask,
+ struct fixup_entry *fcur)
{
u32 *start, *end, *alt_start, *alt_end, *src, *dest;
if ((alt_end - alt_start) > (end - start))
return 1;
- if ((value & fcur->mask) == fcur->value)
+ if ((value & fcur->mask & mask) == (fcur->value & mask))
return 0;
src = alt_start;
return 0;
}
-void do_feature_fixups(unsigned long value, void *fixup_start, void *fixup_end)
+static void do_feature_fixups_mask(unsigned long value, unsigned long mask,
+ void *fixup_start, void *fixup_end)
{
struct fixup_entry *fcur, *fend;
fend = fixup_end;
for (; fcur < fend; fcur++) {
- if (patch_feature_section(value, fcur)) {
+ if (patch_feature_section_mask(value, mask, fcur)) {
WARN_ON(1);
printk("Unable to patch feature section at %p - %p" \
" with %p - %p\n",
}
}
+void do_feature_fixups(unsigned long value, void *fixup_start, void *fixup_end)
+{
+ do_feature_fixups_mask(value, ~0, fixup_start, fixup_end);
+}
+
#ifdef CONFIG_PPC_BARRIER_NOSPEC
static bool is_fixup_addr_valid(void *dest, size_t size)
{
do_final_fixups();
}
+void __init update_mmu_feature_fixups(unsigned long mask)
+{
+ saved_mmu_features &= ~mask;
+ saved_mmu_features |= cur_cpu_spec->mmu_features & mask;
+
+ do_feature_fixups_mask(cur_cpu_spec->mmu_features, mask,
+ PTRRELOC(&__start___mmu_ftr_fixup),
+ PTRRELOC(&__stop___mmu_ftr_fixup));
+ mmu_feature_keys_init();
+}
+
void __init setup_feature_keys(void)
{
/*
#define check(x) \
if (!(x)) printk("feature-fixups: test failed at line %d\n", __LINE__);
+static int patch_feature_section(unsigned long value, struct fixup_entry *fcur)
+{
+ return patch_feature_section_mask(value, ~0, fcur);
+}
+
/* This must be after the text it fixes up, vmlinux.lds.S enforces that atm */
static struct fixup_entry fixup;
*
* Author: Anton Blanchard <anton@au.ibm.com>
*/
+#include <linux/export.h>
#include <asm/processor.h>
#include <asm/ppc_asm.h>
-#include <asm/export.h>
#include <asm/feature-fixups.h>
/* Note: This code relies on -mminimal-toc */
*
* Copyright (C) 1996 Paul Mackerras.
*/
+#include <linux/export.h>
#include <asm/processor.h>
#include <asm/errno.h>
#include <asm/ppc_asm.h>
-#include <asm/export.h>
#include <asm/kasan.h>
#ifndef CONFIG_KASAN
*
*/
+#include <linux/export.h>
#include <asm/ppc_asm.h>
-#include <asm/export.h>
.text
* Author: Anton Blanchard <anton@au.ibm.com>
* Copyright 2015 IBM Corporation.
*/
+#include <linux/export.h>
#include <asm/ppc_asm.h>
-#include <asm/export.h>
#include <asm/ppc-opcode.h>
#define off8 r6
/*
* Copyright (C) 2002 Paul Mackerras, IBM Corp.
*/
+#include <linux/export.h>
#include <asm/processor.h>
#include <asm/ppc_asm.h>
-#include <asm/export.h>
#include <asm/asm-compat.h>
#include <asm/feature-fixups.h>
#include <asm/kasan.h>
* Copy from a buffer to userspace, using the largest possible
* aligned accesses, up to sizeof(long).
*/
-static nokprobe_inline int __copy_mem_out(u8 *dest, unsigned long ea, int nb, struct pt_regs *regs)
+static __always_inline int __copy_mem_out(u8 *dest, unsigned long ea, int nb, struct pt_regs *regs)
{
int c;
}
#endif /* CONFIG_VSX */
-static int __emulate_dcbz(unsigned long ea)
+static __always_inline int __emulate_dcbz(unsigned long ea)
{
unsigned long i;
unsigned long size = l1_dcache_bytes();
*
* Copyright (C) 1996 Paul Mackerras.
*/
+#include <linux/export.h>
#include <asm/ppc_asm.h>
-#include <asm/export.h>
#include <asm/cache.h>
.text
*
*/
+#include <linux/export.h>
#include <asm/ppc_asm.h>
-#include <asm/export.h>
#include <asm/cache.h>
.text
* Author: Anton Blanchard <anton@au.ibm.com>
*/
+#include <linux/export.h>
#include <asm/ppc_asm.h>
#include <asm/linkage.h>
#include <asm/asm-offsets.h>
-#include <asm/export.h>
/**
* __arch_clear_user: - Zero a block of memory in user space, with less checking.
*
* Inspired from glibc implementation
*/
+#include <linux/export.h>
#include <asm/ppc_asm.h>
-#include <asm/export.h>
#include <asm/cache.h>
.text
* hash table, so this file is not used on them.)
*/
+#include <linux/export.h>
#include <linux/pgtable.h>
#include <linux/init.h>
#include <asm/reg.h>
#include <asm/ppc_asm.h>
#include <asm/thread_info.h>
#include <asm/asm-offsets.h>
-#include <asm/export.h>
#include <asm/feature-fixups.h>
#include <asm/code-patching-asm.h>
#include <asm/kup.h>
#include <asm/smp.h>
-struct static_key_false disable_kuap_key;
-EXPORT_SYMBOL(disable_kuap_key);
-
-void kuap_lock_all_ool(void)
-{
- kuap_lock_all();
-}
-EXPORT_SYMBOL(kuap_lock_all_ool);
-
-void kuap_unlock_all_ool(void)
-{
- kuap_unlock_all();
-}
-EXPORT_SYMBOL(kuap_unlock_all_ool);
-
void setup_kuap(bool disabled)
{
if (!disabled) {
- kuap_lock_all_ool();
+ update_user_segments(mfsr(0) | SR_KS);
+ isync(); /* Context sync required after mtsr() */
init_mm.context.sr0 |= SR_KS;
current->thread.sr0 |= SR_KS;
}
return;
if (disabled)
- static_branch_enable(&disable_kuap_key);
+ cur_cpu_spec->mmu_features &= ~MMU_FTR_KUAP;
else
pr_info("Activating Kernel Userspace Access Protection\n");
}
mm->context.id = __init_new_context();
mm->context.sr0 = CTX_TO_VSID(mm->context.id, 0);
- if (!kuep_is_disabled())
+ if (IS_ENABLED(CONFIG_PPC_KUEP))
mm->context.sr0 |= SR_NX;
if (!kuap_is_disabled())
mm->context.sr0 |= SR_KS;
#include <linux/memremap.h>
#include <linux/pkeys.h>
#include <linux/debugfs.h>
+#include <linux/proc_fs.h>
#include <misc/cxl-base.h>
#include <asm/pgalloc.h>
if (smp_processor_id() == boot_cpuid) {
pr_info("Activating Kernel Userspace Access Prevention\n");
- cur_cpu_spec->mmu_features |= MMU_FTR_BOOK3S_KUAP;
+ cur_cpu_spec->mmu_features |= MMU_FTR_KUAP;
}
/*
#include <mm/mmu_decl.h>
unsigned int mmu_base_pid;
-unsigned long radix_mem_block_size __ro_after_init;
static __ref void *early_alloc_pgtable(unsigned long size, int nid,
unsigned long region_start, unsigned long region_end)
bool prev_exec, exec = false;
pgprot_t prot;
int psize;
- unsigned long max_mapping_size = radix_mem_block_size;
+ unsigned long max_mapping_size = memory_block_size;
if (debug_pagealloc_enabled_or_kfence())
max_mapping_size = PAGE_SIZE;
return 1;
}
-#ifdef CONFIG_MEMORY_HOTPLUG
-static int __init probe_memory_block_size(unsigned long node, const char *uname, int
- depth, void *data)
-{
- unsigned long *mem_block_size = (unsigned long *)data;
- const __be32 *prop;
- int len;
-
- if (depth != 1)
- return 0;
-
- if (strcmp(uname, "ibm,dynamic-reconfiguration-memory"))
- return 0;
-
- prop = of_get_flat_dt_prop(node, "ibm,lmb-size", &len);
-
- if (!prop || len < dt_root_size_cells * sizeof(__be32))
- /*
- * Nothing in the device tree
- */
- *mem_block_size = MIN_MEMORY_BLOCK_SIZE;
- else
- *mem_block_size = of_read_number(prop, dt_root_size_cells);
- return 1;
-}
-
-static unsigned long __init radix_memory_block_size(void)
-{
- unsigned long mem_block_size = MIN_MEMORY_BLOCK_SIZE;
-
- /*
- * OPAL firmware feature is set by now. Hence we are ok
- * to test OPAL feature.
- */
- if (firmware_has_feature(FW_FEATURE_OPAL))
- mem_block_size = 1UL * 1024 * 1024 * 1024;
- else
- of_scan_flat_dt(probe_memory_block_size, &mem_block_size);
-
- return mem_block_size;
-}
-
-#else /* CONFIG_MEMORY_HOTPLUG */
-
-static unsigned long __init radix_memory_block_size(void)
-{
- return 1UL * 1024 * 1024 * 1024;
-}
-
-#endif /* CONFIG_MEMORY_HOTPLUG */
-
-
void __init radix__early_init_devtree(void)
{
int rc;
mmu_psize_defs[MMU_PAGE_64K].h_rpt_pgsize =
psize_to_rpti_pgsize(MMU_PAGE_64K);
}
-
- /*
- * Max mapping size used when mapping pages. We don't use
- * ppc_md.memory_block_size() here because this get called
- * early and we don't have machine probe called yet. Also
- * the pseries implementation only check for ibm,lmb-size.
- * All hypervisor supporting radix do expose that device
- * tree node.
- */
- radix_mem_block_size = radix_memory_block_size();
return;
}
trace_tlbie(0, 0, rb, rs, ric, prs, r);
}
-static __always_inline void __tlbie_pid_lpid(unsigned long pid,
- unsigned long lpid,
- unsigned long ric)
-{
- unsigned long rb, rs, prs, r;
-
- rb = PPC_BIT(53); /* IS = 1 */
- rs = (pid << PPC_BITLSHIFT(31)) | (lpid & ~(PPC_BITMASK(0, 31)));
- prs = 1; /* process scoped */
- r = 1; /* radix format */
-
- asm volatile(PPC_TLBIE_5(%0, %4, %3, %2, %1)
- : : "r"(rb), "i"(r), "i"(prs), "i"(ric), "r"(rs) : "memory");
- trace_tlbie(0, 0, rb, rs, ric, prs, r);
-}
static __always_inline void __tlbie_lpid(unsigned long lpid, unsigned long ric)
{
unsigned long rb,rs,prs,r;
trace_tlbie(0, 0, rb, rs, ric, prs, r);
}
-static __always_inline void __tlbie_va_lpid(unsigned long va, unsigned long pid,
- unsigned long lpid,
- unsigned long ap, unsigned long ric)
-{
- unsigned long rb, rs, prs, r;
-
- rb = va & ~(PPC_BITMASK(52, 63));
- rb |= ap << PPC_BITLSHIFT(58);
- rs = (pid << PPC_BITLSHIFT(31)) | (lpid & ~(PPC_BITMASK(0, 31)));
- prs = 1; /* process scoped */
- r = 1; /* radix format */
-
- asm volatile(PPC_TLBIE_5(%0, %4, %3, %2, %1)
- : : "r"(rb), "i"(r), "i"(prs), "i"(ric), "r"(rs) : "memory");
- trace_tlbie(0, 0, rb, rs, ric, prs, r);
-}
-
static __always_inline void __tlbie_lpid_va(unsigned long va, unsigned long lpid,
unsigned long ap, unsigned long ric)
{
}
}
-static inline void fixup_tlbie_va_range_lpid(unsigned long va,
- unsigned long pid,
- unsigned long lpid,
- unsigned long ap)
-{
- if (cpu_has_feature(CPU_FTR_P9_TLBIE_ERAT_BUG)) {
- asm volatile("ptesync" : : : "memory");
- __tlbie_pid_lpid(0, lpid, RIC_FLUSH_TLB);
- }
-
- if (cpu_has_feature(CPU_FTR_P9_TLBIE_STQ_BUG)) {
- asm volatile("ptesync" : : : "memory");
- __tlbie_va_lpid(va, pid, lpid, ap, RIC_FLUSH_TLB);
- }
-}
-
static inline void fixup_tlbie_pid(unsigned long pid)
{
/*
}
}
-static inline void fixup_tlbie_pid_lpid(unsigned long pid, unsigned long lpid)
-{
- /*
- * We can use any address for the invalidation, pick one which is
- * probably unused as an optimisation.
- */
- unsigned long va = ((1UL << 52) - 1);
-
- if (cpu_has_feature(CPU_FTR_P9_TLBIE_ERAT_BUG)) {
- asm volatile("ptesync" : : : "memory");
- __tlbie_pid_lpid(0, lpid, RIC_FLUSH_TLB);
- }
-
- if (cpu_has_feature(CPU_FTR_P9_TLBIE_STQ_BUG)) {
- asm volatile("ptesync" : : : "memory");
- __tlbie_va_lpid(va, pid, lpid, mmu_get_ap(MMU_PAGE_64K),
- RIC_FLUSH_TLB);
- }
-}
-
static inline void fixup_tlbie_lpid_va(unsigned long va, unsigned long lpid,
unsigned long ap)
{
asm volatile("eieio; tlbsync; ptesync": : :"memory");
}
-static inline void _tlbie_pid_lpid(unsigned long pid, unsigned long lpid,
- unsigned long ric)
-{
- asm volatile("ptesync" : : : "memory");
-
- /*
- * Workaround the fact that the "ric" argument to __tlbie_pid
- * must be a compile-time contraint to match the "i" constraint
- * in the asm statement.
- */
- switch (ric) {
- case RIC_FLUSH_TLB:
- __tlbie_pid_lpid(pid, lpid, RIC_FLUSH_TLB);
- fixup_tlbie_pid_lpid(pid, lpid);
- break;
- case RIC_FLUSH_PWC:
- __tlbie_pid_lpid(pid, lpid, RIC_FLUSH_PWC);
- break;
- case RIC_FLUSH_ALL:
- default:
- __tlbie_pid_lpid(pid, lpid, RIC_FLUSH_ALL);
- fixup_tlbie_pid_lpid(pid, lpid);
- }
- asm volatile("eieio; tlbsync; ptesync" : : : "memory");
-}
struct tlbiel_pid {
unsigned long pid;
unsigned long ric;
fixup_tlbie_va_range(addr - page_size, pid, ap);
}
-static inline void __tlbie_va_range_lpid(unsigned long start, unsigned long end,
- unsigned long pid, unsigned long lpid,
- unsigned long page_size,
- unsigned long psize)
-{
- unsigned long addr;
- unsigned long ap = mmu_get_ap(psize);
-
- for (addr = start; addr < end; addr += page_size)
- __tlbie_va_lpid(addr, pid, lpid, ap, RIC_FLUSH_TLB);
-
- fixup_tlbie_va_range_lpid(addr - page_size, pid, lpid, ap);
-}
-
static __always_inline void _tlbie_va(unsigned long va, unsigned long pid,
unsigned long psize, unsigned long ric)
{
asm volatile("eieio; tlbsync; ptesync": : :"memory");
}
-static inline void _tlbie_va_range_lpid(unsigned long start, unsigned long end,
- unsigned long pid, unsigned long lpid,
- unsigned long page_size,
- unsigned long psize, bool also_pwc)
-{
- asm volatile("ptesync" : : : "memory");
- if (also_pwc)
- __tlbie_pid_lpid(pid, lpid, RIC_FLUSH_PWC);
- __tlbie_va_range_lpid(start, end, pid, lpid, page_size, psize);
- asm volatile("eieio; tlbsync; ptesync" : : : "memory");
-}
-
static inline void _tlbiel_va_range_multicast(struct mm_struct *mm,
unsigned long start, unsigned long end,
unsigned long pid, unsigned long page_size,
* that's what the caller expects.
*/
if (cpumask_test_cpu(cpu, mm_cpumask(mm))) {
- atomic_dec(&mm->context.active_cpus);
+ dec_mm_active_cpus(mm);
cpumask_clear_cpu(cpu, mm_cpumask(mm));
always_flush = true;
}
* See the comment for radix in arch_exit_mmap().
*/
if (tlb->fullmm) {
- __flush_all_mm(mm, true);
+ if (IS_ENABLED(CONFIG_MMU_LAZY_TLB_SHOOTDOWN)) {
+ /*
+ * Shootdown based lazy tlb mm refcounting means we
+ * have to IPI everyone in the mm_cpumask anyway soon
+ * when the mm goes away, so might as well do it as
+ * part of the final flush now.
+ *
+ * If lazy shootdown was improved to reduce IPIs (e.g.,
+ * by batching), then it may end up being better to use
+ * tlbies here instead.
+ */
+ preempt_disable();
+
+ smp_mb(); /* see radix__flush_tlb_mm */
+ exit_flush_lazy_tlbs(mm);
+ _tlbiel_pid(mm->context.id, RIC_FLUSH_ALL);
+
+ /*
+ * It should not be possible to have coprocessors still
+ * attached here.
+ */
+ if (WARN_ON_ONCE(atomic_read(&mm->context.copros) > 0))
+ __flush_all_mm(mm, true);
+
+ preempt_enable();
+ } else {
+ __flush_all_mm(mm, true);
+ }
+
} else if ( (psize = radix_get_mmu_psize(page_size)) == -1) {
if (!tlb->freed_tables)
radix__flush_tlb_mm(mm);
}
#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
+static __always_inline void __tlbie_pid_lpid(unsigned long pid,
+ unsigned long lpid,
+ unsigned long ric)
+{
+ unsigned long rb, rs, prs, r;
+
+ rb = PPC_BIT(53); /* IS = 1 */
+ rs = (pid << PPC_BITLSHIFT(31)) | (lpid & ~(PPC_BITMASK(0, 31)));
+ prs = 1; /* process scoped */
+ r = 1; /* radix format */
+
+ asm volatile(PPC_TLBIE_5(%0, %4, %3, %2, %1)
+ : : "r"(rb), "i"(r), "i"(prs), "i"(ric), "r"(rs) : "memory");
+ trace_tlbie(0, 0, rb, rs, ric, prs, r);
+}
+
+static __always_inline void __tlbie_va_lpid(unsigned long va, unsigned long pid,
+ unsigned long lpid,
+ unsigned long ap, unsigned long ric)
+{
+ unsigned long rb, rs, prs, r;
+
+ rb = va & ~(PPC_BITMASK(52, 63));
+ rb |= ap << PPC_BITLSHIFT(58);
+ rs = (pid << PPC_BITLSHIFT(31)) | (lpid & ~(PPC_BITMASK(0, 31)));
+ prs = 1; /* process scoped */
+ r = 1; /* radix format */
+
+ asm volatile(PPC_TLBIE_5(%0, %4, %3, %2, %1)
+ : : "r"(rb), "i"(r), "i"(prs), "i"(ric), "r"(rs) : "memory");
+ trace_tlbie(0, 0, rb, rs, ric, prs, r);
+}
+
+static inline void fixup_tlbie_pid_lpid(unsigned long pid, unsigned long lpid)
+{
+ /*
+ * We can use any address for the invalidation, pick one which is
+ * probably unused as an optimisation.
+ */
+ unsigned long va = ((1UL << 52) - 1);
+
+ if (cpu_has_feature(CPU_FTR_P9_TLBIE_ERAT_BUG)) {
+ asm volatile("ptesync" : : : "memory");
+ __tlbie_pid_lpid(0, lpid, RIC_FLUSH_TLB);
+ }
+
+ if (cpu_has_feature(CPU_FTR_P9_TLBIE_STQ_BUG)) {
+ asm volatile("ptesync" : : : "memory");
+ __tlbie_va_lpid(va, pid, lpid, mmu_get_ap(MMU_PAGE_64K),
+ RIC_FLUSH_TLB);
+ }
+}
+
+static inline void _tlbie_pid_lpid(unsigned long pid, unsigned long lpid,
+ unsigned long ric)
+{
+ asm volatile("ptesync" : : : "memory");
+
+ /*
+ * Workaround the fact that the "ric" argument to __tlbie_pid
+ * must be a compile-time contraint to match the "i" constraint
+ * in the asm statement.
+ */
+ switch (ric) {
+ case RIC_FLUSH_TLB:
+ __tlbie_pid_lpid(pid, lpid, RIC_FLUSH_TLB);
+ fixup_tlbie_pid_lpid(pid, lpid);
+ break;
+ case RIC_FLUSH_PWC:
+ __tlbie_pid_lpid(pid, lpid, RIC_FLUSH_PWC);
+ break;
+ case RIC_FLUSH_ALL:
+ default:
+ __tlbie_pid_lpid(pid, lpid, RIC_FLUSH_ALL);
+ fixup_tlbie_pid_lpid(pid, lpid);
+ }
+ asm volatile("eieio; tlbsync; ptesync" : : : "memory");
+}
+
+static inline void fixup_tlbie_va_range_lpid(unsigned long va,
+ unsigned long pid,
+ unsigned long lpid,
+ unsigned long ap)
+{
+ if (cpu_has_feature(CPU_FTR_P9_TLBIE_ERAT_BUG)) {
+ asm volatile("ptesync" : : : "memory");
+ __tlbie_pid_lpid(0, lpid, RIC_FLUSH_TLB);
+ }
+
+ if (cpu_has_feature(CPU_FTR_P9_TLBIE_STQ_BUG)) {
+ asm volatile("ptesync" : : : "memory");
+ __tlbie_va_lpid(va, pid, lpid, ap, RIC_FLUSH_TLB);
+ }
+}
+
+static inline void __tlbie_va_range_lpid(unsigned long start, unsigned long end,
+ unsigned long pid, unsigned long lpid,
+ unsigned long page_size,
+ unsigned long psize)
+{
+ unsigned long addr;
+ unsigned long ap = mmu_get_ap(psize);
+
+ for (addr = start; addr < end; addr += page_size)
+ __tlbie_va_lpid(addr, pid, lpid, ap, RIC_FLUSH_TLB);
+
+ fixup_tlbie_va_range_lpid(addr - page_size, pid, lpid, ap);
+}
+
+static inline void _tlbie_va_range_lpid(unsigned long start, unsigned long end,
+ unsigned long pid, unsigned long lpid,
+ unsigned long page_size,
+ unsigned long psize, bool also_pwc)
+{
+ asm volatile("ptesync" : : : "memory");
+ if (also_pwc)
+ __tlbie_pid_lpid(pid, lpid, RIC_FLUSH_PWC);
+ __tlbie_va_range_lpid(start, end, pid, lpid, page_size, psize);
+ asm volatile("eieio; tlbsync; ptesync" : : : "memory");
+}
+
/*
* Performs process-scoped invalidations for a given LPID
* as part of H_RPT_INVALIDATE hcall.
#include <asm/mmu.h>
#include <asm/mmu_context.h>
#include <asm/paca.h>
+#include <asm/lppaca.h>
#include <asm/ppc-opcode.h>
#include <asm/cputable.h>
#include <asm/cacheflush.h>
setup_kup();
+ update_mmu_feature_fixups(MMU_FTR_KUAP);
+
/* Shortly after that, the entire linear mapping will be available */
memblock_set_current_limit(lowmem_end_addr);
}
#include <linux/of_fdt.h>
#include <linux/libfdt.h>
#include <linux/memremap.h>
+#include <linux/memory.h>
#include <asm/pgalloc.h>
#include <asm/page.h>
return 1;
}
+/*
+ * Outside hotplug the kernel uses this value to map the kernel direct map
+ * with radix. To be compatible with older kernels, let's keep this value
+ * as 16M which is also SECTION_SIZE with SPARSEMEM. We can ideally map
+ * things with 1GB size in the case where we don't support hotplug.
+ */
+#ifndef CONFIG_MEMORY_HOTPLUG
+#define DEFAULT_MEMORY_BLOCK_SIZE SZ_16M
+#else
+#define DEFAULT_MEMORY_BLOCK_SIZE MIN_MEMORY_BLOCK_SIZE
+#endif
+
+static void update_memory_block_size(unsigned long *block_size, unsigned long mem_size)
+{
+ unsigned long min_memory_block_size = DEFAULT_MEMORY_BLOCK_SIZE;
+
+ for (; *block_size > min_memory_block_size; *block_size >>= 2) {
+ if ((mem_size & *block_size) == 0)
+ break;
+ }
+}
+
+static int __init probe_memory_block_size(unsigned long node, const char *uname, int
+ depth, void *data)
+{
+ const char *type;
+ unsigned long *block_size = (unsigned long *)data;
+ const __be32 *reg, *endp;
+ int l;
+
+ if (depth != 1)
+ return 0;
+ /*
+ * If we have dynamic-reconfiguration-memory node, use the
+ * lmb value.
+ */
+ if (strcmp(uname, "ibm,dynamic-reconfiguration-memory") == 0) {
+
+ const __be32 *prop;
+
+ prop = of_get_flat_dt_prop(node, "ibm,lmb-size", &l);
+
+ if (!prop || l < dt_root_size_cells * sizeof(__be32))
+ /*
+ * Nothing in the device tree
+ */
+ *block_size = DEFAULT_MEMORY_BLOCK_SIZE;
+ else
+ *block_size = of_read_number(prop, dt_root_size_cells);
+ /*
+ * We have found the final value. Don't probe further.
+ */
+ return 1;
+ }
+ /*
+ * Find all the device tree nodes of memory type and make sure
+ * the area can be mapped using the memory block size value
+ * we end up using. We start with 1G value and keep reducing
+ * it such that we can map the entire area using memory_block_size.
+ * This will be used on powernv and older pseries that don't
+ * have ibm,lmb-size node.
+ * For ex: with P5 we can end up with
+ * memory@0 -> 128MB
+ * memory@128M -> 64M
+ * This will end up using 64MB memory block size value.
+ */
+ type = of_get_flat_dt_prop(node, "device_type", NULL);
+ if (type == NULL || strcmp(type, "memory") != 0)
+ return 0;
+
+ reg = of_get_flat_dt_prop(node, "linux,usable-memory", &l);
+ if (!reg)
+ reg = of_get_flat_dt_prop(node, "reg", &l);
+ if (!reg)
+ return 0;
+
+ endp = reg + (l / sizeof(__be32));
+ while ((endp - reg) >= (dt_root_addr_cells + dt_root_size_cells)) {
+ const char *compatible;
+ u64 size;
+
+ dt_mem_next_cell(dt_root_addr_cells, ®);
+ size = dt_mem_next_cell(dt_root_size_cells, ®);
+
+ if (size) {
+ update_memory_block_size(block_size, size);
+ continue;
+ }
+ /*
+ * ibm,coherent-device-memory with linux,usable-memory = 0
+ * Force 256MiB block size. Work around for GPUs on P9 PowerNV
+ * linux,usable-memory == 0 implies driver managed memory and
+ * we can't use large memory block size due to hotplug/unplug
+ * limitations.
+ */
+ compatible = of_get_flat_dt_prop(node, "compatible", NULL);
+ if (compatible && !strcmp(compatible, "ibm,coherent-device-memory")) {
+ if (*block_size > SZ_256M)
+ *block_size = SZ_256M;
+ /*
+ * We keep 256M as the upper limit with GPU present.
+ */
+ return 0;
+ }
+ }
+ /* continue looking for other memory device types */
+ return 0;
+}
+
+/*
+ * start with 1G memory block size. Early init will
+ * fix this with correct value.
+ */
+unsigned long memory_block_size __ro_after_init = 1UL << 30;
+static void __init early_init_memory_block_size(void)
+{
+ /*
+ * We need to do memory_block_size probe early so that
+ * radix__early_init_mmu() can use this as limit for
+ * mapping page size.
+ */
+ of_scan_flat_dt(probe_memory_block_size, &memory_block_size);
+}
+
void __init mmu_early_init_devtree(void)
{
bool hvmode = !!(mfmsr() & MSR_HV);
if (!hvmode)
early_check_vec5();
+ early_init_memory_block_size();
+
if (early_radix_enabled()) {
radix__early_init_devtree();
void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next,
struct task_struct *tsk)
{
+ int cpu = smp_processor_id();
bool new_on_cpu = false;
/* Mark this context has been used on the new CPU */
- if (!cpumask_test_cpu(smp_processor_id(), mm_cpumask(next))) {
- cpumask_set_cpu(smp_processor_id(), mm_cpumask(next));
+ if (!cpumask_test_cpu(cpu, mm_cpumask(next))) {
+ VM_WARN_ON_ONCE(next == &init_mm);
+ cpumask_set_cpu(cpu, mm_cpumask(next));
inc_mm_active_cpus(next);
/*
* sub architectures. Out of line for now
*/
switch_mmu_context(prev, next, tsk);
+
+ VM_WARN_ON_ONCE(!cpumask_test_cpu(cpu, mm_cpumask(prev)));
}
#ifndef CONFIG_PPC_BOOK3S_64
void MMU_init_hw_patch(void);
unsigned long mmu_mapin_ram(unsigned long base, unsigned long top);
#endif
+void mmu_init_secondary(int cpu);
#ifdef CONFIG_PPC_E500
extern unsigned long map_mem_in_cams(unsigned long ram, int max_cam_idx,
#include <linux/export.h>
#include <linux/init.h>
-#include <linux/jump_label.h>
#include <linux/printk.h>
#include <linux/smp.h>
#include <asm/smp.h>
#ifdef CONFIG_PPC_KUAP
-struct static_key_false disable_kuap_key;
-EXPORT_SYMBOL(disable_kuap_key);
-
void setup_kuap(bool disabled)
{
if (disabled) {
if (IS_ENABLED(CONFIG_40x))
disable_kuep = true;
if (smp_processor_id() == boot_cpuid)
- static_branch_enable(&disable_kuap_key);
+ cur_cpu_spec->mmu_features &= ~MMU_FTR_KUAP;
return;
}
pr_info("Activating Kernel Userspace Access Protection\n");
- __prevent_user_access(KUAP_READ_WRITE);
+ prevent_user_access(KUAP_READ_WRITE);
}
#endif
#endif /* CONFIG_SMP */
-#ifdef CONFIG_PPC_47x
-void __init early_init_mmu_47x(void)
-{
-#ifdef CONFIG_SMP
- unsigned long root = of_get_flat_dt_root();
- if (of_get_flat_dt_prop(root, "cooperative-partition", NULL))
- mmu_clear_feature(MMU_FTR_USE_TLBIVAX_BCAST);
-#endif /* CONFIG_SMP */
-}
-#endif /* CONFIG_PPC_47x */
-
/*
* Flush kernel TLB entries in the given range
*/
#else /* ! CONFIG_PPC64 */
void __init early_init_mmu(void)
{
-#ifdef CONFIG_PPC_47x
- early_init_mmu_47x();
-#endif
+ unsigned long root = of_get_flat_dt_root();
+
+ if (IS_ENABLED(CONFIG_PPC_47x) && IS_ENABLED(CONFIG_SMP) &&
+ of_get_flat_dt_prop(root, "cooperative-partition", NULL))
+ mmu_clear_feature(MMU_FTR_USE_TLBIVAX_BCAST);
}
#endif /* CONFIG_PPC64 */
#include <asm/hvcall.h>
#include <asm/setup.h>
#include <asm/vdso.h>
+#include <asm/vphn.h>
#include <asm/drmem.h>
static int numa_enabled = 1;
struct cpu_hw_events *cpuhw = this_cpu_ptr(&cpu_hw_events);
struct perf_event *event;
unsigned long val;
- int found = 0;
for (i = 0; i < ppmu->n_counter; ++i) {
event = cpuhw->event[i];
if ((int)val < 0) {
if (event) {
/* event has overflowed */
- found = 1;
record_and_restart(event, val, regs);
} else {
/*
isync();
}
-void hw_perf_event_setup(int cpu)
+static int fsl_emb_pmu_prepare_cpu(unsigned int cpu)
{
struct cpu_hw_events *cpuhw = &per_cpu(cpu_hw_events, cpu);
memset(cpuhw, 0, sizeof(*cpuhw));
+
+ return 0;
}
int register_fsl_emb_pmu(struct fsl_emb_pmu *pmu)
pmu->name);
perf_pmu_register(&fsl_emb_pmu, "cpu", PERF_TYPE_RAW);
+ cpuhp_setup_state(CPUHP_PERF_POWER, "perf/powerpc:prepare",
+ fsl_emb_pmu_prepare_cpu, NULL);
return 0;
}
return cpumap_print_to_pagebuf(true, buf, &hv_gpci_cpumask);
}
+/* Interface attribute array index to store system information */
+#define INTERFACE_PROCESSOR_BUS_TOPOLOGY_ATTR 6
+#define INTERFACE_PROCESSOR_CONFIG_ATTR 7
+#define INTERFACE_AFFINITY_DOMAIN_VIA_VP_ATTR 8
+#define INTERFACE_AFFINITY_DOMAIN_VIA_DOM_ATTR 9
+#define INTERFACE_AFFINITY_DOMAIN_VIA_PAR_ATTR 10
+#define INTERFACE_NULL_ATTR 11
+
+/* Counter request value to retrieve system information */
+enum {
+ PROCESSOR_BUS_TOPOLOGY,
+ PROCESSOR_CONFIG,
+ AFFINITY_DOMAIN_VIA_VP, /* affinity domain via virtual processor */
+ AFFINITY_DOMAIN_VIA_DOM, /* affinity domain via domain */
+ AFFINITY_DOMAIN_VIA_PAR, /* affinity domain via partition */
+};
+
+static int sysinfo_counter_request[] = {
+ [PROCESSOR_BUS_TOPOLOGY] = 0xD0,
+ [PROCESSOR_CONFIG] = 0x90,
+ [AFFINITY_DOMAIN_VIA_VP] = 0xA0,
+ [AFFINITY_DOMAIN_VIA_DOM] = 0xB0,
+ [AFFINITY_DOMAIN_VIA_PAR] = 0xB1,
+};
+
+static DEFINE_PER_CPU(char, hv_gpci_reqb[HGPCI_REQ_BUFFER_SIZE]) __aligned(sizeof(uint64_t));
+
+static unsigned long systeminfo_gpci_request(u32 req, u32 starting_index,
+ u16 secondary_index, char *buf,
+ size_t *n, struct hv_gpci_request_buffer *arg)
+{
+ unsigned long ret;
+ size_t i, j;
+
+ arg->params.counter_request = cpu_to_be32(req);
+ arg->params.starting_index = cpu_to_be32(starting_index);
+ arg->params.secondary_index = cpu_to_be16(secondary_index);
+
+ ret = plpar_hcall_norets(H_GET_PERF_COUNTER_INFO,
+ virt_to_phys(arg), HGPCI_REQ_BUFFER_SIZE);
+
+ /*
+ * ret value as 'H_PARAMETER' corresponds to 'GEN_BUF_TOO_SMALL',
+ * which means that the current buffer size cannot accommodate
+ * all the information and a partial buffer returned.
+ * hcall fails incase of ret value other than H_SUCCESS or H_PARAMETER.
+ *
+ * ret value as H_AUTHORITY implies that partition is not permitted to retrieve
+ * performance information, and required to set
+ * "Enable Performance Information Collection" option.
+ */
+ if (ret == H_AUTHORITY)
+ return -EPERM;
+
+ /*
+ * hcall can fail with other possible ret value like H_PRIVILEGE/H_HARDWARE
+ * because of invalid buffer-length/address or due to some hardware
+ * error.
+ */
+ if (ret && (ret != H_PARAMETER))
+ return -EIO;
+
+ /*
+ * hcall H_GET_PERF_COUNTER_INFO populates the 'returned_values'
+ * to show the total number of counter_value array elements
+ * returned via hcall.
+ * hcall also populates 'cv_element_size' corresponds to individual
+ * counter_value array element size. Below loop go through all
+ * counter_value array elements as per their size and add it to
+ * the output buffer.
+ */
+ for (i = 0; i < be16_to_cpu(arg->params.returned_values); i++) {
+ j = i * be16_to_cpu(arg->params.cv_element_size);
+
+ for (; j < (i + 1) * be16_to_cpu(arg->params.cv_element_size); j++)
+ *n += sprintf(buf + *n, "%02x", (u8)arg->bytes[j]);
+ *n += sprintf(buf + *n, "\n");
+ }
+
+ if (*n >= PAGE_SIZE) {
+ pr_info("System information exceeds PAGE_SIZE\n");
+ return -EFBIG;
+ }
+
+ return ret;
+}
+
+static ssize_t processor_bus_topology_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct hv_gpci_request_buffer *arg;
+ unsigned long ret;
+ size_t n = 0;
+
+ arg = (void *)get_cpu_var(hv_gpci_reqb);
+ memset(arg, 0, HGPCI_REQ_BUFFER_SIZE);
+
+ /*
+ * Pass the counter request value 0xD0 corresponds to request
+ * type 'Processor_bus_topology', to retrieve
+ * the system topology information.
+ * starting_index value implies the starting hardware
+ * chip id.
+ */
+ ret = systeminfo_gpci_request(sysinfo_counter_request[PROCESSOR_BUS_TOPOLOGY],
+ 0, 0, buf, &n, arg);
+
+ if (!ret)
+ return n;
+
+ if (ret != H_PARAMETER)
+ goto out;
+
+ /*
+ * ret value as 'H_PARAMETER' corresponds to 'GEN_BUF_TOO_SMALL', which
+ * implies that buffer can't accommodate all information, and a partial buffer
+ * returned. To handle that, we need to make subsequent requests
+ * with next starting index to retrieve additional (missing) data.
+ * Below loop do subsequent hcalls with next starting index and add it
+ * to buffer util we get all the information.
+ */
+ while (ret == H_PARAMETER) {
+ int returned_values = be16_to_cpu(arg->params.returned_values);
+ int elementsize = be16_to_cpu(arg->params.cv_element_size);
+ int last_element = (returned_values - 1) * elementsize;
+
+ /*
+ * Since the starting index value is part of counter_value
+ * buffer elements, use the starting index value in the last
+ * element and add 1 to make subsequent hcalls.
+ */
+ u32 starting_index = arg->bytes[last_element + 3] +
+ (arg->bytes[last_element + 2] << 8) +
+ (arg->bytes[last_element + 1] << 16) +
+ (arg->bytes[last_element] << 24) + 1;
+
+ memset(arg, 0, HGPCI_REQ_BUFFER_SIZE);
+
+ ret = systeminfo_gpci_request(sysinfo_counter_request[PROCESSOR_BUS_TOPOLOGY],
+ starting_index, 0, buf, &n, arg);
+
+ if (!ret)
+ return n;
+
+ if (ret != H_PARAMETER)
+ goto out;
+ }
+
+ return n;
+
+out:
+ put_cpu_var(hv_gpci_reqb);
+ return ret;
+}
+
+static ssize_t processor_config_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct hv_gpci_request_buffer *arg;
+ unsigned long ret;
+ size_t n = 0;
+
+ arg = (void *)get_cpu_var(hv_gpci_reqb);
+ memset(arg, 0, HGPCI_REQ_BUFFER_SIZE);
+
+ /*
+ * Pass the counter request value 0x90 corresponds to request
+ * type 'Processor_config', to retrieve
+ * the system processor information.
+ * starting_index value implies the starting hardware
+ * processor index.
+ */
+ ret = systeminfo_gpci_request(sysinfo_counter_request[PROCESSOR_CONFIG],
+ 0, 0, buf, &n, arg);
+
+ if (!ret)
+ return n;
+
+ if (ret != H_PARAMETER)
+ goto out;
+
+ /*
+ * ret value as 'H_PARAMETER' corresponds to 'GEN_BUF_TOO_SMALL', which
+ * implies that buffer can't accommodate all information, and a partial buffer
+ * returned. To handle that, we need to take subsequent requests
+ * with next starting index to retrieve additional (missing) data.
+ * Below loop do subsequent hcalls with next starting index and add it
+ * to buffer util we get all the information.
+ */
+ while (ret == H_PARAMETER) {
+ int returned_values = be16_to_cpu(arg->params.returned_values);
+ int elementsize = be16_to_cpu(arg->params.cv_element_size);
+ int last_element = (returned_values - 1) * elementsize;
+
+ /*
+ * Since the starting index is part of counter_value
+ * buffer elements, use the starting index value in the last
+ * element and add 1 to subsequent hcalls.
+ */
+ u32 starting_index = arg->bytes[last_element + 3] +
+ (arg->bytes[last_element + 2] << 8) +
+ (arg->bytes[last_element + 1] << 16) +
+ (arg->bytes[last_element] << 24) + 1;
+
+ memset(arg, 0, HGPCI_REQ_BUFFER_SIZE);
+
+ ret = systeminfo_gpci_request(sysinfo_counter_request[PROCESSOR_CONFIG],
+ starting_index, 0, buf, &n, arg);
+
+ if (!ret)
+ return n;
+
+ if (ret != H_PARAMETER)
+ goto out;
+ }
+
+ return n;
+
+out:
+ put_cpu_var(hv_gpci_reqb);
+ return ret;
+}
+
+static ssize_t affinity_domain_via_virtual_processor_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct hv_gpci_request_buffer *arg;
+ unsigned long ret;
+ size_t n = 0;
+
+ arg = (void *)get_cpu_var(hv_gpci_reqb);
+ memset(arg, 0, HGPCI_REQ_BUFFER_SIZE);
+
+ /*
+ * Pass the counter request 0xA0 corresponds to request
+ * type 'Affinity_domain_information_by_virutal_processor',
+ * to retrieve the system affinity domain information.
+ * starting_index value refers to the starting hardware
+ * processor index.
+ */
+ ret = systeminfo_gpci_request(sysinfo_counter_request[AFFINITY_DOMAIN_VIA_VP],
+ 0, 0, buf, &n, arg);
+
+ if (!ret)
+ return n;
+
+ if (ret != H_PARAMETER)
+ goto out;
+
+ /*
+ * ret value as 'H_PARAMETER' corresponds to 'GEN_BUF_TOO_SMALL', which
+ * implies that buffer can't accommodate all information, and a partial buffer
+ * returned. To handle that, we need to take subsequent requests
+ * with next secondary index to retrieve additional (missing) data.
+ * Below loop do subsequent hcalls with next secondary index and add it
+ * to buffer util we get all the information.
+ */
+ while (ret == H_PARAMETER) {
+ int returned_values = be16_to_cpu(arg->params.returned_values);
+ int elementsize = be16_to_cpu(arg->params.cv_element_size);
+ int last_element = (returned_values - 1) * elementsize;
+
+ /*
+ * Since the starting index and secondary index type is part of the
+ * counter_value buffer elements, use the starting index value in the
+ * last array element as subsequent starting index, and use secondary index
+ * value in the last array element plus 1 as subsequent secondary index.
+ * For counter request '0xA0', starting index points to partition id
+ * and secondary index points to corresponding virtual processor index.
+ */
+ u32 starting_index = arg->bytes[last_element + 1] + (arg->bytes[last_element] << 8);
+ u16 secondary_index = arg->bytes[last_element + 3] +
+ (arg->bytes[last_element + 2] << 8) + 1;
+
+ memset(arg, 0, HGPCI_REQ_BUFFER_SIZE);
+
+ ret = systeminfo_gpci_request(sysinfo_counter_request[AFFINITY_DOMAIN_VIA_VP],
+ starting_index, secondary_index, buf, &n, arg);
+
+ if (!ret)
+ return n;
+
+ if (ret != H_PARAMETER)
+ goto out;
+ }
+
+ return n;
+
+out:
+ put_cpu_var(hv_gpci_reqb);
+ return ret;
+}
+
+static ssize_t affinity_domain_via_domain_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct hv_gpci_request_buffer *arg;
+ unsigned long ret;
+ size_t n = 0;
+
+ arg = (void *)get_cpu_var(hv_gpci_reqb);
+ memset(arg, 0, HGPCI_REQ_BUFFER_SIZE);
+
+ /*
+ * Pass the counter request 0xB0 corresponds to request
+ * type 'Affinity_domain_information_by_domain',
+ * to retrieve the system affinity domain information.
+ * starting_index value refers to the starting hardware
+ * processor index.
+ */
+ ret = systeminfo_gpci_request(sysinfo_counter_request[AFFINITY_DOMAIN_VIA_DOM],
+ 0, 0, buf, &n, arg);
+
+ if (!ret)
+ return n;
+
+ if (ret != H_PARAMETER)
+ goto out;
+
+ /*
+ * ret value as 'H_PARAMETER' corresponds to 'GEN_BUF_TOO_SMALL', which
+ * implies that buffer can't accommodate all information, and a partial buffer
+ * returned. To handle that, we need to take subsequent requests
+ * with next starting index to retrieve additional (missing) data.
+ * Below loop do subsequent hcalls with next starting index and add it
+ * to buffer util we get all the information.
+ */
+ while (ret == H_PARAMETER) {
+ int returned_values = be16_to_cpu(arg->params.returned_values);
+ int elementsize = be16_to_cpu(arg->params.cv_element_size);
+ int last_element = (returned_values - 1) * elementsize;
+
+ /*
+ * Since the starting index value is part of counter_value
+ * buffer elements, use the starting index value in the last
+ * element and add 1 to make subsequent hcalls.
+ */
+ u32 starting_index = arg->bytes[last_element + 1] +
+ (arg->bytes[last_element] << 8) + 1;
+
+ memset(arg, 0, HGPCI_REQ_BUFFER_SIZE);
+
+ ret = systeminfo_gpci_request(sysinfo_counter_request[AFFINITY_DOMAIN_VIA_DOM],
+ starting_index, 0, buf, &n, arg);
+
+ if (!ret)
+ return n;
+
+ if (ret != H_PARAMETER)
+ goto out;
+ }
+
+ return n;
+
+out:
+ put_cpu_var(hv_gpci_reqb);
+ return ret;
+}
+
+static void affinity_domain_via_partition_result_parse(int returned_values,
+ int element_size, char *buf, size_t *last_element,
+ size_t *n, struct hv_gpci_request_buffer *arg)
+{
+ size_t i = 0, j = 0;
+ size_t k, l, m;
+ uint16_t total_affinity_domain_ele, size_of_each_affinity_domain_ele;
+
+ /*
+ * hcall H_GET_PERF_COUNTER_INFO populates the 'returned_values'
+ * to show the total number of counter_value array elements
+ * returned via hcall.
+ * Unlike other request types, the data structure returned by this
+ * request is variable-size. For this counter request type,
+ * hcall populates 'cv_element_size' corresponds to minimum size of
+ * the structure returned i.e; the size of the structure with no domain
+ * information. Below loop go through all counter_value array
+ * to determine the number and size of each domain array element and
+ * add it to the output buffer.
+ */
+ while (i < returned_values) {
+ k = j;
+ for (; k < j + element_size; k++)
+ *n += sprintf(buf + *n, "%02x", (u8)arg->bytes[k]);
+ *n += sprintf(buf + *n, "\n");
+
+ total_affinity_domain_ele = (u8)arg->bytes[k - 2] << 8 | (u8)arg->bytes[k - 3];
+ size_of_each_affinity_domain_ele = (u8)arg->bytes[k] << 8 | (u8)arg->bytes[k - 1];
+
+ for (l = 0; l < total_affinity_domain_ele; l++) {
+ for (m = 0; m < size_of_each_affinity_domain_ele; m++) {
+ *n += sprintf(buf + *n, "%02x", (u8)arg->bytes[k]);
+ k++;
+ }
+ *n += sprintf(buf + *n, "\n");
+ }
+
+ *n += sprintf(buf + *n, "\n");
+ i++;
+ j = k;
+ }
+
+ *last_element = k;
+}
+
+static ssize_t affinity_domain_via_partition_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct hv_gpci_request_buffer *arg;
+ unsigned long ret;
+ size_t n = 0;
+ size_t last_element = 0;
+ u32 starting_index;
+
+ arg = (void *)get_cpu_var(hv_gpci_reqb);
+ memset(arg, 0, HGPCI_REQ_BUFFER_SIZE);
+
+ /*
+ * Pass the counter request value 0xB1 corresponds to counter request
+ * type 'Affinity_domain_information_by_partition',
+ * to retrieve the system affinity domain by partition information.
+ * starting_index value refers to the starting hardware
+ * processor index.
+ */
+ arg->params.counter_request = cpu_to_be32(sysinfo_counter_request[AFFINITY_DOMAIN_VIA_PAR]);
+ arg->params.starting_index = cpu_to_be32(0);
+
+ ret = plpar_hcall_norets(H_GET_PERF_COUNTER_INFO,
+ virt_to_phys(arg), HGPCI_REQ_BUFFER_SIZE);
+
+ if (!ret)
+ goto parse_result;
+
+ /*
+ * ret value as 'H_PARAMETER' implies that the current buffer size
+ * can't accommodate all the information, and a partial buffer
+ * returned. To handle that, we need to make subsequent requests
+ * with next starting index to retrieve additional (missing) data.
+ * Below loop do subsequent hcalls with next starting index and add it
+ * to buffer util we get all the information.
+ */
+ while (ret == H_PARAMETER) {
+ affinity_domain_via_partition_result_parse(
+ be16_to_cpu(arg->params.returned_values) - 1,
+ be16_to_cpu(arg->params.cv_element_size), buf,
+ &last_element, &n, arg);
+
+ if (n >= PAGE_SIZE) {
+ put_cpu_var(hv_gpci_reqb);
+ pr_debug("System information exceeds PAGE_SIZE\n");
+ return -EFBIG;
+ }
+
+ /*
+ * Since the starting index value is part of counter_value
+ * buffer elements, use the starting_index value in the last
+ * element and add 1 to make subsequent hcalls.
+ */
+ starting_index = (u8)arg->bytes[last_element] << 8 |
+ (u8)arg->bytes[last_element + 1];
+
+ memset(arg, 0, HGPCI_REQ_BUFFER_SIZE);
+ arg->params.counter_request = cpu_to_be32(
+ sysinfo_counter_request[AFFINITY_DOMAIN_VIA_PAR]);
+ arg->params.starting_index = cpu_to_be32(starting_index);
+
+ ret = plpar_hcall_norets(H_GET_PERF_COUNTER_INFO,
+ virt_to_phys(arg), HGPCI_REQ_BUFFER_SIZE);
+
+ if (ret && (ret != H_PARAMETER))
+ goto out;
+ }
+
+parse_result:
+ affinity_domain_via_partition_result_parse(
+ be16_to_cpu(arg->params.returned_values),
+ be16_to_cpu(arg->params.cv_element_size),
+ buf, &last_element, &n, arg);
+
+ put_cpu_var(hv_gpci_reqb);
+ return n;
+
+out:
+ put_cpu_var(hv_gpci_reqb);
+
+ /*
+ * ret value as 'H_PARAMETER' corresponds to 'GEN_BUF_TOO_SMALL',
+ * which means that the current buffer size cannot accommodate
+ * all the information and a partial buffer returned.
+ * hcall fails incase of ret value other than H_SUCCESS or H_PARAMETER.
+ *
+ * ret value as H_AUTHORITY implies that partition is not permitted to retrieve
+ * performance information, and required to set
+ * "Enable Performance Information Collection" option.
+ */
+ if (ret == H_AUTHORITY)
+ return -EPERM;
+
+ /*
+ * hcall can fail with other possible ret value like H_PRIVILEGE/H_HARDWARE
+ * because of invalid buffer-length/address or due to some hardware
+ * error.
+ */
+ return -EIO;
+}
+
static DEVICE_ATTR_RO(kernel_version);
static DEVICE_ATTR_RO(cpumask);
&hv_caps_attr_expanded.attr,
&hv_caps_attr_lab.attr,
&hv_caps_attr_collect_privileged.attr,
+ /*
+ * This NULL is a placeholder for the processor_bus_topology
+ * attribute, set in init function if applicable.
+ */
+ NULL,
+ /*
+ * This NULL is a placeholder for the processor_config
+ * attribute, set in init function if applicable.
+ */
+ NULL,
+ /*
+ * This NULL is a placeholder for the affinity_domain_via_virtual_processor
+ * attribute, set in init function if applicable.
+ */
+ NULL,
+ /*
+ * This NULL is a placeholder for the affinity_domain_via_domain
+ * attribute, set in init function if applicable.
+ */
+ NULL,
+ /*
+ * This NULL is a placeholder for the affinity_domain_via_partition
+ * attribute, set in init function if applicable.
+ */
+ NULL,
NULL,
};
NULL,
};
-static DEFINE_PER_CPU(char, hv_gpci_reqb[HGPCI_REQ_BUFFER_SIZE]) __aligned(sizeof(uint64_t));
-
static unsigned long single_gpci_request(u32 req, u32 starting_index,
u16 secondary_index, u8 version_in, u32 offset, u8 length,
u64 *value)
ppc_hv_gpci_cpu_offline);
}
+static struct device_attribute *sysinfo_device_attr_create(int
+ sysinfo_interface_group_index, u32 req)
+{
+ struct device_attribute *attr = NULL;
+ unsigned long ret;
+ struct hv_gpci_request_buffer *arg;
+
+ if (sysinfo_interface_group_index < INTERFACE_PROCESSOR_BUS_TOPOLOGY_ATTR ||
+ sysinfo_interface_group_index >= INTERFACE_NULL_ATTR) {
+ pr_info("Wrong interface group index for system information\n");
+ return NULL;
+ }
+
+ /* Check for given counter request value support */
+ arg = (void *)get_cpu_var(hv_gpci_reqb);
+ memset(arg, 0, HGPCI_REQ_BUFFER_SIZE);
+
+ arg->params.counter_request = cpu_to_be32(req);
+
+ ret = plpar_hcall_norets(H_GET_PERF_COUNTER_INFO,
+ virt_to_phys(arg), HGPCI_REQ_BUFFER_SIZE);
+
+ put_cpu_var(hv_gpci_reqb);
+
+ /*
+ * Add given counter request value attribute in the interface_attrs
+ * attribute array, only for valid return types.
+ */
+ if (!ret || ret == H_AUTHORITY || ret == H_PARAMETER) {
+ attr = kzalloc(sizeof(*attr), GFP_KERNEL);
+ if (!attr)
+ return NULL;
+
+ sysfs_attr_init(&attr->attr);
+ attr->attr.mode = 0444;
+
+ switch (sysinfo_interface_group_index) {
+ case INTERFACE_PROCESSOR_BUS_TOPOLOGY_ATTR:
+ attr->attr.name = "processor_bus_topology";
+ attr->show = processor_bus_topology_show;
+ break;
+ case INTERFACE_PROCESSOR_CONFIG_ATTR:
+ attr->attr.name = "processor_config";
+ attr->show = processor_config_show;
+ break;
+ case INTERFACE_AFFINITY_DOMAIN_VIA_VP_ATTR:
+ attr->attr.name = "affinity_domain_via_virtual_processor";
+ attr->show = affinity_domain_via_virtual_processor_show;
+ break;
+ case INTERFACE_AFFINITY_DOMAIN_VIA_DOM_ATTR:
+ attr->attr.name = "affinity_domain_via_domain";
+ attr->show = affinity_domain_via_domain_show;
+ break;
+ case INTERFACE_AFFINITY_DOMAIN_VIA_PAR_ATTR:
+ attr->attr.name = "affinity_domain_via_partition";
+ attr->show = affinity_domain_via_partition_show;
+ break;
+ }
+ } else
+ pr_devel("hcall failed, with error: 0x%lx\n", ret);
+
+ return attr;
+}
+
+static void add_sysinfo_interface_files(void)
+{
+ int sysfs_count;
+ struct device_attribute *attr[INTERFACE_NULL_ATTR - INTERFACE_PROCESSOR_BUS_TOPOLOGY_ATTR];
+ int i;
+
+ sysfs_count = INTERFACE_NULL_ATTR - INTERFACE_PROCESSOR_BUS_TOPOLOGY_ATTR;
+
+ /* Get device attribute for a given counter request value */
+ for (i = 0; i < sysfs_count; i++) {
+ attr[i] = sysinfo_device_attr_create(i + INTERFACE_PROCESSOR_BUS_TOPOLOGY_ATTR,
+ sysinfo_counter_request[i]);
+
+ if (!attr[i])
+ goto out;
+ }
+
+ /* Add sysinfo interface attributes in the interface_attrs attribute array */
+ for (i = 0; i < sysfs_count; i++)
+ interface_attrs[i + INTERFACE_PROCESSOR_BUS_TOPOLOGY_ATTR] = &attr[i]->attr;
+
+ return;
+
+out:
+ /*
+ * The sysinfo interface attributes will be added, only if hcall passed for
+ * all the counter request values. Free the device attribute array incase
+ * of any hcall failure.
+ */
+ if (i > 0) {
+ while (i >= 0) {
+ kfree(attr[i]);
+ i--;
+ }
+ }
+}
+
static int hv_gpci_init(void)
{
int r;
if (r)
return r;
+ /* sysinfo interface files are only available for power10 and above platforms */
+ if (PVR_VER(mfspr(SPRN_PVR)) >= PVR_POWER10)
+ add_sysinfo_interface_files();
+
return 0;
}
#ifdef CONFIG_SENSORS_AD7414
-static LIST_HEAD(dtm_shutdown_list);
static void __iomem *dtm_fpga;
-struct dtm_shutdown {
- struct list_head list;
- void (*func)(void *arg);
- void *arg;
-};
-
-int pika_dtm_register_shutdown(void (*func)(void *arg), void *arg)
-{
- struct dtm_shutdown *shutdown;
-
- shutdown = kmalloc(sizeof(struct dtm_shutdown), GFP_KERNEL);
- if (shutdown == NULL)
- return -ENOMEM;
-
- shutdown->func = func;
- shutdown->arg = arg;
-
- list_add(&shutdown->list, &dtm_shutdown_list);
-
- return 0;
-}
-
-int pika_dtm_unregister_shutdown(void (*func)(void *arg), void *arg)
-{
- struct dtm_shutdown *shutdown;
-
- list_for_each_entry(shutdown, &dtm_shutdown_list, list)
- if (shutdown->func == func && shutdown->arg == arg) {
- list_del(&shutdown->list);
- kfree(shutdown);
- return 0;
- }
-
- return -EINVAL;
-}
-
#define WARP_GREEN_LED 0
#define WARP_RED_LED 1
static irqreturn_t temp_isr(int irq, void *context)
{
- struct dtm_shutdown *shutdown;
int value = 1;
local_irq_disable();
gpiod_set_value(warp_gpio_led_pins[WARP_GREEN_LED].gpiod, 0);
- /* Run through the shutdown list. */
- list_for_each_entry(shutdown, &dtm_shutdown_list, list)
- shutdown->func(shutdown->arg);
-
printk(KERN_EMERG "\n\nCritical Temperature Shutdown\n\n");
while (1) {
#else /* !CONFIG_SENSORS_AD7414 */
-int pika_dtm_register_shutdown(void (*func)(void *arg), void *arg)
-{
- return 0;
-}
-
-int pika_dtm_unregister_shutdown(void (*func)(void *arg), void *arg)
-{
- return 0;
-}
-
machine_late_initcall(warp, warp_post_info);
#endif
-
-EXPORT_SYMBOL(pika_dtm_register_shutdown);
-EXPORT_SYMBOL(pika_dtm_unregister_shutdown);
*/
#include <linux/kernel.h>
-#include <linux/of_platform.h>
+#include <linux/of.h>
#include <linux/sysfs.h>
#include <linux/cpu.h>
#include <linux/suspend.h>
#include <linux/msi.h>
#include <linux/of.h>
#include <linux/of_irq.h>
-#include <linux/of_platform.h>
+#include <linux/platform_device.h>
#include <linux/pci.h>
#include <linux/semaphore.h>
#include <asm/msi_bitmap.h>
#include <linux/errno.h>
#include <linux/interrupt.h>
#include <linux/irq.h>
+#include <linux/of.h>
#include <linux/of_irq.h>
-#include <linux/of_platform.h>
#include <asm/dcr.h>
#include <asm/dcr-regs.h>
#include <asm/reg.h>
+#include <asm/ppc4xx.h>
static u32 dcrbase_l2c;
#include <asm/irq.h>
#include <asm/io.h>
#include <asm/dcr.h>
+#include <asm/uic.h>
#define NR_UIC_INTS 32
#include <linux/kernel.h>
#include <linux/io.h>
-#include <linux/of_platform.h>
+#include <linux/of.h>
#include <asm/machdep.h>
#include <asm/ipic.h>
extern void __init mpc512x_setup_arch(void);
extern int __init mpc5121_clk_init(void);
const char *__init mpc512x_select_psc_compat(void);
-const char *__init mpc512x_select_reset_compat(void);
extern void __noreturn mpc512x_restart(char *cmd);
#endif /* __MPC512X_H__ */
*/
#include <linux/kernel.h>
-#include <linux/of_platform.h>
+#include <linux/of.h>
#include <asm/machdep.h>
#include <asm/ipic.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/of.h>
-#include <linux/of_platform.h>
#include <linux/of_address.h>
#include <linux/of_irq.h>
+#include <linux/platform_device.h>
#include <asm/mpc5121.h>
#include <asm/io.h>
#include <linux/spinlock.h>
static struct mpc512x_reset_module __iomem *reset_module_base;
-static void __init mpc512x_restart_init(void)
-{
- struct device_node *np;
- const char *reset_compat;
-
- reset_compat = mpc512x_select_reset_compat();
- np = of_find_compatible_node(NULL, NULL, reset_compat);
- if (!np)
- return;
-
- reset_module_base = of_iomap(np, 0);
- of_node_put(np);
-}
-
void __noreturn mpc512x_restart(char *cmd)
{
if (reset_module_base) {
return NULL;
}
-const char *__init mpc512x_select_reset_compat(void)
+static const char *__init mpc512x_select_reset_compat(void)
{
if (of_machine_is_compatible("fsl,mpc5121"))
return "fsl,mpc5121-reset";
}
}
+static void __init mpc512x_restart_init(void)
+{
+ struct device_node *np;
+ const char *reset_compat;
+
+ reset_compat = mpc512x_select_reset_compat();
+ np = of_find_compatible_node(NULL, NULL, reset_compat);
+ if (!np)
+ return;
+
+ reset_module_base = of_iomap(np, 0);
+ of_node_put(np);
+}
+
void __init mpc512x_init_early(void)
{
mpc512x_restart_init();
* PDM360NG board setup
*/
+#include <linux/device.h>
#include <linux/kernel.h>
#include <linux/io.h>
+#include <linux/of.h>
#include <linux/of_address.h>
#include <linux/of_fdt.h>
-#include <linux/of_platform.h>
#include <asm/machdep.h>
#include <asm/ipic.h>
* the output mode. This driver does not change the output mode setting.
*/
-#include <linux/device.h>
#include <linux/irq.h>
#include <linux/interrupt.h>
#include <linux/io.h>
#include <linux/of.h>
#include <linux/of_address.h>
#include <linux/of_irq.h>
-#include <linux/of_platform.h>
#include <linux/of_gpio.h>
+#include <linux/platform_device.h>
#include <linux/kernel.h>
#include <linux/property.h>
#include <linux/slab.h>
config EP8248E
bool "Embedded Planet EP8248E (a.k.a. CWH-PPC-8248N-VE)"
- select 8272
- select 8260
+ select CPM2
+ select PPC_INDIRECT_PCI if PCI
select FSL_SOC
select PHYLIB if NETDEVICES
select MDIO_BITBANG if PHYLIB
config MGCOGE
bool "Keymile MGCOGE"
- select 8272
- select 8260
+ select CPM2
+ select PPC_INDIRECT_PCI if PCI
select FSL_SOC
help
This enables support for the Keymile MGCOGE board.
endif
-
-config 8260
- bool
- depends on PPC_BOOK3S_32
- select CPM2
- help
- The MPC8260 is a typical embedded CPU made by Freescale. Selecting
- this option means that you wish to build a kernel for a machine with
- an 8260 class CPU.
-
-config 8272
- bool
- select 8260
- help
- The MPC8272 CPM has a different internal dpram setup than other CPM2
- devices
#include <linux/of_mdio.h>
#include <linux/slab.h>
#include <linux/of_platform.h>
+#include <linux/platform_device.h>
#include <asm/io.h>
#include <asm/cpm2.h>
#include <asm/udbg.h>
#include <asm/machdep.h>
#include <asm/time.h>
-#include <asm/mpc8260.h>
#include <sysdev/fsl_soc.h>
#include <sysdev/cpm2_pic.h>
return ret;
}
-static int ep8248e_mdio_remove(struct platform_device *ofdev)
-{
- BUG();
- return 0;
-}
-
static const struct of_device_id ep8248e_mdio_match[] = {
{
.compatible = "fsl,ep8248e-mdio-bitbang",
.driver = {
.name = "ep8248e-mdio-bitbang",
.of_match_table = ep8248e_mdio_match,
+ .suppress_bind_attrs = true,
},
.probe = ep8248e_mdio_probe,
- .remove = ep8248e_mdio_remove,
};
struct cpm_pin {
#include <asm/udbg.h>
#include <asm/machdep.h>
#include <linux/time.h>
-#include <asm/mpc8260.h>
#include <sysdev/fsl_soc.h>
#include <sysdev/cpm2_pic.h>
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0-or-later */
-#ifndef _PPC_KERNEL_M82XX_PCI_H
-#define _PPC_KERNEL_M82XX_PCI_H
-
-/*
- */
-
-#define SIU_INT_IRQ1 ((uint)0x13 + CPM_IRQ_OFFSET)
-
-#ifndef _IO_BASE
-#define _IO_BASE isa_io_base
-#endif
-
-#endif /* _PPC_KERNEL_M8260_PCI_H */
panic("Restart failed\n");
}
NOKPROBE_SYMBOL(pq2_restart)
-
-#ifdef CONFIG_PCI
-static int pq2_pci_exclude_device(struct pci_controller *hose,
- u_char bus, u8 devfn)
-{
- if (bus == 0 && PCI_SLOT(devfn) == 0)
- return PCIBIOS_DEVICE_NOT_FOUND;
- else
- return PCIBIOS_SUCCESSFUL;
-}
-
-static void __init pq2_pci_add_bridge(struct device_node *np)
-{
- struct pci_controller *hose;
- struct resource r;
-
- if (of_address_to_resource(np, 0, &r) || r.end - r.start < 0x10b)
- goto err;
-
- pci_add_flags(PCI_REASSIGN_ALL_BUS);
-
- hose = pcibios_alloc_controller(np);
- if (!hose)
- return;
-
- hose->dn = np;
-
- setup_indirect_pci(hose, r.start + 0x100, r.start + 0x104, 0);
- pci_process_bridge_OF_ranges(hose, np, 1);
-
- return;
-
-err:
- printk(KERN_ERR "No valid PCI reg property in device tree\n");
-}
-
-void __init pq2_init_pci(void)
-{
- struct device_node *np;
-
- ppc_md.pci_exclude_device = pq2_pci_exclude_device;
-
- for_each_compatible_node(np, NULL, "fsl,pq2-pci")
- pq2_pci_add_bridge(np);
-}
-#endif
#
# Makefile for the PowerPC 83xx linux kernel.
#
-obj-y := misc.o usb.o
+obj-y := misc.o
obj-$(CONFIG_SUSPEND) += suspend.o suspend-asm.o
obj-$(CONFIG_MCU_MPC8349EMITX) += mcu_mpc8349emitx.o
obj-$(CONFIG_MPC830x_RDB) += mpc830x_rdb.o
obj-$(CONFIG_MPC837x_RDB) += mpc837x_rdb.o
obj-$(CONFIG_ASP834x) += asp834x.o
obj-$(CONFIG_KMETER1) += km83xx.o
+obj-$(CONFIG_PPC_MPC831x) += usb_831x.o
+obj-$(CONFIG_PPC_MPC834x) += usb_834x.o
+obj-$(CONFIG_PPC_MPC837x) += usb_837x.o
#include <linux/seq_file.h>
#include <linux/root_dev.h>
#include <linux/initrd.h>
-#include <linux/of_platform.h>
-#include <linux/of_device.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
#include <linux/atomic.h>
#include <linux/time.h>
#include <linux/spi/spi.h>
#include <linux/spi/mmc_spi.h>
#include <linux/mmc/host.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
#include <linux/of_irq.h>
-#include <linux/of_platform.h>
+#include <linux/platform_device.h>
#include <linux/fsl_devices.h>
#include <asm/time.h>
#define __MPC83XX_H__
#include <linux/init.h>
-#include <linux/device.h>
-#include <asm/pci-bridge.h>
/* System Clock Control Register */
#define MPC83XX_SCCR_OFFS 0xA08
#include <linux/fsl_devices.h>
#include <linux/of_address.h>
#include <linux/of_irq.h>
-#include <linux/of_platform.h>
+#include <linux/platform_device.h>
#include <linux/export.h>
#include <asm/reg.h>
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0-or-later
-/*
- * Freescale 83xx USB SOC setup code
- *
- * Copyright (C) 2007 Freescale Semiconductor, Inc.
- * Author: Li Yang
- */
-
-
-#include <linux/stddef.h>
-#include <linux/kernel.h>
-#include <linux/errno.h>
-#include <linux/of.h>
-#include <linux/of_address.h>
-
-#include <asm/io.h>
-#include <sysdev/fsl_soc.h>
-
-#include "mpc83xx.h"
-
-
-#ifdef CONFIG_PPC_MPC834x
-int __init mpc834x_usb_cfg(void)
-{
- unsigned long sccr, sicrl, sicrh;
- void __iomem *immap;
- struct device_node *np = NULL;
- int port0_is_dr = 0, port1_is_dr = 0;
- const void *prop, *dr_mode;
-
- immap = ioremap(get_immrbase(), 0x1000);
- if (!immap)
- return -ENOMEM;
-
- /* Read registers */
- /* Note: DR and MPH must use the same clock setting in SCCR */
- sccr = in_be32(immap + MPC83XX_SCCR_OFFS) & ~MPC83XX_SCCR_USB_MASK;
- sicrl = in_be32(immap + MPC83XX_SICRL_OFFS) & ~MPC834X_SICRL_USB_MASK;
- sicrh = in_be32(immap + MPC83XX_SICRH_OFFS) & ~MPC834X_SICRH_USB_UTMI;
-
- np = of_find_compatible_node(NULL, NULL, "fsl-usb2-dr");
- if (np) {
- sccr |= MPC83XX_SCCR_USB_DRCM_11; /* 1:3 */
-
- prop = of_get_property(np, "phy_type", NULL);
- port1_is_dr = 1;
- if (prop && (!strcmp(prop, "utmi") ||
- !strcmp(prop, "utmi_wide"))) {
- sicrl |= MPC834X_SICRL_USB0 | MPC834X_SICRL_USB1;
- sicrh |= MPC834X_SICRH_USB_UTMI;
- port0_is_dr = 1;
- } else if (prop && !strcmp(prop, "serial")) {
- dr_mode = of_get_property(np, "dr_mode", NULL);
- if (dr_mode && !strcmp(dr_mode, "otg")) {
- sicrl |= MPC834X_SICRL_USB0 | MPC834X_SICRL_USB1;
- port0_is_dr = 1;
- } else {
- sicrl |= MPC834X_SICRL_USB1;
- }
- } else if (prop && !strcmp(prop, "ulpi")) {
- sicrl |= MPC834X_SICRL_USB1;
- } else {
- printk(KERN_WARNING "834x USB PHY type not supported\n");
- }
- of_node_put(np);
- }
- np = of_find_compatible_node(NULL, NULL, "fsl-usb2-mph");
- if (np) {
- sccr |= MPC83XX_SCCR_USB_MPHCM_11; /* 1:3 */
-
- prop = of_get_property(np, "port0", NULL);
- if (prop) {
- if (port0_is_dr)
- printk(KERN_WARNING
- "834x USB port0 can't be used by both DR and MPH!\n");
- sicrl &= ~MPC834X_SICRL_USB0;
- }
- prop = of_get_property(np, "port1", NULL);
- if (prop) {
- if (port1_is_dr)
- printk(KERN_WARNING
- "834x USB port1 can't be used by both DR and MPH!\n");
- sicrl &= ~MPC834X_SICRL_USB1;
- }
- of_node_put(np);
- }
-
- /* Write back */
- out_be32(immap + MPC83XX_SCCR_OFFS, sccr);
- out_be32(immap + MPC83XX_SICRL_OFFS, sicrl);
- out_be32(immap + MPC83XX_SICRH_OFFS, sicrh);
-
- iounmap(immap);
- return 0;
-}
-#endif /* CONFIG_PPC_MPC834x */
-
-#ifdef CONFIG_PPC_MPC831x
-int __init mpc831x_usb_cfg(void)
-{
- u32 temp;
- void __iomem *immap, *usb_regs;
- struct device_node *np = NULL;
- struct device_node *immr_node = NULL;
- const void *prop;
- struct resource res;
- int ret = 0;
-#ifdef CONFIG_USB_OTG
- const void *dr_mode;
-#endif
-
- np = of_find_compatible_node(NULL, NULL, "fsl-usb2-dr");
- if (!np)
- return -ENODEV;
- prop = of_get_property(np, "phy_type", NULL);
-
- /* Map IMMR space for pin and clock settings */
- immap = ioremap(get_immrbase(), 0x1000);
- if (!immap) {
- of_node_put(np);
- return -ENOMEM;
- }
-
- /* Configure clock */
- immr_node = of_get_parent(np);
- if (immr_node && (of_device_is_compatible(immr_node, "fsl,mpc8315-immr") ||
- of_device_is_compatible(immr_node, "fsl,mpc8308-immr")))
- clrsetbits_be32(immap + MPC83XX_SCCR_OFFS,
- MPC8315_SCCR_USB_MASK,
- MPC8315_SCCR_USB_DRCM_01);
- else
- clrsetbits_be32(immap + MPC83XX_SCCR_OFFS,
- MPC83XX_SCCR_USB_MASK,
- MPC83XX_SCCR_USB_DRCM_11);
-
- /* Configure pin mux for ULPI. There is no pin mux for UTMI */
- if (prop && !strcmp(prop, "ulpi")) {
- if (of_device_is_compatible(immr_node, "fsl,mpc8308-immr")) {
- clrsetbits_be32(immap + MPC83XX_SICRH_OFFS,
- MPC8308_SICRH_USB_MASK,
- MPC8308_SICRH_USB_ULPI);
- } else if (of_device_is_compatible(immr_node, "fsl,mpc8315-immr")) {
- clrsetbits_be32(immap + MPC83XX_SICRL_OFFS,
- MPC8315_SICRL_USB_MASK,
- MPC8315_SICRL_USB_ULPI);
- clrsetbits_be32(immap + MPC83XX_SICRH_OFFS,
- MPC8315_SICRH_USB_MASK,
- MPC8315_SICRH_USB_ULPI);
- } else {
- clrsetbits_be32(immap + MPC83XX_SICRL_OFFS,
- MPC831X_SICRL_USB_MASK,
- MPC831X_SICRL_USB_ULPI);
- clrsetbits_be32(immap + MPC83XX_SICRH_OFFS,
- MPC831X_SICRH_USB_MASK,
- MPC831X_SICRH_USB_ULPI);
- }
- }
-
- iounmap(immap);
-
- of_node_put(immr_node);
-
- /* Map USB SOC space */
- ret = of_address_to_resource(np, 0, &res);
- if (ret) {
- of_node_put(np);
- return ret;
- }
- usb_regs = ioremap(res.start, resource_size(&res));
-
- /* Using on-chip PHY */
- if (prop && (!strcmp(prop, "utmi_wide") ||
- !strcmp(prop, "utmi"))) {
- u32 refsel;
-
- if (of_device_is_compatible(immr_node, "fsl,mpc8308-immr"))
- goto out;
-
- if (of_device_is_compatible(immr_node, "fsl,mpc8315-immr"))
- refsel = CONTROL_REFSEL_24MHZ;
- else
- refsel = CONTROL_REFSEL_48MHZ;
- /* Set UTMI_PHY_EN and REFSEL */
- out_be32(usb_regs + FSL_USB2_CONTROL_OFFS,
- CONTROL_UTMI_PHY_EN | refsel);
- /* Using external UPLI PHY */
- } else if (prop && !strcmp(prop, "ulpi")) {
- /* Set PHY_CLK_SEL to ULPI */
- temp = CONTROL_PHY_CLK_SEL_ULPI;
-#ifdef CONFIG_USB_OTG
- /* Set OTG_PORT */
- if (!of_device_is_compatible(immr_node, "fsl,mpc8308-immr")) {
- dr_mode = of_get_property(np, "dr_mode", NULL);
- if (dr_mode && !strcmp(dr_mode, "otg"))
- temp |= CONTROL_OTG_PORT;
- }
-#endif /* CONFIG_USB_OTG */
- out_be32(usb_regs + FSL_USB2_CONTROL_OFFS, temp);
- } else {
- printk(KERN_WARNING "831x USB PHY type not supported\n");
- ret = -EINVAL;
- }
-
-out:
- iounmap(usb_regs);
- of_node_put(np);
- return ret;
-}
-#endif /* CONFIG_PPC_MPC831x */
-
-#ifdef CONFIG_PPC_MPC837x
-int __init mpc837x_usb_cfg(void)
-{
- void __iomem *immap;
- struct device_node *np = NULL;
- const void *prop;
- int ret = 0;
-
- np = of_find_compatible_node(NULL, NULL, "fsl-usb2-dr");
- if (!np || !of_device_is_available(np)) {
- of_node_put(np);
- return -ENODEV;
- }
- prop = of_get_property(np, "phy_type", NULL);
-
- if (!prop || (strcmp(prop, "ulpi") && strcmp(prop, "serial"))) {
- printk(KERN_WARNING "837x USB PHY type not supported\n");
- of_node_put(np);
- return -EINVAL;
- }
-
- /* Map IMMR space for pin and clock settings */
- immap = ioremap(get_immrbase(), 0x1000);
- if (!immap) {
- of_node_put(np);
- return -ENOMEM;
- }
-
- /* Configure clock */
- clrsetbits_be32(immap + MPC83XX_SCCR_OFFS, MPC837X_SCCR_USB_DRCM_11,
- MPC837X_SCCR_USB_DRCM_11);
-
- /* Configure pin mux for ULPI/serial */
- clrsetbits_be32(immap + MPC83XX_SICRL_OFFS, MPC837X_SICRL_USB_MASK,
- MPC837X_SICRL_USB_ULPI);
-
- iounmap(immap);
- of_node_put(np);
- return ret;
-}
-#endif /* CONFIG_PPC_MPC837x */
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Freescale 83xx USB SOC setup code
+ *
+ * Copyright (C) 2007 Freescale Semiconductor, Inc.
+ * Author: Li Yang
+ */
+
+#include <linux/stddef.h>
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/io.h>
+
+#include <sysdev/fsl_soc.h>
+
+#include "mpc83xx.h"
+
+int __init mpc831x_usb_cfg(void)
+{
+ u32 temp;
+ void __iomem *immap, *usb_regs;
+ struct device_node *np = NULL;
+ struct device_node *immr_node = NULL;
+ const void *prop;
+ struct resource res;
+ int ret = 0;
+#ifdef CONFIG_USB_OTG
+ const void *dr_mode;
+#endif
+
+ np = of_find_compatible_node(NULL, NULL, "fsl-usb2-dr");
+ if (!np)
+ return -ENODEV;
+ prop = of_get_property(np, "phy_type", NULL);
+
+ /* Map IMMR space for pin and clock settings */
+ immap = ioremap(get_immrbase(), 0x1000);
+ if (!immap) {
+ of_node_put(np);
+ return -ENOMEM;
+ }
+
+ /* Configure clock */
+ immr_node = of_get_parent(np);
+ if (immr_node && (of_device_is_compatible(immr_node, "fsl,mpc8315-immr") ||
+ of_device_is_compatible(immr_node, "fsl,mpc8308-immr")))
+ clrsetbits_be32(immap + MPC83XX_SCCR_OFFS,
+ MPC8315_SCCR_USB_MASK,
+ MPC8315_SCCR_USB_DRCM_01);
+ else
+ clrsetbits_be32(immap + MPC83XX_SCCR_OFFS,
+ MPC83XX_SCCR_USB_MASK,
+ MPC83XX_SCCR_USB_DRCM_11);
+
+ /* Configure pin mux for ULPI. There is no pin mux for UTMI */
+ if (prop && !strcmp(prop, "ulpi")) {
+ if (of_device_is_compatible(immr_node, "fsl,mpc8308-immr")) {
+ clrsetbits_be32(immap + MPC83XX_SICRH_OFFS,
+ MPC8308_SICRH_USB_MASK,
+ MPC8308_SICRH_USB_ULPI);
+ } else if (of_device_is_compatible(immr_node, "fsl,mpc8315-immr")) {
+ clrsetbits_be32(immap + MPC83XX_SICRL_OFFS,
+ MPC8315_SICRL_USB_MASK,
+ MPC8315_SICRL_USB_ULPI);
+ clrsetbits_be32(immap + MPC83XX_SICRH_OFFS,
+ MPC8315_SICRH_USB_MASK,
+ MPC8315_SICRH_USB_ULPI);
+ } else {
+ clrsetbits_be32(immap + MPC83XX_SICRL_OFFS,
+ MPC831X_SICRL_USB_MASK,
+ MPC831X_SICRL_USB_ULPI);
+ clrsetbits_be32(immap + MPC83XX_SICRH_OFFS,
+ MPC831X_SICRH_USB_MASK,
+ MPC831X_SICRH_USB_ULPI);
+ }
+ }
+
+ iounmap(immap);
+
+ of_node_put(immr_node);
+
+ /* Map USB SOC space */
+ ret = of_address_to_resource(np, 0, &res);
+ if (ret) {
+ of_node_put(np);
+ return ret;
+ }
+ usb_regs = ioremap(res.start, resource_size(&res));
+
+ /* Using on-chip PHY */
+ if (prop && (!strcmp(prop, "utmi_wide") || !strcmp(prop, "utmi"))) {
+ u32 refsel;
+
+ if (of_device_is_compatible(immr_node, "fsl,mpc8308-immr"))
+ goto out;
+
+ if (of_device_is_compatible(immr_node, "fsl,mpc8315-immr"))
+ refsel = CONTROL_REFSEL_24MHZ;
+ else
+ refsel = CONTROL_REFSEL_48MHZ;
+ /* Set UTMI_PHY_EN and REFSEL */
+ out_be32(usb_regs + FSL_USB2_CONTROL_OFFS,
+ CONTROL_UTMI_PHY_EN | refsel);
+ /* Using external UPLI PHY */
+ } else if (prop && !strcmp(prop, "ulpi")) {
+ /* Set PHY_CLK_SEL to ULPI */
+ temp = CONTROL_PHY_CLK_SEL_ULPI;
+#ifdef CONFIG_USB_OTG
+ /* Set OTG_PORT */
+ if (!of_device_is_compatible(immr_node, "fsl,mpc8308-immr")) {
+ dr_mode = of_get_property(np, "dr_mode", NULL);
+ if (dr_mode && !strcmp(dr_mode, "otg"))
+ temp |= CONTROL_OTG_PORT;
+ }
+#endif /* CONFIG_USB_OTG */
+ out_be32(usb_regs + FSL_USB2_CONTROL_OFFS, temp);
+ } else {
+ pr_warn("831x USB PHY type not supported\n");
+ ret = -EINVAL;
+ }
+
+out:
+ iounmap(usb_regs);
+ of_node_put(np);
+ return ret;
+}
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Freescale 83xx USB SOC setup code
+ *
+ * Copyright (C) 2007 Freescale Semiconductor, Inc.
+ * Author: Li Yang
+ */
+
+#include <linux/stddef.h>
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/io.h>
+
+#include <sysdev/fsl_soc.h>
+
+#include "mpc83xx.h"
+
+int __init mpc834x_usb_cfg(void)
+{
+ unsigned long sccr, sicrl, sicrh;
+ void __iomem *immap;
+ struct device_node *np = NULL;
+ int port0_is_dr = 0, port1_is_dr = 0;
+ const void *prop, *dr_mode;
+
+ immap = ioremap(get_immrbase(), 0x1000);
+ if (!immap)
+ return -ENOMEM;
+
+ /* Read registers */
+ /* Note: DR and MPH must use the same clock setting in SCCR */
+ sccr = in_be32(immap + MPC83XX_SCCR_OFFS) & ~MPC83XX_SCCR_USB_MASK;
+ sicrl = in_be32(immap + MPC83XX_SICRL_OFFS) & ~MPC834X_SICRL_USB_MASK;
+ sicrh = in_be32(immap + MPC83XX_SICRH_OFFS) & ~MPC834X_SICRH_USB_UTMI;
+
+ np = of_find_compatible_node(NULL, NULL, "fsl-usb2-dr");
+ if (np) {
+ sccr |= MPC83XX_SCCR_USB_DRCM_11; /* 1:3 */
+
+ prop = of_get_property(np, "phy_type", NULL);
+ port1_is_dr = 1;
+ if (prop &&
+ (!strcmp(prop, "utmi") || !strcmp(prop, "utmi_wide"))) {
+ sicrl |= MPC834X_SICRL_USB0 | MPC834X_SICRL_USB1;
+ sicrh |= MPC834X_SICRH_USB_UTMI;
+ port0_is_dr = 1;
+ } else if (prop && !strcmp(prop, "serial")) {
+ dr_mode = of_get_property(np, "dr_mode", NULL);
+ if (dr_mode && !strcmp(dr_mode, "otg")) {
+ sicrl |= MPC834X_SICRL_USB0 | MPC834X_SICRL_USB1;
+ port0_is_dr = 1;
+ } else {
+ sicrl |= MPC834X_SICRL_USB1;
+ }
+ } else if (prop && !strcmp(prop, "ulpi")) {
+ sicrl |= MPC834X_SICRL_USB1;
+ } else {
+ pr_warn("834x USB PHY type not supported\n");
+ }
+ of_node_put(np);
+ }
+ np = of_find_compatible_node(NULL, NULL, "fsl-usb2-mph");
+ if (np) {
+ sccr |= MPC83XX_SCCR_USB_MPHCM_11; /* 1:3 */
+
+ prop = of_get_property(np, "port0", NULL);
+ if (prop) {
+ if (port0_is_dr)
+ pr_warn("834x USB port0 can't be used by both DR and MPH!\n");
+ sicrl &= ~MPC834X_SICRL_USB0;
+ }
+ prop = of_get_property(np, "port1", NULL);
+ if (prop) {
+ if (port1_is_dr)
+ pr_warn("834x USB port1 can't be used by both DR and MPH!\n");
+ sicrl &= ~MPC834X_SICRL_USB1;
+ }
+ of_node_put(np);
+ }
+
+ /* Write back */
+ out_be32(immap + MPC83XX_SCCR_OFFS, sccr);
+ out_be32(immap + MPC83XX_SICRL_OFFS, sicrl);
+ out_be32(immap + MPC83XX_SICRH_OFFS, sicrh);
+
+ iounmap(immap);
+ return 0;
+}
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Freescale 83xx USB SOC setup code
+ *
+ * Copyright (C) 2007 Freescale Semiconductor, Inc.
+ * Author: Li Yang
+ */
+
+#include <linux/stddef.h>
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/io.h>
+
+#include <sysdev/fsl_soc.h>
+
+#include "mpc83xx.h"
+
+int __init mpc837x_usb_cfg(void)
+{
+ void __iomem *immap;
+ struct device_node *np = NULL;
+ const void *prop;
+ int ret = 0;
+
+ np = of_find_compatible_node(NULL, NULL, "fsl-usb2-dr");
+ if (!np || !of_device_is_available(np)) {
+ of_node_put(np);
+ return -ENODEV;
+ }
+ prop = of_get_property(np, "phy_type", NULL);
+
+ if (!prop || (strcmp(prop, "ulpi") && strcmp(prop, "serial"))) {
+ pr_warn("837x USB PHY type not supported\n");
+ of_node_put(np);
+ return -EINVAL;
+ }
+
+ /* Map IMMR space for pin and clock settings */
+ immap = ioremap(get_immrbase(), 0x1000);
+ if (!immap) {
+ of_node_put(np);
+ return -ENOMEM;
+ }
+
+ /* Configure clock */
+ clrsetbits_be32(immap + MPC83XX_SCCR_OFFS, MPC837X_SCCR_USB_DRCM_11,
+ MPC837X_SCCR_USB_DRCM_11);
+
+ /* Configure pin mux for ULPI/serial */
+ clrsetbits_be32(immap + MPC83XX_SICRL_OFFS, MPC837X_SICRL_USB_MASK,
+ MPC837X_SICRL_USB_ULPI);
+
+ iounmap(immap);
+ of_node_put(np);
+ return ret;
+}
* Copyright 2014 Freescale Semiconductor Inc.
*/
-#include <linux/of_platform.h>
+#include <linux/of.h>
#include <linux/pci.h>
#include <asm/mpic.h>
#include <sysdev/fsl_soc.h>
* Copyright 2011-2012 Freescale Semiconductor Inc.
*/
-#include <linux/of_platform.h>
+#include <linux/of.h>
#include <linux/pci.h>
#include <asm/mpic.h>
#include <sysdev/fsl_soc.h>
#include <linux/stddef.h>
#include <linux/kernel.h>
-#include <linux/of_fdt.h>
-#include <linux/of_platform.h>
+#include <linux/of.h>
#include <asm/machdep.h>
#include <asm/udbg.h>
* Routines common to most mpc85xx-based boards.
*/
+#include <linux/of.h>
#include <linux/of_irq.h>
#include <linux/of_platform.h>
#include "smp.h"
#include "mpc85xx.h"
-void __init corenet_gen_pic_init(void)
+static void __init corenet_gen_pic_init(void)
{
struct mpic *mpic;
unsigned int flags = MPIC_BIG_ENDIAN | MPIC_SINGLE_DEST_CPU |
/*
* Setup the architecture
*/
-void __init corenet_gen_setup_arch(void)
+static void __init corenet_gen_setup_arch(void)
{
mpc85xx_smp_init();
{}
};
-int __init corenet_gen_publish_devices(void)
+static int __init corenet_gen_publish_devices(void)
{
return of_platform_bus_probe(NULL, of_device_ids, NULL);
}
#include <linux/delay.h>
#include <linux/seq_file.h>
#include <linux/interrupt.h>
+#include <linux/of.h>
#include <linux/of_address.h>
-#include <linux/of_platform.h>
#include <asm/time.h>
#include <asm/machdep.h>
#include <linux/kdev_t.h>
#include <linux/delay.h>
#include <linux/seq_file.h>
-#include <linux/of_platform.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
#include <asm/time.h>
#include <asm/machdep.h>
#include <linux/delay.h>
#include <linux/seq_file.h>
#include <linux/interrupt.h>
-#include <linux/of_platform.h>
+#include <linux/of.h>
#include <asm/time.h>
#include <asm/machdep.h>
#include <linux/delay.h>
#include <linux/seq_file.h>
#include <linux/interrupt.h>
+#include <linux/of.h>
#include <linux/of_irq.h>
-#include <linux/of_platform.h>
#include <asm/time.h>
#include <asm/machdep.h>
#include <linux/seq_file.h>
#include <linux/initrd.h>
#include <linux/fsl_devices.h>
-#include <linux/of_platform.h>
-#include <linux/of_device.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
#include <linux/phy.h>
#include <linux/memblock.h>
#include <linux/fsl/guts.h>
#include <linux/delay.h>
#include <linux/seq_file.h>
#include <linux/interrupt.h>
-#include <linux/of_platform.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
#include <linux/fsl/guts.h>
#include <asm/time.h>
#include <linux/pci.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
-#include <linux/of_platform.h>
+#include <linux/of.h>
#include <asm/time.h>
#include <asm/machdep.h>
#include <linux/fsl/guts.h>
#include <linux/pci.h>
+#include <linux/of.h>
#include <linux/of_address.h>
-#include <linux/of_platform.h>
#include <asm/div64.h>
#include <asm/mpic.h>
#include <asm/swiotlb.h>
#include <linux/fsl/guts.h>
#include <linux/pci.h>
+#include <linux/of.h>
#include <linux/of_address.h>
-#include <linux/of_platform.h>
#include <asm/div64.h>
#include <asm/mpic.h>
#include <asm/swiotlb.h>
#include <linux/delay.h>
#include <linux/module.h>
#include <linux/fsl_devices.h>
+#include <linux/of.h>
#include <linux/of_address.h>
-#include <linux/of_platform.h>
-#include <linux/of_device.h>
#include <asm/time.h>
#include <asm/machdep.h>
#include "smp.h"
#include "mpc85xx.h"
-void __init qemu_e500_pic_init(void)
+static void __init qemu_e500_pic_init(void)
{
struct mpic *mpic;
unsigned int flags = MPIC_BIG_ENDIAN | MPIC_SINGLE_DEST_CPU |
#include <linux/kdev_t.h>
#include <linux/delay.h>
#include <linux/seq_file.h>
-#include <linux/of_platform.h>
+#include <linux/of.h>
#include <asm/time.h>
#include <asm/machdep.h>
#include <linux/irq.h>
#include <linux/of_address.h>
#include <linux/of_irq.h>
-#include <linux/of_platform.h>
#include <linux/io.h>
/*
#include <linux/kdev_t.h>
#include <linux/delay.h>
#include <linux/seq_file.h>
-#include <linux/of_platform.h>
+#include <linux/of.h>
#include <asm/time.h>
#include <asm/machdep.h>
#include <linux/kdev_t.h>
#include <linux/delay.h>
#include <linux/seq_file.h>
-#include <linux/of_platform.h>
+#include <linux/of.h>
#include <asm/time.h>
#include <asm/machdep.h>
#include <linux/errno.h>
#include <linux/fsl/guts.h>
#include <linux/pci.h>
-#include <linux/of_platform.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
#include <asm/pci-bridge.h>
#include <asm/udbg.h>
#include <linux/delay.h>
#include <linux/seq_file.h>
#include <linux/interrupt.h>
+#include <linux/of.h>
#include <linux/of_address.h>
-#include <linux/of_platform.h>
#include <asm/time.h>
#include <asm/machdep.h>
* Routines common to most mpc86xx-based boards.
*/
+#include <linux/init.h>
+#include <linux/mod_devicetable.h>
#include <linux/of_platform.h>
+#include <asm/reg.h>
#include <asm/synch.h>
#include "mpc86xx.h"
#include <linux/kdev_t.h>
#include <linux/delay.h>
#include <linux/seq_file.h>
+#include <linux/of.h>
#include <linux/of_address.h>
-#include <linux/of_platform.h>
#include <asm/time.h>
#include <asm/machdep.h>
#include <linux/kdev_t.h>
#include <linux/delay.h>
#include <linux/seq_file.h>
+#include <linux/of.h>
#include <linux/of_address.h>
-#include <linux/of_platform.h>
#include <asm/time.h>
#include <asm/machdep.h>
#include <linux/kdev_t.h>
#include <linux/delay.h>
#include <linux/seq_file.h>
+#include <linux/of.h>
#include <linux/of_address.h>
-#include <linux/of_platform.h>
#include <asm/time.h>
#include <asm/machdep.h>
#include <linux/pci.h>
#include <linux/of.h>
#include <linux/of_fdt.h>
-#include <linux/of_platform.h>
#include <linux/of_address.h>
#include <asm/udbg.h>
#include <asm/mpic.h>
#include <linux/stddef.h>
#include <linux/kernel.h>
#include <linux/interrupt.h>
+#include <linux/of.h>
#include <linux/of_irq.h>
-#include <linux/of_platform.h>
#include <asm/mpic.h>
#include <asm/i8259.h>
+#include "mpc86xx.h"
+
#ifdef CONFIG_PPC_I8259
static void mpc86xx_8259_cascade(struct irq_desc *desc)
{
#include <asm/time.h>
#include <asm/machdep.h>
#include <asm/cpm1.h>
-#include <asm/fs_pd.h>
+#include <asm/8xx_immap.h>
#include <asm/udbg.h>
#include "mpc8xx.h"
#include <asm/rheap.h>
#include <asm/cpm.h>
-#include <asm/fs_pd.h>
+#include <sysdev/fsl_soc.h>
#ifdef CONFIG_8xx_GPIO
#include <linux/gpio/legacy-of-mm-gpiochip.h>
void __init cpm_reset(void)
{
- sysconf8xx_t __iomem *siu_conf;
-
cpmp = &mpc8xx_immr->im_cpm;
#ifndef CONFIG_PPC_EARLY_DEBUG_CPM
* manual recommends it.
* Bit 25, FAM can also be set to use FEC aggressive mode (860T).
*/
- siu_conf = immr_map(im_siu_conf);
if ((mfspr(SPRN_IMMR) & 0xffff) == 0x0900) /* MPC885 */
- out_be32(&siu_conf->sc_sdcr, 0x40);
+ out_be32(&mpc8xx_immr->im_siu_conf.sc_sdcr, 0x40);
else
- out_be32(&siu_conf->sc_sdcr, 1);
- immr_unmap(siu_conf);
+ out_be32(&mpc8xx_immr->im_siu_conf.sc_sdcr, 1);
}
static DEFINE_SPINLOCK(cmd_lock);
#include <asm/io.h>
#include <asm/8xx_immap.h>
-#include <asm/fs_pd.h>
#include <mm/mmu_decl.h>
#include "pic.h"
return IRQ_HANDLED;
}
-/* per-board overridable init_internal_rtc() function. */
-void __init __attribute__ ((weak))
-init_internal_rtc(void)
-{
- sit8xx_t __iomem *sys_tmr = immr_map(im_sit);
-
- /* Disable the RTC one second and alarm interrupts. */
- clrbits16(&sys_tmr->sit_rtcsc, (RTCSC_SIE | RTCSC_ALE));
-
- /* Enable the RTC */
- setbits16(&sys_tmr->sit_rtcsc, (RTCSC_RTF | RTCSC_RTE));
- immr_unmap(sys_tmr);
-}
-
static int __init get_freq(char *name, unsigned long *val)
{
struct device_node *cpu;
void __init mpc8xx_calibrate_decr(void)
{
struct device_node *cpu;
- cark8xx_t __iomem *clk_r1;
- car8xx_t __iomem *clk_r2;
- sitk8xx_t __iomem *sys_tmr1;
- sit8xx_t __iomem *sys_tmr2;
int irq, virq;
- clk_r1 = immr_map(im_clkrstk);
-
/* Unlock the SCCR. */
- out_be32(&clk_r1->cark_sccrk, ~KAPWR_KEY);
- out_be32(&clk_r1->cark_sccrk, KAPWR_KEY);
- immr_unmap(clk_r1);
+ out_be32(&mpc8xx_immr->im_clkrstk.cark_sccrk, ~KAPWR_KEY);
+ out_be32(&mpc8xx_immr->im_clkrstk.cark_sccrk, KAPWR_KEY);
/* Force all 8xx processors to use divide by 16 processor clock. */
- clk_r2 = immr_map(im_clkrst);
- setbits32(&clk_r2->car_sccr, 0x02000000);
- immr_unmap(clk_r2);
+ setbits32(&mpc8xx_immr->im_clkrst.car_sccr, 0x02000000);
/* Processor frequency is MHz.
*/
* we guarantee the registers are locked, then we unlock them
* for our use.
*/
- sys_tmr1 = immr_map(im_sitk);
- out_be32(&sys_tmr1->sitk_tbscrk, ~KAPWR_KEY);
- out_be32(&sys_tmr1->sitk_rtcsck, ~KAPWR_KEY);
- out_be32(&sys_tmr1->sitk_tbk, ~KAPWR_KEY);
- out_be32(&sys_tmr1->sitk_tbscrk, KAPWR_KEY);
- out_be32(&sys_tmr1->sitk_rtcsck, KAPWR_KEY);
- out_be32(&sys_tmr1->sitk_tbk, KAPWR_KEY);
- immr_unmap(sys_tmr1);
+ out_be32(&mpc8xx_immr->im_sitk.sitk_tbscrk, ~KAPWR_KEY);
+ out_be32(&mpc8xx_immr->im_sitk.sitk_rtcsck, ~KAPWR_KEY);
+ out_be32(&mpc8xx_immr->im_sitk.sitk_tbk, ~KAPWR_KEY);
+ out_be32(&mpc8xx_immr->im_sitk.sitk_tbscrk, KAPWR_KEY);
+ out_be32(&mpc8xx_immr->im_sitk.sitk_rtcsck, KAPWR_KEY);
+ out_be32(&mpc8xx_immr->im_sitk.sitk_tbk, KAPWR_KEY);
+
+ /* Disable the RTC one second and alarm interrupts. */
+ clrbits16(&mpc8xx_immr->im_sit.sit_rtcsc, (RTCSC_SIE | RTCSC_ALE));
- init_internal_rtc();
+ /* Enable the RTC */
+ setbits16(&mpc8xx_immr->im_sit.sit_rtcsc, (RTCSC_RTF | RTCSC_RTE));
/* Enabling the decrementer also enables the timebase interrupts
* (or from the other point of view, to get decrementer interrupts
of_node_put(cpu);
irq = virq_to_hw(virq);
- sys_tmr2 = immr_map(im_sit);
- out_be16(&sys_tmr2->sit_tbscr, ((1 << (7 - (irq/2))) << 8) |
- (TBSCR_TBF | TBSCR_TBE));
- immr_unmap(sys_tmr2);
+ out_be16(&mpc8xx_immr->im_sit.sit_tbscr,
+ ((1 << (7 - (irq / 2))) << 8) | (TBSCR_TBF | TBSCR_TBE));
if (request_irq(virq, timebase_interrupt, IRQF_NO_THREAD, "tbint",
NULL))
int mpc8xx_set_rtc_time(struct rtc_time *tm)
{
- sitk8xx_t __iomem *sys_tmr1;
- sit8xx_t __iomem *sys_tmr2;
time64_t time;
- sys_tmr1 = immr_map(im_sitk);
- sys_tmr2 = immr_map(im_sit);
time = rtc_tm_to_time64(tm);
- out_be32(&sys_tmr1->sitk_rtck, KAPWR_KEY);
- out_be32(&sys_tmr2->sit_rtc, (u32)time);
- out_be32(&sys_tmr1->sitk_rtck, ~KAPWR_KEY);
+ out_be32(&mpc8xx_immr->im_sitk.sitk_rtck, KAPWR_KEY);
+ out_be32(&mpc8xx_immr->im_sit.sit_rtc, (u32)time);
+ out_be32(&mpc8xx_immr->im_sitk.sitk_rtck, ~KAPWR_KEY);
- immr_unmap(sys_tmr2);
- immr_unmap(sys_tmr1);
return 0;
}
void mpc8xx_get_rtc_time(struct rtc_time *tm)
{
unsigned long data;
- sit8xx_t __iomem *sys_tmr = immr_map(im_sit);
/* Get time from the RTC. */
- data = in_be32(&sys_tmr->sit_rtc);
+ data = in_be32(&mpc8xx_immr->im_sit.sit_rtc);
rtc_time64_to_tm(data, tm);
- immr_unmap(sys_tmr);
return;
}
void __noreturn mpc8xx_restart(char *cmd)
{
- car8xx_t __iomem *clk_r = immr_map(im_clkrst);
-
-
local_irq_disable();
- setbits32(&clk_r->car_plprcr, 0x00000080);
+ setbits32(&mpc8xx_immr->im_clkrst.car_plprcr, 0x00000080);
/* Clear the ME bit in MSR to cause checkstop on machine check
*/
mtmsr(mfmsr() & ~0x1000);
- in_8(&clk_r->res[0]);
+ in_8(&mpc8xx_immr->im_clkrst.res[0]);
panic("Restart failed\n");
}
#include <asm/time.h>
#include <asm/8xx_immap.h>
#include <asm/cpm1.h>
-#include <asm/fs_pd.h>
#include <asm/udbg.h>
#include "mpc86xads.h"
#include <asm/time.h>
#include <asm/8xx_immap.h>
#include <asm/cpm1.h>
-#include <asm/fs_pd.h>
#include <asm/udbg.h>
#include "mpc885ads.h"
#include <asm/time.h>
#include <asm/8xx_immap.h>
#include <asm/cpm1.h>
-#include <asm/fs_pd.h>
#include <asm/udbg.h>
#include "mpc8xx.h"
config CPM2
bool "Enable support for the CPM2 (Communications Processor Module)"
- depends on (FSL_SOC_BOOKE && PPC32) || 8260
+ depends on (FSL_SOC_BOOKE && PPC32) || PPC_82xx
select CPM
select HAVE_PCI
select GPIOLIB
default "e500mc" if E500MC_CPU
default "powerpc" if POWERPC_CPU
+config TUNE_CPU
+ string
+ depends on POWERPC64_CPU
+ default "-mtune=power10" if $(cc-option,-mtune=power10)
+ default "-mtune=power9" if $(cc-option,-mtune=power9)
+ default "-mtune=power8" if $(cc-option,-mtune=power8)
+
config PPC_BOOK3S
def_bool y
depends on PPC_BOOK3S_32 || PPC_BOOK3S_64
#include <linux/pci.h>
#include <linux/msi.h>
#include <linux/export.h>
-#include <linux/of_platform.h>
#include <linux/slab.h>
#include <linux/debugfs.h>
+#include <linux/of.h>
#include <linux/of_irq.h>
+#include <linux/platform_device.h>
#include <asm/dcr.h>
#include <asm/machdep.h>
#include <linux/percpu.h>
#include <linux/types.h>
#include <linux/export.h>
+#include <linux/of.h>
#include <linux/of_address.h>
-#include <linux/of_device.h>
-#include <linux/of_platform.h>
#include <linux/pgtable.h>
#include <asm/io.h>
#include <linux/notifier.h>
#include <linux/of.h>
#include <linux/of_address.h>
-#include <linux/of_platform.h>
+#include <linux/platform_device.h>
#include <linux/slab.h>
#include <linux/memblock.h>
#include <asm/cell-regs.h>
#include "ras.h"
-
+#include "pervasive.h"
static void dump_fir(int cpu)
{
#include <linux/mutex.h>
#include <linux/memory_hotplug.h>
#include <linux/of_platform.h>
+#include <linux/platform_device.h>
#include <asm/mmu.h>
#include <asm/processor.h>
#include <linux/kernel.h>
#include <linux/of_address.h>
-#include <linux/of_platform.h>
#include <linux/slab.h>
#include <linux/io.h>
#include "spufs/spufs.h"
#include "interrupt.h"
+#include "spu_priv1_mmio.h"
struct device_node *spu_devnode(struct spu *spu)
{
#include <linux/serial.h>
#include <linux/tty.h>
#include <linux/serial_core.h>
+#include <linux/of.h>
#include <linux/of_address.h>
#include <linux/of_irq.h>
-#include <linux/of_platform.h>
#include <linux/extable.h>
#include <asm/time.h>
#include <linux/serial.h>
#include <linux/smp.h>
#include <linux/bitops.h>
+#include <linux/of.h>
#include <linux/of_address.h>
-#include <linux/of_device.h>
+#include <linux/platform_device.h>
#include <linux/memblock.h>
#include <asm/processor.h>
#include <linux/phy.h>
#include <linux/of_address.h>
#include <linux/of_mdio.h>
-#include <linux/of_platform.h>
+#include <linux/platform_device.h>
#define DELAY 1
extern time64_t pas_get_boot_time(void);
extern void pas_pci_init(void);
+struct pci_dev;
extern void pas_pci_irq_fixup(struct pci_dev *dev);
extern void pas_pci_dma_dev_setup(struct pci_dev *dev);
#include <linux/console.h>
#include <linux/export.h>
#include <linux/pci.h>
+#include <linux/of.h>
#include <linux/of_platform.h>
+#include <linux/platform_device.h>
#include <linux/gfp.h>
#include <linux/irqdomain.h>
#include <asm/time.h>
+#include "pasemi.h"
+
time64_t __init pas_get_boot_time(void)
{
/* Let's just return a fake date right now */
#include <asm/pci-bridge.h>
#include <asm/pmac_low_i2c.h>
+#include "pmac.h"
+
#undef DEBUG_FEATURE
#ifdef DEBUG_FEATURE
* Here are the chip specific feature functions
*/
-static inline int simple_feature_tweak(struct device_node *node, int type,
- int reg, u32 mask, int value)
+#ifndef CONFIG_PPC64
+
+static int simple_feature_tweak(struct device_node *node, int type, int reg,
+ u32 mask, int value)
{
struct macio_chip* macio;
unsigned long flags;
return 0;
}
-#ifndef CONFIG_PPC64
-
static long ohare_htw_scc_enable(struct device_node *node, long param,
long value)
{
#include <linux/root_dev.h>
#include <linux/bitops.h>
#include <linux/suspend.h>
-#include <linux/of_device.h>
+#include <linux/of.h>
#include <linux/of_platform.h>
#include <asm/reg.h>
struct pci_controller *hose = pci_bus_to_host(pdev->bus);
struct pnv_phb *phb = hose->private_data;
struct device_node *dn = pci_device_to_OF_node(pdev);
- uint64_t id = PCI_SLOT_ID(phb->opal_id,
- (pdev->bus->number << 8) | pdev->devfn);
+ uint64_t id = PCI_SLOT_ID(phb->opal_id, pci_dev_id(pdev));
uint8_t scope;
int64_t rc;
if (!data)
return -ENOMEM;
- bdfn = (dev->bus->number << 8) | dev->devfn;
+ bdfn = pci_dev_id(dev);
rc = opal_npu_spa_setup(phb->opal_id, bdfn, virt_to_phys(spa_mem),
PE_mask);
if (rc) {
#include <linux/platform_device.h>
#include <linux/of.h>
#include <linux/of_address.h>
-#include <linux/of_platform.h>
#include <linux/crash_dump.h>
#include <linux/debugfs.h>
#include <asm/opal.h>
#include <linux/uaccess.h>
+struct opal_prd_msg {
+ union {
+ struct opal_prd_msg_header header;
+ DECLARE_FLEX_ARRAY(u8, data);
+ };
+};
+
/*
* The msg member must be at the end of the struct, as it's followed by the
* message data.
*/
struct opal_prd_msg_queue_item {
- struct list_head list;
- struct opal_prd_msg_header msg;
+ struct list_head list;
+ struct opal_prd_msg msg;
};
static struct device_node *prd_node;
int rc;
/* we need at least a header's worth of data */
- if (count < sizeof(item->msg))
+ if (count < sizeof(item->msg.header))
return -EINVAL;
if (*ppos)
return -EINTR;
}
- size = be16_to_cpu(item->msg.size);
+ size = be16_to_cpu(item->msg.header.size);
if (size > count) {
err = -EINVAL;
goto err_requeue;
size_t count, loff_t *ppos)
{
struct opal_prd_msg_header hdr;
+ struct opal_prd_msg *msg;
ssize_t size;
- void *msg;
int rc;
size = sizeof(hdr);
static int opal_prd_release(struct inode *inode, struct file *file)
{
- struct opal_prd_msg_header msg;
+ struct opal_prd_msg msg;
- msg.size = cpu_to_be16(sizeof(msg));
- msg.type = OPAL_PRD_MSG_TYPE_FINI;
+ msg.header.size = cpu_to_be16(sizeof(msg));
+ msg.header.type = OPAL_PRD_MSG_TYPE_FINI;
- opal_prd_msg((struct opal_prd_msg *)&msg);
+ opal_prd_msg(&msg);
atomic_xchg(&prd_usage, 0);
if (!item)
return -ENOMEM;
- memcpy(&item->msg, msg->params, msg_size);
+ memcpy(&item->msg.data, msg->params, msg_size);
spin_lock_irqsave(&opal_prd_msg_queue_lock, flags);
list_add_tail(&item->list, &opal_prd_msg_queue);
#include <linux/bcd.h>
#include <linux/rtc.h>
#include <linux/delay.h>
-#include <linux/platform_device.h>
+#include <linux/of.h>
#include <linux/of_platform.h>
+#include <linux/platform_device.h>
#include <asm/opal.h>
#include <asm/firmware.h>
#define pr_fmt(fmt) "secvar: "fmt
#include <linux/types.h>
+#include <linux/of.h>
#include <linux/platform_device.h>
-#include <linux/of_platform.h>
#include <asm/opal.h>
#include <asm/secvar.h>
#include <asm/secure_boot.h>
*/
#include <linux/delay.h>
+#include <linux/of.h>
#include <linux/of_platform.h>
+#include <linux/platform_device.h>
#include <asm/opal.h>
#include <asm/machdep.h>
ent->path.size = strlen((char *)ent->path.data);
dir = debugfs_create_dir(ent->name, root);
- if (!dir) {
+ if (IS_ERR(dir)) {
kfree(ent->path.data);
kfree(ent);
return -1;
return 0;
root = debugfs_create_dir("scom", arch_debugfs_dir);
- if (!root)
+ if (IS_ERR(root))
return -1;
rc = 0;
struct pnv_ioda_pe *pe;
/* Check if the BDFN for this device is associated with a PE yet */
- pe = pnv_pci_bdfn_to_pe(phb, pdev->devfn | (pdev->bus->number << 8));
+ pe = pnv_pci_bdfn_to_pe(phb, pci_dev_id(pdev));
if (!pe) {
/* VF PEs should be pre-configured in pnv_pci_sriov_enable() */
if (WARN_ON(pdev->is_virtfn))
return;
pnv_pci_configure_bus(pdev->bus);
- pe = pnv_pci_bdfn_to_pe(phb, pdev->devfn | (pdev->bus->number << 8));
+ pe = pnv_pci_bdfn_to_pe(phb, pci_dev_id(pdev));
pci_info(pdev, "Configured PE#%x\n", pe ? pe->pe_number : 0xfffff);
if (WARN_ON(!phb))
return ERR_PTR(-ENODEV);
- pe = pnv_pci_bdfn_to_pe(phb, pdev->devfn | (pdev->bus->number << 8));
+ pe = pnv_pci_bdfn_to_pe(phb, pci_dev_id(pdev));
if (!pe)
return ERR_PTR(-ENODEV);
#ifdef CONFIG_MEMORY_HOTPLUG
static unsigned long pnv_memory_block_size(void)
{
- /*
- * We map the kernel linear region with 1GB large pages on radix. For
- * memory hot unplug to work our memory block size must be at least
- * this size.
- */
- if (radix_enabled())
- return radix_mem_block_size;
- else
- return 256UL * 1024 * 1024;
+ return memory_block_size;
}
#endif
static u64 make_first_field(const char *text, u64 index)
{
- u64 n;
+ u64 n = 0;
- strncpy((char *)&n, text, 8);
+ memcpy((char *)&n, text, strnlen(text, sizeof(n)));
return PS3_VENDOR_ID_NONE + (n >> 32) + index;
}
for_each_present_cpu(cpu) {
if (get_hard_smp_processor_id(cpu) != thread)
continue;
+
+ if (!topology_is_primary_thread(cpu)) {
+ if (cpu_smt_control != CPU_SMT_ENABLED)
+ break;
+ if (!topology_smt_thread_allowed(cpu))
+ break;
+ }
+
cpu_maps_update_done();
find_and_update_cpu_nid(cpu);
rc = device_online(get_cpu_device(cpu));
.notifier_call = pseries_smp_notifier,
};
-static int __init pseries_cpu_hotplug_init(void)
+void __init pseries_cpu_hotplug_init(void)
{
int qcss_tok;
- unsigned int node;
-
-#ifdef CONFIG_ARCH_CPU_PROBE_RELEASE
- ppc_md.cpu_probe = dlpar_cpu_probe;
- ppc_md.cpu_release = dlpar_cpu_release;
-#endif /* CONFIG_ARCH_CPU_PROBE_RELEASE */
rtas_stop_self_token = rtas_function_token(RTAS_FN_STOP_SELF);
qcss_tok = rtas_function_token(RTAS_FN_QUERY_CPU_STOPPED_STATE);
qcss_tok == RTAS_UNKNOWN_SERVICE) {
printk(KERN_INFO "CPU Hotplug not supported by firmware "
"- disabling.\n");
- return 0;
+ return;
}
smp_ops->cpu_offline_self = pseries_cpu_offline_self;
smp_ops->cpu_disable = pseries_cpu_disable;
smp_ops->cpu_die = pseries_cpu_die;
+}
+
+static int __init pseries_dlpar_init(void)
+{
+ unsigned int node;
+
+#ifdef CONFIG_ARCH_CPU_PROBE_RELEASE
+ ppc_md.cpu_probe = dlpar_cpu_probe;
+ ppc_md.cpu_release = dlpar_cpu_release;
+#endif /* CONFIG_ARCH_CPU_PROBE_RELEASE */
/* Processors can be added/removed only on LPAR */
if (firmware_has_feature(FW_FEATURE_LPAR)) {
return 0;
}
-machine_arch_initcall(pseries, pseries_cpu_hotplug_init);
+machine_arch_initcall(pseries, pseries_dlpar_init);
#include <asm/drmem.h>
#include "pseries.h"
-unsigned long pseries_memory_block_size(void)
-{
- struct device_node *np;
- u64 memblock_size = MIN_MEMORY_BLOCK_SIZE;
- struct resource r;
-
- np = of_find_node_by_path("/ibm,dynamic-reconfiguration-memory");
- if (np) {
- int len;
- int size_cells;
- const __be32 *prop;
-
- size_cells = of_n_size_cells(np);
-
- prop = of_get_property(np, "ibm,lmb-size", &len);
- if (prop && len >= size_cells * sizeof(__be32))
- memblock_size = of_read_number(prop, size_cells);
- of_node_put(np);
-
- } else if (machine_is(pseries)) {
- /* This fallback really only applies to pseries */
- unsigned int memzero_size = 0;
-
- np = of_find_node_by_path("/memory@0");
- if (np) {
- if (!of_address_to_resource(np, 0, &r))
- memzero_size = resource_size(&r);
- of_node_put(np);
- }
-
- if (memzero_size) {
- /* We now know the size of memory@0, use this to find
- * the first memoryblock and get its size.
- */
- char buf[64];
-
- sprintf(buf, "/memory@%x", memzero_size);
- np = of_find_node_by_path(buf);
- if (np) {
- if (!of_address_to_resource(np, 0, &r))
- memblock_size = resource_size(&r);
- of_node_put(np);
- }
- }
- }
- return memblock_size;
-}
-
static void dlpar_free_property(struct property *prop)
{
kfree(prop->name);
static int pseries_remove_memblock(unsigned long base, unsigned long memblock_size)
{
- unsigned long block_sz, start_pfn;
+ unsigned long start_pfn;
int sections_per_block;
int i;
if (!pfn_valid(start_pfn))
goto out;
- block_sz = pseries_memory_block_size();
- sections_per_block = block_sz / MIN_MEMORY_BLOCK_SIZE;
+ sections_per_block = memory_block_size / MIN_MEMORY_BLOCK_SIZE;
for (i = 0; i < sections_per_block; i++) {
__remove_memory(base, MIN_MEMORY_BLOCK_SIZE);
static int dlpar_remove_lmb(struct drmem_lmb *lmb)
{
struct memory_block *mem_block;
- unsigned long block_sz;
int rc;
if (!lmb_is_removable(lmb))
return rc;
}
- block_sz = pseries_memory_block_size();
-
- __remove_memory(lmb->base_addr, block_sz);
+ __remove_memory(lmb->base_addr, memory_block_size);
put_device(&mem_block->dev);
/* Update memory regions for memory remove */
- memblock_remove(lmb->base_addr, block_sz);
+ memblock_remove(lmb->base_addr, memory_block_size);
invalidate_lmb_associativity_index(lmb);
lmb->flags &= ~DRCONF_MEM_ASSIGNED;
b 1f; \
END_FTR_SECTION(0, 1); \
LOAD_REG_ADDR(r12, hcall_tracepoint_refcount) ; \
- std r12,32(r1); \
+ ld r12,0(r12); \
cmpdi r12,0; \
bne- LABEL; \
1:
#include <linux/slab.h>
#include <linux/stat.h>
#include <linux/of_platform.h>
+#include <linux/platform_device.h>
#include <asm/ibmebus.h>
#include <asm/machdep.h>
if (err) {
printk(KERN_WARNING "%s: device_register returned %i\n",
__func__, err);
+ put_device(&ibmebus_bus_device);
bus_unregister(&ibmebus_bus_type);
return err;
static DEFINE_SPINLOCK(dma_win_list_lock);
/* protects initializing window twice for same device */
static DEFINE_MUTEX(dma_win_init_mutex);
-#define DIRECT64_PROPNAME "linux,direct64-ddr-window-info"
-#define DMA64_PROPNAME "linux,dma64-ddr-window-info"
static int tce_clearrange_multi_pSeriesLP(unsigned long start_pfn,
unsigned long num_pfn, const void *arg)
#include <asm/kexec.h>
#include <asm/fadump.h>
#include <asm/dtl.h>
+#include <asm/vphn.h>
#include "pseries.h"
static int __init vcpudispatch_stats_procfs_init(void)
{
- /*
- * Avoid smp_processor_id while preemptible. All CPUs should have
- * the same value for lppaca_shared_proc.
- */
- preempt_disable();
- if (!lppaca_shared_proc(get_lppaca())) {
- preempt_enable();
+ if (!lppaca_shared_proc())
return 0;
- }
- preempt_enable();
if (!proc_create("powerpc/vcpudispatch_stats", 0600, NULL,
&vcpudispatch_stats_proc_ops))
ppp_data.active_system_procs);
/* pool related entries are appropriate for shared configs */
- if (lppaca_shared_proc(get_lppaca())) {
+ if (lppaca_shared_proc()) {
unsigned long pool_idle_time, pool_procs;
seq_printf(m, "pool=%d\n", ppp_data.pool_num);
partition_potential_processors);
seq_printf(m, "shared_processor_mode=%d\n",
- lppaca_shared_proc(get_lppaca()));
+ lppaca_shared_proc());
#ifdef CONFIG_PPC_64S_HASH_MMU
if (!radix_enabled())
return auth;
}
-/**
+/*
* Label is combination of label attributes + name.
* Label attributes are used internally by kernel and not exposed to the user.
*/
#ifdef CONFIG_HOTPLUG_CPU
int dlpar_cpu(struct pseries_hp_errorlog *hp_elog);
+void pseries_cpu_hotplug_init(void);
#else
static inline int dlpar_cpu(struct pseries_hp_errorlog *hp_elog)
{
return -EOPNOTSUPP;
}
+static inline void pseries_cpu_hotplug_init(void) { }
#endif
/* PCI root bridge prepare function override for pseries */
int pseries_msi_allocate_domains(struct pci_controller *phb);
void pseries_msi_free_domains(struct pci_controller *phb);
-unsigned long pseries_memory_block_size(void);
-
extern int CMO_PrPSP;
extern int CMO_SecPSP;
extern unsigned long CMO_PageSize;
/* Discover PIC type and setup ppc_md accordingly */
smp_init_pseries();
+ // Setup CPU hotplug callbacks
+ pseries_cpu_hotplug_init();
if (radix_enabled() && !mmu_has_feature(MMU_FTR_GTSE))
if (!firmware_has_feature(FW_FEATURE_RPT_INVALIDATE))
if (firmware_has_feature(FW_FEATURE_LPAR)) {
vpa_init(boot_cpuid);
- if (lppaca_shared_proc(get_lppaca())) {
+ if (lppaca_shared_proc()) {
static_branch_enable(&shared_processor);
pv_spinlocks_init();
#ifdef CONFIG_PARAVIRT_TIME_ACCOUNTING
return PCI_PROBE_NORMAL;
}
+#ifdef CONFIG_MEMORY_HOTPLUG
+static unsigned long pseries_memory_block_size(void)
+{
+ return memory_block_size;
+}
+#endif
+
struct pci_controller_ops pseries_pci_controller_ops = {
.probe_mode = pSeries_pci_probe_mode,
#ifdef CONFIG_SPAPR_TCE_IOMMU
#include <asm/hvcall.h>
#include <asm/plpar_wrappers.h>
#include <asm/firmware.h>
+#include <asm/vphn.h>
#include <asm/vas.h>
#include "vas.h"
// SPDX-License-Identifier: GPL-2.0
#include <asm/byteorder.h>
-#include <asm/lppaca.h>
+#include <asm/vphn.h>
/*
* The associativity domain numbers are returned from the hypervisor as a
#include <asm/io.h>
#include <asm/irq.h>
-#include <asm/mpc8260.h>
#include <asm/page.h>
#include <asm/cpm2.h>
#include <asm/rheap.h>
-#include <asm/fs_pd.h>
#include <sysdev/fsl_soc.h>
/* This is good enough to get SMCs running.....
*/
if (brg < 4) {
- bp = cpm2_map_size(im_brgc1, 16);
+ bp = &cpm2_immr->im_brgc1;
} else {
- bp = cpm2_map_size(im_brgc5, 16);
+ bp = &cpm2_immr->im_brgc5;
brg -= 4;
}
bp += brg;
val |= CPM_BRG_DIV16;
out_be32(bp, val);
- cpm2_unmap(bp);
}
EXPORT_SYMBOL(__cpm2_setbrg);
int ret = 0;
int shift;
int i, bits = 0;
- cpmux_t __iomem *im_cpmux;
u32 __iomem *reg;
u32 mask = 7;
{CPM_CLK_SCC4, CPM_CLK8, 7},
};
- im_cpmux = cpm2_map(im_cpmux);
-
switch (target) {
case CPM_CLK_SCC1:
- reg = &im_cpmux->cmx_scr;
+ reg = &cpm2_immr->im_cpmux.cmx_scr;
shift = 24;
break;
case CPM_CLK_SCC2:
- reg = &im_cpmux->cmx_scr;
+ reg = &cpm2_immr->im_cpmux.cmx_scr;
shift = 16;
break;
case CPM_CLK_SCC3:
- reg = &im_cpmux->cmx_scr;
+ reg = &cpm2_immr->im_cpmux.cmx_scr;
shift = 8;
break;
case CPM_CLK_SCC4:
- reg = &im_cpmux->cmx_scr;
+ reg = &cpm2_immr->im_cpmux.cmx_scr;
shift = 0;
break;
case CPM_CLK_FCC1:
- reg = &im_cpmux->cmx_fcr;
+ reg = &cpm2_immr->im_cpmux.cmx_fcr;
shift = 24;
break;
case CPM_CLK_FCC2:
- reg = &im_cpmux->cmx_fcr;
+ reg = &cpm2_immr->im_cpmux.cmx_fcr;
shift = 16;
break;
case CPM_CLK_FCC3:
- reg = &im_cpmux->cmx_fcr;
+ reg = &cpm2_immr->im_cpmux.cmx_fcr;
shift = 8;
break;
default:
out_be32(reg, (in_be32(reg) & ~mask) | bits);
- cpm2_unmap(im_cpmux);
return ret;
}
int ret = 0;
int shift;
int i, bits = 0;
- cpmux_t __iomem *im_cpmux;
u8 __iomem *reg;
u8 mask = 3;
{CPM_CLK_SMC2, CPM_CLK15, 3},
};
- im_cpmux = cpm2_map(im_cpmux);
-
switch (target) {
case CPM_CLK_SMC1:
- reg = &im_cpmux->cmx_smr;
+ reg = &cpm2_immr->im_cpmux.cmx_smr;
mask = 3;
shift = 4;
break;
case CPM_CLK_SMC2:
- reg = &im_cpmux->cmx_smr;
+ reg = &cpm2_immr->im_cpmux.cmx_smr;
mask = 3;
shift = 0;
break;
out_8(reg, (in_8(reg) & ~mask) | bits);
- cpm2_unmap(im_cpmux);
return ret;
}
#include <linux/irqdomain.h>
#include <asm/immap_cpm2.h>
-#include <asm/mpc8260.h>
#include <asm/io.h>
-#include <asm/fs_pd.h>
#include "cpm2_pic.h"
{
int i;
- cpm2_intctl = cpm2_map(im_intctl);
+ cpm2_intctl = &cpm2_immr->im_intctl;
/* Clear the CPM IRQ controller, in case it has any bits set
* from the bootloader
*/
#include <linux/init.h>
-#include <linux/of_device.h>
#include <linux/spinlock.h>
#include <linux/export.h>
#include <linux/of.h>
-#include <linux/of_address.h>
#include <linux/slab.h>
#include <asm/udbg.h>
*/
#include <linux/module.h>
-#include <linux/of_device.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
#include <asm/cpm.h>
#ifdef CONFIG_8xx_GPIO
* Copyright (c) 2004 Eugene Surovegin <ebs@ebshome.net>
*/
+#include <linux/export.h>
#include <asm/ppc_asm.h>
#include <asm/processor.h>
#include <asm/bug.h>
-#include <asm/export.h>
#define DCR_ACCESS_PROLOG(table) \
cmplwi cr0,r3,1024; \
* Linux descriptor level callbacks
*/
-void ehv_pic_unmask_irq(struct irq_data *d)
+static void ehv_pic_unmask_irq(struct irq_data *d)
{
unsigned int src = virq_to_hw(d->irq);
ev_int_set_mask(src, 0);
}
-void ehv_pic_mask_irq(struct irq_data *d)
+static void ehv_pic_mask_irq(struct irq_data *d)
{
unsigned int src = virq_to_hw(d->irq);
ev_int_set_mask(src, 1);
}
-void ehv_pic_end_irq(struct irq_data *d)
+static void ehv_pic_end_irq(struct irq_data *d)
{
unsigned int src = virq_to_hw(d->irq);
ev_int_eoi(src);
}
-void ehv_pic_direct_end_irq(struct irq_data *d)
+static void ehv_pic_direct_end_irq(struct irq_data *d)
{
out_be32(mpic_percpu_base_vaddr + MPIC_EOI / 4, 0);
}
-int ehv_pic_set_affinity(struct irq_data *d, const struct cpumask *dest,
+static int ehv_pic_set_affinity(struct irq_data *d, const struct cpumask *dest,
bool force)
{
unsigned int src = virq_to_hw(d->irq);
}
}
-int ehv_pic_set_irq_type(struct irq_data *d, unsigned int flow_type)
+static int ehv_pic_set_irq_type(struct irq_data *d, unsigned int flow_type)
{
unsigned int src = virq_to_hw(d->irq);
unsigned int vecpri, vold, vnew, prio, cpu_dest;
}
}
-int fsl_add_bridge(struct platform_device *pdev, int is_primary)
+static int fsl_add_bridge(struct platform_device *pdev, int is_primary)
{
int len;
struct pci_controller *hose;
u32 cfg_bar;
int ret = -ENOMEM;
- pcie = zalloc_maybe_bootmem(sizeof(*pcie), GFP_KERNEL);
+ pcie = kzalloc(sizeof(*pcie), GFP_KERNEL);
if (!pcie)
return ret;
};
-extern int fsl_add_bridge(struct platform_device *pdev, int is_primary);
extern void fsl_pcibios_fixup_bus(struct pci_bus *bus);
extern void fsl_pcibios_fixup_phb(struct pci_controller *phb);
extern int mpc83xx_add_bridge(struct device_node *dev);
#include <linux/export.h>
#include <linux/suspend.h>
#include <linux/delay.h>
-#include <linux/device.h>
+#include <linux/mod_devicetable.h>
#include <linux/of_address.h>
-#include <linux/of_platform.h>
+#include <linux/platform_device.h>
struct pmc_regs {
__be32 devdisr;
#include <linux/types.h>
#include <linux/dma-mapping.h>
#include <linux/interrupt.h>
-#include <linux/device.h>
+#include <linux/of.h>
#include <linux/of_address.h>
#include <linux/of_irq.h>
-#include <linux/of_platform.h>
+#include <linux/platform_device.h>
#include <linux/delay.h>
#include <linux/slab.h>
#include <linux/io.h>
#include <linux/uaccess.h>
#include <asm/machdep.h>
+#include <asm/rio.h>
#include "fsl_rio.h"
out_be32(&priv->inb_atmu_regs[i].riwar, 0);
}
-int fsl_map_inb_mem(struct rio_mport *mport, dma_addr_t lstart,
- u64 rstart, u64 size, u32 flags)
+static int fsl_map_inb_mem(struct rio_mport *mport, dma_addr_t lstart,
+ u64 rstart, u64 size, u32 flags)
{
struct rio_priv *priv = mport->priv;
u32 base_size;
return 0;
}
-void fsl_unmap_inb_mem(struct rio_mport *mport, dma_addr_t lstart)
+static void fsl_unmap_inb_mem(struct rio_mport *mport, dma_addr_t lstart)
{
u32 win_start_shift, base_start_shift;
struct rio_priv *priv = mport->priv;
* master port with system-specific info, and registers the
* master port with the RapidIO subsystem.
*/
-int fsl_rio_setup(struct platform_device *dev)
+static int fsl_rio_setup(struct platform_device *dev)
{
struct rio_ops *ops;
struct rio_mport *port;
#include <linux/interrupt.h>
#include <linux/of_address.h>
#include <linux/of_irq.h>
-#include <linux/of_platform.h>
#include <linux/slab.h>
#include "fsl_rio.h"
return IRQ_HANDLED;
}
-void msg_unit_error_handler(void)
+static void msg_unit_error_handler(void)
{
/*XXX: Error recovery is not implemented, we just clear errors */
#include <linux/device.h>
#include <linux/platform_device.h>
#include <linux/of.h>
-#include <linux/of_platform.h>
#include <linux/phy.h>
#include <linux/spi/spi.h>
#include <linux/fsl_devices.h>
fwnode_for_each_parent_node(fwnode, parent) {
ret = fwnode_property_read_u32(parent, "bus-frequency", &bus_freq);
- if (!ret)
+ if (!ret) {
+ fwnode_handle_put(parent);
return bus_freq;
+ }
}
return 0;
*/
#include <linux/list.h>
+#include <linux/of.h>
#include <linux/of_address.h>
#include <linux/of_irq.h>
-#include <linux/of_platform.h>
+#include <linux/platform_device.h>
#include <linux/errno.h>
#include <linux/err.h>
#include <linux/export.h>
#include <linux/slab.h>
#include <linux/of.h>
#include <linux/of_address.h>
-#include <linux/of_device.h>
#include <linux/of_irq.h>
#include <linux/syscore_ops.h>
#include <sysdev/fsl_soc.h>
* Copyright 2007 David Gibson <dwg@au1.ibm.com>, IBM Corporation.
*/
#include <linux/kernel.h>
-#include <linux/of.h>
#include <linux/init.h>
+#include <linux/of.h>
#include <linux/of_address.h>
-#include <linux/of_platform.h>
+#include <linux/platform_device.h>
#include <linux/slab.h>
#include <asm/prom.h>
#include <linux/completion.h>
#include <linux/spinlock.h>
#include <linux/module.h>
+#include <linux/mod_devicetable.h>
#include <linux/workqueue.h>
#include <linux/of_address.h>
-#include <linux/of_device.h>
#include <linux/of_irq.h>
-#include <linux/of_platform.h>
+#include <linux/platform_device.h>
#include <asm/io.h>
#include <asm/pmi.h>
__func__, d->irq, hw_irq, rc);
return -1;
}
- server = be16_to_cpu(oserver);
wanted_server = xics_get_irq_server(d->irq, cpumask, 1);
if (wanted_server < 0) {
--- /dev/null
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+set -e
+set -o pipefail
+
+# To debug, uncomment the following line
+# set -x
+
+# Output from -fpatchable-function-entry can only vary on ppc64 elfv2, so this
+# should not be invoked for other targets. Therefore we can pass in -m64 and
+# -mabi explicitly, to take care of toolchains defaulting to other targets.
+
+# Test whether the compile option -fpatchable-function-entry exists and
+# generates appropriate code
+echo "int func() { return 0; }" | \
+ $* -m64 -mabi=elfv2 -S -x c -O2 -fpatchable-function-entry=2 - -o - 2> /dev/null | \
+ grep -q "__patchable_function_entries"
+
+# Test whether nops are generated after the local entry point
+echo "int x; int func() { return x; }" | \
+ $* -m64 -mabi=elfv2 -S -x c -O2 -fpatchable-function-entry=2 - -o - 2> /dev/null | \
+ awk 'BEGIN { RS = ";" } /\.localentry.*nop.*\n[[:space:]]*nop/ { print $0 }' | \
+ grep -q "func:"
+
+exit 0
# Disable ftrace for the entire directory
ccflags-remove-$(CONFIG_FUNCTION_TRACER) += $(CC_FLAGS_FTRACE)
-ifdef CONFIG_CC_IS_CLANG
-# clang stores addresses on the stack causing the frame size to blow
-# out. See https://github.com/ClangBuiltLinux/linux/issues/252
-KBUILD_CFLAGS += -Wframe-larger-than=4096
-endif
-
ccflags-$(CONFIG_PPC64) := $(NO_MINIMAL_TOC)
+# Clang stores addresses on the stack causing the frame size to blow
+# out. See https://github.com/ClangBuiltLinux/linux/issues/252
+ccflags-$(CONFIG_CC_IS_CLANG) += -Wframe-larger-than=4096
+
obj-y += xmon.o nonstdio.o spr_access.o xmon_bpts.o
ifdef CONFIG_XMON_DISASSEMBLY
#ifdef CONFIG_PPC64
#include <asm/hvcall.h>
#include <asm/paca.h>
+#include <asm/lppaca.h>
#endif
#include "nonstdio.h"
{
unsigned long tskv = 0;
struct task_struct *volatile tsk = NULL;
- struct mm_struct *mm;
+ struct mm_struct *volatile mm;
pgd_t *pgdp;
p4d_t *p4dp;
pud_t *pudp;
#ifdef CONFIG_PPC_BOOK3E_64
static void dump_tlb_book3e(void)
{
- u32 mmucfg, pidmask, lpidmask;
+ u32 mmucfg;
u64 ramask;
- int i, tlb, ntlbs, pidsz, lpidsz, rasz, lrat = 0;
+ int i, tlb, ntlbs, pidsz, lpidsz, rasz;
int mmu_version;
static const char *pgsz_names[] = {
" 1K",
pidsz = ((mmucfg >> 6) & 0x1f) + 1;
lpidsz = (mmucfg >> 24) & 0xf;
rasz = (mmucfg >> 16) & 0x7f;
- if ((mmu_version > 1) && (mmucfg & 0x10000))
- lrat = 1;
printf("Book3E MMU MAV=%d.0,%d TLBs,%d-bit PID,%d-bit LPID,%d-bit RA\n",
mmu_version, ntlbs, pidsz, lpidsz, rasz);
- pidmask = (1ul << pidsz) - 1;
- lpidmask = (1ul << lpidsz) - 1;
ramask = (1ull << rasz) - 1;
for (tlb = 0; tlb < ntlbs; tlb++) {
return -ENODEV;
if (firmware_has_feature(FW_FEATURE_SPLPAR)) {
- /*
- * Use local_paca instead of get_lppaca() since
- * preemption is not disabled, and it is not required in
- * fact, since lppaca_ptr does not need to be the value
- * associated to the current CPU, it can be from any CPU.
- */
- if (lppaca_shared_proc(local_paca->lppaca_ptr)) {
+ if (lppaca_shared_proc()) {
cpuidle_state_table = shared_states;
max_idle_state = ARRAY_SIZE(shared_states);
} else {
return result;
}
-int __init ams_init(void)
+static int __init ams_init(void)
{
struct device_node *np;
#include <linux/input.h>
#include <linux/kthread.h>
#include <linux/mutex.h>
+#include <linux/platform_device.h>
#include <linux/spinlock.h>
#include <linux/types.h>
cxl_p1n_write(afu, CXL_PSL_SPAP_An, spap);
}
-static inline void detach_spa(struct cxl_afu *afu)
-{
- cxl_p1n_write(afu, CXL_PSL_SPAP_An, 0);
-}
-
void cxl_release_spa(struct cxl_afu *afu)
{
if (afu->native->spa) {
static int find_cxl_vsec(struct pci_dev *dev)
{
- int vsec = 0;
- u16 val;
-
- while ((vsec = pci_find_next_ext_capability(dev, vsec, PCI_EXT_CAP_ID_VNDR))) {
- pci_read_config_word(dev, vsec + 0x4, &val);
- if (val == CXL_PCI_VSEC_ID)
- return vsec;
- }
- return 0;
-
+ return pci_find_vsec_capability(dev, PCI_VENDOR_ID_IBM, CXL_PCI_VSEC_ID);
}
static void dump_cxl_config_space(struct pci_dev *dev)
#include <linux/phy.h>
#include <linux/dma-mapping.h>
-#include <asm/fs_pd.h>
-
#ifdef CONFIG_CPM1
#include <asm/cpm1.h>
#endif
#include <linux/pgtable.h>
#include <asm/immap_cpm2.h>
-#include <asm/mpc8260.h>
#include <asm/cpm2.h>
#include <asm/irq.h>
#include "../pci.h" /* for pci_add_new_bus */
#include "rpaphp.h"
+/*
+ * RTAS call get-sensor-state(DR_ENTITY_SENSE) return values as per PAPR:
+ * -- generic return codes ---
+ * -1: Hardware Error
+ * -2: RTAS_BUSY
+ * -3: Invalid sensor. RTAS Parameter Error.
+ * -- rtas_get_sensor function specific return codes ---
+ * -9000: Need DR entity to be powered up and unisolated before RTAS call
+ * -9001: Need DR entity to be powered up, but not unisolated, before RTAS call
+ * -9002: DR entity unusable
+ * 990x: Extended delay - where x is a number in the range of 0-5
+ */
+#define RTAS_SLOT_UNISOLATED -9000
+#define RTAS_SLOT_NOT_UNISOLATED -9001
+#define RTAS_SLOT_NOT_USABLE -9002
+
+static int rtas_get_sensor_errno(int rtas_rc)
+{
+ switch (rtas_rc) {
+ case 0:
+ /* Success case */
+ return 0;
+ case RTAS_SLOT_UNISOLATED:
+ case RTAS_SLOT_NOT_UNISOLATED:
+ return -EFAULT;
+ case RTAS_SLOT_NOT_USABLE:
+ return -ENODEV;
+ case RTAS_BUSY:
+ case RTAS_EXTENDED_DELAY_MIN...RTAS_EXTENDED_DELAY_MAX:
+ return -EBUSY;
+ default:
+ return rtas_error_rc(rtas_rc);
+ }
+}
+
+/*
+ * get_adapter_status() can be called by the EEH handler during EEH recovery.
+ * On certain PHB failures, the RTAS call rtas_call(get-sensor-state) returns
+ * extended busy error (9902) until PHB is recovered by pHyp. The RTAS call
+ * interface rtas_get_sensor() loops over the RTAS call on extended delay
+ * return code (9902) until the return value is either success (0) or error
+ * (-1). This causes the EEH handler to get stuck for ~6 seconds before it
+ * could notify that the PCI error has been detected and stop any active
+ * operations. This sometimes causes EEH recovery to fail. To avoid this issue,
+ * invoke rtas_call(get-sensor-state) directly if the respective PE is in EEH
+ * recovery state and return -EBUSY error based on RTAS return status. This
+ * will help the EEH handler to notify the driver about the PCI error
+ * immediately and successfully proceed with EEH recovery steps.
+ */
+
+static int __rpaphp_get_sensor_state(struct slot *slot, int *state)
+{
+ int rc;
+ int token = rtas_token("get-sensor-state");
+ struct pci_dn *pdn;
+ struct eeh_pe *pe;
+ struct pci_controller *phb = PCI_DN(slot->dn)->phb;
+
+ if (token == RTAS_UNKNOWN_SERVICE)
+ return -ENOENT;
+
+ /*
+ * Fallback to existing method for empty slot or PE isn't in EEH
+ * recovery.
+ */
+ pdn = list_first_entry_or_null(&PCI_DN(phb->dn)->child_list,
+ struct pci_dn, list);
+ if (!pdn)
+ goto fallback;
+
+ pe = eeh_dev_to_pe(pdn->edev);
+ if (pe && (pe->state & EEH_PE_RECOVERING)) {
+ rc = rtas_call(token, 2, 2, state, DR_ENTITY_SENSE,
+ slot->index);
+ return rtas_get_sensor_errno(rc);
+ }
+fallback:
+ return rtas_get_sensor(DR_ENTITY_SENSE, slot->index, state);
+}
+
int rpaphp_get_sensor_state(struct slot *slot, int *state)
{
int rc;
int setlevel;
- rc = rtas_get_sensor(DR_ENTITY_SENSE, slot->index, state);
+ rc = __rpaphp_get_sensor_state(slot, state);
if (rc < 0) {
if (rc == -EFAULT || rc == -EEXIST) {
dbg("%s: power on slot[%s] failed rc=%d.\n",
__func__, slot->name, rc);
} else {
- rc = rtas_get_sensor(DR_ENTITY_SENSE,
- slot->index, state);
+ rc = __rpaphp_get_sensor_state(slot, state);
}
} else if (rc == -ENODEV)
info("%s: slot is unusable\n", __func__);
extern int dbg_release_bp_slot(struct perf_event *bp);
extern int reserve_bp_slot(struct perf_event *bp);
extern void release_bp_slot(struct perf_event *bp);
-int arch_reserve_bp_slot(struct perf_event *bp);
-void arch_release_bp_slot(struct perf_event *bp);
-void arch_unregister_hw_breakpoint(struct perf_event *bp);
extern void flush_ptrace_hw_breakpoint(struct task_struct *tsk);
return 0;
}
-__weak int arch_reserve_bp_slot(struct perf_event *bp)
-{
- return 0;
-}
-
-__weak void arch_release_bp_slot(struct perf_event *bp)
-{
-}
-
-/*
- * Function to perform processor-specific cleanup during unregistration
- */
-__weak void arch_unregister_hw_breakpoint(struct perf_event *bp)
-{
- /*
- * A weak stub function here for those archs that don't define
- * it inside arch/.../kernel/hw_breakpoint.c
- */
-}
-
/*
* Constraints to check before allowing this new breakpoint counter.
*
enum bp_type_idx type;
int max_pinned_slots;
int weight;
- int ret;
/* We couldn't initialize breakpoint constraints on boot */
if (!constraints_initialized)
if (max_pinned_slots > hw_breakpoint_slots_cached(type))
return -ENOSPC;
- ret = arch_reserve_bp_slot(bp);
- if (ret)
- return ret;
-
return toggle_bp_slot(bp, true, type, weight);
}
enum bp_type_idx type;
int weight;
- arch_release_bp_slot(bp);
-
type = find_slot_idx(bp_type);
weight = hw_breakpoint_weight(bp);
WARN_ON(toggle_bp_slot(bp, false, type, weight));
{
struct mutex *mtx = bp_constraints_lock(bp);
- arch_unregister_hw_breakpoint(bp);
__release_bp_slot(bp, bp->attr.bp_type);
bp_constraints_unlock(mtx);
}
/* Setting timeout to -1 disables the alarm */
static uint64_t timeout = 120;
-int run_test(int (test_function)(void), char *name)
+int run_test(int (test_function)(void), const char *name)
{
bool terminated;
int rc, status;
timeout = time;
}
-int test_harness(int (test_function)(void), char *name)
+int test_harness(int (test_function)(void), const char *name)
{
int rc;
#ifndef _SELFTESTS_POWERPC_SUBUNIT_H
#define _SELFTESTS_POWERPC_SUBUNIT_H
-static inline void test_start(char *name)
+static inline void test_start(const char *name)
{
printf("test: %s\n", name);
}
-static inline void test_failure_detail(char *name, char *detail)
+static inline void test_failure_detail(const char *name, const char *detail)
{
printf("failure: %s [%s]\n", name, detail);
}
-static inline void test_failure(char *name)
+static inline void test_failure(const char *name)
{
printf("failure: %s\n", name);
}
-static inline void test_error(char *name)
+static inline void test_error(const char *name)
{
printf("error: %s\n", name);
}
-static inline void test_skip(char *name)
+static inline void test_skip(const char *name)
{
printf("skip: %s\n", name);
}
-static inline void test_success(char *name)
+static inline void test_success(const char *name)
{
printf("success: %s\n", name);
}
-static inline void test_finish(char *name, int status)
+static inline void test_finish(const char *name, int status)
{
if (status)
test_failure(name);
test_success(name);
}
-static inline void test_set_git_version(char *value)
+static inline void test_set_git_version(const char *value)
{
printf("tags: git_version:%s\n", value);
}
typedef uint8_t u8;
void test_harness_set_timeout(uint64_t time);
-int test_harness(int (test_function)(void), char *name);
+int test_harness(int (test_function)(void), const char *name);
int read_auxv(char *buf, ssize_t buf_size);
void *find_auxv_entry(int type, char *auxv);
# SPDX-License-Identifier: GPL-2.0-only
+bad_accesses
+exec_prot
hugetlb_vs_thp_test
-subpage_prot
-tempfile
-prot_sao
-segv_errors
-wild_bctr
large_vm_fork_separation
-bad_accesses
-tlbie_test
+large_vm_gpr_corruption
pkey_exec_prot
pkey_siginfo
+prot_sao
+segv_errors
stack_expansion_ldst
stack_expansion_signal
-large_vm_gpr_corruption
+subpage_prot
+tempfile
+tlbie_test
+wild_bctr
CFLAGS += $(KHDR_INCLUDES) -fno-pie
$(OUTPUT)/ptrace-gpr: ptrace-gpr.S
+$(OUTPUT)/ptrace-perf-hwbreak: ptrace-perf-asm.S
$(OUTPUT)/ptrace-pkey $(OUTPUT)/core-pkey: LDLIBS += -pthread
$(TEST_GEN_PROGS): ../harness.c ../utils.c ../lib/reg.S
} \
} while (0)
-#define PARENT_SKIP_IF_UNSUPPORTED(x, sync) \
+#define PARENT_SKIP_IF_UNSUPPORTED(x, sync, msg) \
do { \
if ((x) == -1 && (errno == ENODEV || errno == EINVAL)) { \
(sync)->parent_gave_up = true; \
prod_child(sync); \
- SKIP_IF(1); \
+ SKIP_IF_MSG(1, msg); \
} \
} while (0)
* to the child.
*/
ret = ptrace_read_regs(pid, NT_PPC_PKEY, regs, 3);
- PARENT_SKIP_IF_UNSUPPORTED(ret, &info->child_sync);
+ PARENT_SKIP_IF_UNSUPPORTED(ret, &info->child_sync, "PKEYs not supported");
PARENT_FAIL_IF(ret, &info->child_sync);
info->amr = regs[0];
{
srand ( time(NULL) );
- SKIP_IF(!perf_breakpoint_supported());
+ SKIP_IF_MSG(!perf_breakpoint_supported(), "Perf breakpoints not supported");
return runtest();
}
static void write_var(int len)
{
- __u8 *pcvar;
- __u16 *psvar;
- __u32 *pivar;
- __u64 *plvar;
+ volatile __u8 *pcvar;
+ volatile __u16 *psvar;
+ volatile __u32 *pivar;
+ volatile __u64 *plvar;
switch (len) {
case 1:
- pcvar = (__u8 *)&glvar;
+ pcvar = (volatile __u8 *)&glvar;
*pcvar = 0xff;
break;
case 2:
- psvar = (__u16 *)&glvar;
+ psvar = (volatile __u16 *)&glvar;
*psvar = 0xffff;
break;
case 4:
- pivar = (__u32 *)&glvar;
+ pivar = (volatile __u32 *)&glvar;
*pivar = 0xffffffff;
break;
case 8:
- plvar = (__u64 *)&glvar;
+ plvar = (volatile __u64 *)&glvar;
*plvar = 0xffffffffffffffffLL;
break;
}
switch (len) {
case 1:
- cvar = (__u8)glvar;
+ cvar = (volatile __u8)glvar;
break;
case 2:
- svar = (__u16)glvar;
+ svar = (volatile __u16)glvar;
break;
case 4:
- ivar = (__u32)glvar;
+ ivar = (volatile __u32)glvar;
break;
case 8:
- lvar = (__u64)glvar;
+ lvar = (volatile __u64)glvar;
break;
}
}
wait(NULL);
get_dbginfo(child_pid, &dbginfo);
- SKIP_IF(dbginfo.num_data_bps == 0);
+ SKIP_IF_MSG(dbginfo.num_data_bps == 0, "No data breakpoints present");
dawr = dawr_present(&dbginfo);
run_tests(child_pid, &dbginfo, dawr);
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include <ppc-asm.h>
+
+.global same_watch_addr_load
+.global same_watch_addr_trap
+
+FUNC_START(same_watch_addr_child)
+ nop
+same_watch_addr_load:
+ ld 0,0(3)
+ nop
+same_watch_addr_trap:
+ trap
+ blr
+FUNC_END(same_watch_addr_child)
+
+
+.global perf_then_ptrace_load1
+.global perf_then_ptrace_load2
+.global perf_then_ptrace_trap
+
+FUNC_START(perf_then_ptrace_child)
+ nop
+perf_then_ptrace_load1:
+ ld 0,0(3)
+perf_then_ptrace_load2:
+ ld 0,0(4)
+ nop
+perf_then_ptrace_trap:
+ trap
+ blr
+FUNC_END(perf_then_ptrace_child)
// SPDX-License-Identifier: GPL-2.0+
-#include <stdio.h>
-#include <string.h>
-#include <signal.h>
-#include <stdlib.h>
-#include <unistd.h>
-#include <errno.h>
-#include <linux/hw_breakpoint.h>
-#include <linux/perf_event.h>
+
#include <asm/unistd.h>
-#include <sys/ptrace.h>
+#include <linux/hw_breakpoint.h>
+#include <linux/ptrace.h>
+#include <memory.h>
+#include <stdlib.h>
#include <sys/wait.h>
-#include "ptrace.h"
-char data[16];
+#include "utils.h"
-/* Overlapping address range */
-volatile __u64 *ptrace_data1 = (__u64 *)&data[0];
-volatile __u64 *perf_data1 = (__u64 *)&data[4];
+/*
+ * Child subroutine that performs a load on the address, then traps
+ */
+void same_watch_addr_child(unsigned long *addr);
-/* Non-overlapping address range */
-volatile __u64 *ptrace_data2 = (__u64 *)&data[0];
-volatile __u64 *perf_data2 = (__u64 *)&data[8];
+/* Address of the ld instruction in same_watch_addr_child() */
+extern char same_watch_addr_load[];
-static unsigned long pid_max_addr(void)
-{
- FILE *fp;
- char *line, *c;
- char addr[100];
- size_t len = 0;
-
- fp = fopen("/proc/kallsyms", "r");
- if (!fp) {
- printf("Failed to read /proc/kallsyms. Exiting..\n");
- exit(EXIT_FAILURE);
- }
+/* Address of the end trap instruction in same_watch_addr_child() */
+extern char same_watch_addr_trap[];
- while (getline(&line, &len, fp) != -1) {
- if (!strstr(line, "pid_max") || strstr(line, "pid_max_max") ||
- strstr(line, "pid_max_min"))
- continue;
+/*
+ * Child subroutine that performs a load on the first address, then a load on
+ * the second address (with no instructions separating this from the first
+ * load), then traps.
+ */
+void perf_then_ptrace_child(unsigned long *first_addr, unsigned long *second_addr);
- strncpy(addr, line, len < 100 ? len : 100);
- c = strchr(addr, ' ');
- *c = '\0';
- return strtoul(addr, &c, 16);
- }
- fclose(fp);
- printf("Could not find pix_max. Exiting..\n");
- exit(EXIT_FAILURE);
- return -1;
-}
+/* Address of the first ld instruction in perf_then_ptrace_child() */
+extern char perf_then_ptrace_load1[];
-static void perf_user_event_attr_set(struct perf_event_attr *attr, __u64 addr, __u64 len)
-{
- memset(attr, 0, sizeof(struct perf_event_attr));
- attr->type = PERF_TYPE_BREAKPOINT;
- attr->size = sizeof(struct perf_event_attr);
- attr->bp_type = HW_BREAKPOINT_R;
- attr->bp_addr = addr;
- attr->bp_len = len;
- attr->exclude_kernel = 1;
- attr->exclude_hv = 1;
-}
+/* Address of the second ld instruction in perf_then_ptrace_child() */
+extern char perf_then_ptrace_load2[];
-static void perf_kernel_event_attr_set(struct perf_event_attr *attr)
+/* Address of the end trap instruction in perf_then_ptrace_child() */
+extern char perf_then_ptrace_trap[];
+
+static inline long sys_ptrace(long request, pid_t pid, unsigned long addr, unsigned long data)
{
- memset(attr, 0, sizeof(struct perf_event_attr));
- attr->type = PERF_TYPE_BREAKPOINT;
- attr->size = sizeof(struct perf_event_attr);
- attr->bp_type = HW_BREAKPOINT_R;
- attr->bp_addr = pid_max_addr();
- attr->bp_len = sizeof(unsigned long);
- attr->exclude_user = 1;
- attr->exclude_hv = 1;
+ return syscall(__NR_ptrace, request, pid, addr, data);
}
-static int perf_cpu_event_open(int cpu, __u64 addr, __u64 len)
+static long ptrace_traceme(void)
{
- struct perf_event_attr attr;
-
- perf_user_event_attr_set(&attr, addr, len);
- return syscall(__NR_perf_event_open, &attr, -1, cpu, -1, 0);
+ return sys_ptrace(PTRACE_TRACEME, 0, 0, 0);
}
-static int perf_thread_event_open(pid_t child_pid, __u64 addr, __u64 len)
+static long ptrace_getregs(pid_t pid, struct pt_regs *result)
{
- struct perf_event_attr attr;
-
- perf_user_event_attr_set(&attr, addr, len);
- return syscall(__NR_perf_event_open, &attr, child_pid, -1, -1, 0);
+ return sys_ptrace(PTRACE_GETREGS, pid, 0, (unsigned long)result);
}
-static int perf_thread_cpu_event_open(pid_t child_pid, int cpu, __u64 addr, __u64 len)
+static long ptrace_setregs(pid_t pid, struct pt_regs *result)
{
- struct perf_event_attr attr;
-
- perf_user_event_attr_set(&attr, addr, len);
- return syscall(__NR_perf_event_open, &attr, child_pid, cpu, -1, 0);
+ return sys_ptrace(PTRACE_SETREGS, pid, 0, (unsigned long)result);
}
-static int perf_thread_kernel_event_open(pid_t child_pid)
+static long ptrace_cont(pid_t pid, long signal)
{
- struct perf_event_attr attr;
-
- perf_kernel_event_attr_set(&attr);
- return syscall(__NR_perf_event_open, &attr, child_pid, -1, -1, 0);
+ return sys_ptrace(PTRACE_CONT, pid, 0, signal);
}
-static int perf_cpu_kernel_event_open(int cpu)
+static long ptrace_singlestep(pid_t pid, long signal)
{
- struct perf_event_attr attr;
-
- perf_kernel_event_attr_set(&attr);
- return syscall(__NR_perf_event_open, &attr, -1, cpu, -1, 0);
+ return sys_ptrace(PTRACE_SINGLESTEP, pid, 0, signal);
}
-static int child(void)
+static long ppc_ptrace_gethwdbginfo(pid_t pid, struct ppc_debug_info *dbginfo)
{
- int ret;
-
- ret = ptrace(PTRACE_TRACEME, 0, NULL, 0);
- if (ret) {
- printf("Error: PTRACE_TRACEME failed\n");
- return 0;
- }
- kill(getpid(), SIGUSR1); /* --> parent (SIGUSR1) */
-
- return 0;
+ return sys_ptrace(PPC_PTRACE_GETHWDBGINFO, pid, 0, (unsigned long)dbginfo);
}
-static void ptrace_ppc_hw_breakpoint(struct ppc_hw_breakpoint *info, int type,
- __u64 addr, int len)
+static long ppc_ptrace_sethwdbg(pid_t pid, struct ppc_hw_breakpoint *bp_info)
{
- info->version = 1;
- info->trigger_type = type;
- info->condition_mode = PPC_BREAKPOINT_CONDITION_NONE;
- info->addr = addr;
- info->addr2 = addr + len;
- info->condition_value = 0;
- if (!len)
- info->addr_mode = PPC_BREAKPOINT_MODE_EXACT;
- else
- info->addr_mode = PPC_BREAKPOINT_MODE_RANGE_INCLUSIVE;
+ return sys_ptrace(PPC_PTRACE_SETHWDEBUG, pid, 0, (unsigned long)bp_info);
}
-static int ptrace_open(pid_t child_pid, __u64 wp_addr, int len)
+static long ppc_ptrace_delhwdbg(pid_t pid, int bp_id)
{
- struct ppc_hw_breakpoint info;
-
- ptrace_ppc_hw_breakpoint(&info, PPC_BREAKPOINT_TRIGGER_RW, wp_addr, len);
- return ptrace(PPC_PTRACE_SETHWDEBUG, child_pid, 0, &info);
+ return sys_ptrace(PPC_PTRACE_DELHWDEBUG, pid, 0L, bp_id);
}
-static int test1(pid_t child_pid)
+static long ptrace_getreg_pc(pid_t pid, void **pc)
{
- int perf_fd;
- int ptrace_fd;
- int ret = 0;
-
- /* Test:
- * if (new per thread event by ptrace)
- * if (existing cpu event by perf)
- * if (addr range overlaps)
- * fail;
- */
+ struct pt_regs regs;
+ long err;
- perf_fd = perf_cpu_event_open(0, (__u64)perf_data1, sizeof(*perf_data1));
- if (perf_fd < 0)
- return -1;
+ err = ptrace_getregs(pid, ®s);
+ if (err)
+ return err;
- ptrace_fd = ptrace_open(child_pid, (__u64)ptrace_data1, sizeof(*ptrace_data1));
- if (ptrace_fd > 0 || errno != ENOSPC)
- ret = -1;
+ *pc = (void *)regs.nip;
- close(perf_fd);
- return ret;
+ return 0;
}
-static int test2(pid_t child_pid)
+static long ptrace_setreg_pc(pid_t pid, void *pc)
{
- int perf_fd;
- int ptrace_fd;
- int ret = 0;
-
- /* Test:
- * if (new per thread event by ptrace)
- * if (existing cpu event by perf)
- * if (addr range does not overlaps)
- * allow;
- */
+ struct pt_regs regs;
+ long err;
- perf_fd = perf_cpu_event_open(0, (__u64)perf_data2, sizeof(*perf_data2));
- if (perf_fd < 0)
- return -1;
-
- ptrace_fd = ptrace_open(child_pid, (__u64)ptrace_data2, sizeof(*ptrace_data2));
- if (ptrace_fd < 0) {
- ret = -1;
- goto perf_close;
- }
- ptrace(PPC_PTRACE_DELHWDEBUG, child_pid, 0, ptrace_fd);
-
-perf_close:
- close(perf_fd);
- return ret;
-}
+ err = ptrace_getregs(pid, ®s);
+ if (err)
+ return err;
-static int test3(pid_t child_pid)
-{
- int perf_fd;
- int ptrace_fd;
- int ret = 0;
-
- /* Test:
- * if (new per thread event by ptrace)
- * if (existing thread event by perf on the same thread)
- * if (addr range overlaps)
- * fail;
- */
- perf_fd = perf_thread_event_open(child_pid, (__u64)perf_data1,
- sizeof(*perf_data1));
- if (perf_fd < 0)
- return -1;
+ regs.nip = (unsigned long)pc;
- ptrace_fd = ptrace_open(child_pid, (__u64)ptrace_data1, sizeof(*ptrace_data1));
- if (ptrace_fd > 0 || errno != ENOSPC)
- ret = -1;
+ err = ptrace_setregs(pid, ®s);
+ if (err)
+ return err;
- close(perf_fd);
- return ret;
+ return 0;
}
-static int test4(pid_t child_pid)
+static int perf_event_open(struct perf_event_attr *attr, pid_t pid, int cpu,
+ int group_fd, unsigned long flags)
{
- int perf_fd;
- int ptrace_fd;
- int ret = 0;
-
- /* Test:
- * if (new per thread event by ptrace)
- * if (existing thread event by perf on the same thread)
- * if (addr range does not overlaps)
- * fail;
- */
- perf_fd = perf_thread_event_open(child_pid, (__u64)perf_data2,
- sizeof(*perf_data2));
- if (perf_fd < 0)
- return -1;
-
- ptrace_fd = ptrace_open(child_pid, (__u64)ptrace_data2, sizeof(*ptrace_data2));
- if (ptrace_fd < 0) {
- ret = -1;
- goto perf_close;
- }
- ptrace(PPC_PTRACE_DELHWDEBUG, child_pid, 0, ptrace_fd);
-
-perf_close:
- close(perf_fd);
- return ret;
+ return syscall(__NR_perf_event_open, attr, pid, cpu, group_fd, flags);
}
-static int test5(pid_t child_pid)
+static void perf_user_event_attr_set(struct perf_event_attr *attr, void *addr, u64 len)
{
- int perf_fd;
- int ptrace_fd;
- int cpid;
- int ret = 0;
-
- /* Test:
- * if (new per thread event by ptrace)
- * if (existing thread event by perf on the different thread)
- * allow;
- */
- cpid = fork();
- if (!cpid) {
- /* Temporary Child */
- pause();
- exit(EXIT_SUCCESS);
- }
-
- perf_fd = perf_thread_event_open(cpid, (__u64)perf_data1, sizeof(*perf_data1));
- if (perf_fd < 0) {
- ret = -1;
- goto kill_child;
- }
-
- ptrace_fd = ptrace_open(child_pid, (__u64)ptrace_data1, sizeof(*ptrace_data1));
- if (ptrace_fd < 0) {
- ret = -1;
- goto perf_close;
- }
+ memset(attr, 0, sizeof(struct perf_event_attr));
- ptrace(PPC_PTRACE_DELHWDEBUG, child_pid, 0, ptrace_fd);
-perf_close:
- close(perf_fd);
-kill_child:
- kill(cpid, SIGINT);
- return ret;
+ attr->type = PERF_TYPE_BREAKPOINT;
+ attr->size = sizeof(struct perf_event_attr);
+ attr->bp_type = HW_BREAKPOINT_R;
+ attr->bp_addr = (u64)addr;
+ attr->bp_len = len;
+ attr->exclude_kernel = 1;
+ attr->exclude_hv = 1;
}
-static int test6(pid_t child_pid)
+static int perf_watchpoint_open(pid_t child_pid, void *addr, u64 len)
{
- int perf_fd;
- int ptrace_fd;
- int ret = 0;
-
- /* Test:
- * if (new per thread kernel event by perf)
- * if (existing thread event by ptrace on the same thread)
- * allow;
- * -- OR --
- * if (new per cpu kernel event by perf)
- * if (existing thread event by ptrace)
- * allow;
- */
- ptrace_fd = ptrace_open(child_pid, (__u64)ptrace_data1, sizeof(*ptrace_data1));
- if (ptrace_fd < 0)
- return -1;
-
- perf_fd = perf_thread_kernel_event_open(child_pid);
- if (perf_fd < 0) {
- ret = -1;
- goto ptrace_close;
- }
- close(perf_fd);
-
- perf_fd = perf_cpu_kernel_event_open(0);
- if (perf_fd < 0) {
- ret = -1;
- goto ptrace_close;
- }
- close(perf_fd);
+ struct perf_event_attr attr;
-ptrace_close:
- ptrace(PPC_PTRACE_DELHWDEBUG, child_pid, 0, ptrace_fd);
- return ret;
+ perf_user_event_attr_set(&attr, addr, len);
+ return perf_event_open(&attr, child_pid, -1, -1, 0);
}
-static int test7(pid_t child_pid)
+static int perf_read_counter(int perf_fd, u64 *count)
{
- int perf_fd;
- int ptrace_fd;
- int ret = 0;
-
- /* Test:
- * if (new per thread event by perf)
- * if (existing thread event by ptrace on the same thread)
- * if (addr range overlaps)
- * fail;
+ /*
+ * A perf counter is retrieved by the read() syscall. It contains
+ * the current count as 8 bytes that are interpreted as a u64
*/
- ptrace_fd = ptrace_open(child_pid, (__u64)ptrace_data1, sizeof(*ptrace_data1));
- if (ptrace_fd < 0)
- return -1;
+ ssize_t len = read(perf_fd, count, sizeof(*count));
- perf_fd = perf_thread_event_open(child_pid, (__u64)perf_data1,
- sizeof(*perf_data1));
- if (perf_fd > 0 || errno != ENOSPC)
- ret = -1;
+ if (len != sizeof(*count))
+ return -1;
- ptrace(PPC_PTRACE_DELHWDEBUG, child_pid, 0, ptrace_fd);
- return ret;
+ return 0;
}
-static int test8(pid_t child_pid)
+static void ppc_ptrace_init_breakpoint(struct ppc_hw_breakpoint *info,
+ int type, void *addr, int len)
{
- int perf_fd;
- int ptrace_fd;
- int ret = 0;
-
- /* Test:
- * if (new per thread event by perf)
- * if (existing thread event by ptrace on the same thread)
- * if (addr range does not overlaps)
- * allow;
- */
- ptrace_fd = ptrace_open(child_pid, (__u64)ptrace_data2, sizeof(*ptrace_data2));
- if (ptrace_fd < 0)
- return -1;
-
- perf_fd = perf_thread_event_open(child_pid, (__u64)perf_data2,
- sizeof(*perf_data2));
- if (perf_fd < 0) {
- ret = -1;
- goto ptrace_close;
- }
- close(perf_fd);
-
-ptrace_close:
- ptrace(PPC_PTRACE_DELHWDEBUG, child_pid, 0, ptrace_fd);
- return ret;
+ info->version = 1;
+ info->trigger_type = type;
+ info->condition_mode = PPC_BREAKPOINT_CONDITION_NONE;
+ info->addr = (u64)addr;
+ info->addr2 = (u64)addr + len;
+ info->condition_value = 0;
+ if (!len)
+ info->addr_mode = PPC_BREAKPOINT_MODE_EXACT;
+ else
+ info->addr_mode = PPC_BREAKPOINT_MODE_RANGE_INCLUSIVE;
}
-static int test9(pid_t child_pid)
+/*
+ * Checks if we can place at least 2 watchpoints on the child process
+ */
+static int check_watchpoints(pid_t pid)
{
- int perf_fd;
- int ptrace_fd;
- int cpid;
- int ret = 0;
-
- /* Test:
- * if (new per thread event by perf)
- * if (existing thread event by ptrace on the other thread)
- * allow;
- */
- ptrace_fd = ptrace_open(child_pid, (__u64)ptrace_data1, sizeof(*ptrace_data1));
- if (ptrace_fd < 0)
- return -1;
-
- cpid = fork();
- if (!cpid) {
- /* Temporary Child */
- pause();
- exit(EXIT_SUCCESS);
- }
+ struct ppc_debug_info dbginfo;
- perf_fd = perf_thread_event_open(cpid, (__u64)perf_data1, sizeof(*perf_data1));
- if (perf_fd < 0) {
- ret = -1;
- goto kill_child;
- }
- close(perf_fd);
+ FAIL_IF_MSG(ppc_ptrace_gethwdbginfo(pid, &dbginfo), "PPC_PTRACE_GETHWDBGINFO failed");
+ SKIP_IF_MSG(dbginfo.num_data_bps <= 1, "Not enough data watchpoints (need at least 2)");
-kill_child:
- kill(cpid, SIGINT);
- ptrace(PPC_PTRACE_DELHWDEBUG, child_pid, 0, ptrace_fd);
- return ret;
+ return 0;
}
-static int test10(pid_t child_pid)
+/*
+ * Wrapper around a plain fork() call that sets up the child for
+ * ptrace-ing. Both the parent and child return from this, though
+ * the child is stopped until ptrace_cont(pid) is run by the parent.
+ */
+static int ptrace_fork_child(pid_t *pid)
{
- int perf_fd;
- int ptrace_fd;
- int ret = 0;
-
- /* Test:
- * if (new per cpu event by perf)
- * if (existing thread event by ptrace on the same thread)
- * if (addr range overlaps)
- * fail;
- */
- ptrace_fd = ptrace_open(child_pid, (__u64)ptrace_data1, sizeof(*ptrace_data1));
- if (ptrace_fd < 0)
- return -1;
+ int status;
- perf_fd = perf_cpu_event_open(0, (__u64)perf_data1, sizeof(*perf_data1));
- if (perf_fd > 0 || errno != ENOSPC)
- ret = -1;
-
- ptrace(PPC_PTRACE_DELHWDEBUG, child_pid, 0, ptrace_fd);
- return ret;
-}
+ *pid = fork();
-static int test11(pid_t child_pid)
-{
- int perf_fd;
- int ptrace_fd;
- int ret = 0;
-
- /* Test:
- * if (new per cpu event by perf)
- * if (existing thread event by ptrace on the same thread)
- * if (addr range does not overlap)
- * allow;
- */
- ptrace_fd = ptrace_open(child_pid, (__u64)ptrace_data2, sizeof(*ptrace_data2));
- if (ptrace_fd < 0)
- return -1;
+ if (*pid < 0)
+ FAIL_IF_MSG(1, "Failed to fork child");
- perf_fd = perf_cpu_event_open(0, (__u64)perf_data2, sizeof(*perf_data2));
- if (perf_fd < 0) {
- ret = -1;
- goto ptrace_close;
+ if (!*pid) {
+ FAIL_IF_EXIT_MSG(ptrace_traceme(), "PTRACE_TRACEME failed");
+ FAIL_IF_EXIT_MSG(raise(SIGSTOP), "Child failed to raise SIGSTOP");
+ } else {
+ /* Synchronise on child SIGSTOP */
+ FAIL_IF_MSG(waitpid(*pid, &status, 0) == -1, "Failed to wait for child");
+ FAIL_IF_MSG(!WIFSTOPPED(status), "Child is not stopped");
}
- close(perf_fd);
-ptrace_close:
- ptrace(PPC_PTRACE_DELHWDEBUG, child_pid, 0, ptrace_fd);
- return ret;
+ return 0;
}
-static int test12(pid_t child_pid)
+/*
+ * Tests the interaction between ptrace and perf watching the same data.
+ *
+ * We expect ptrace to take 'priority', as it is has before-execute
+ * semantics.
+ *
+ * The perf counter should not be incremented yet because perf has after-execute
+ * semantics. E.g., if ptrace changes the child PC, we don't even execute the
+ * instruction at all.
+ *
+ * When the child is stopped for ptrace, we test both continue and single step.
+ * Both should increment the perf counter. We also test changing the PC somewhere
+ * different and stepping, which should not increment the perf counter.
+ */
+int same_watch_addr_test(void)
{
- int perf_fd;
- int ptrace_fd;
- int ret = 0;
-
- /* Test:
- * if (new per thread and per cpu event by perf)
- * if (existing thread event by ptrace on the same thread)
- * if (addr range overlaps)
- * fail;
- */
- ptrace_fd = ptrace_open(child_pid, (__u64)ptrace_data1, sizeof(*ptrace_data1));
- if (ptrace_fd < 0)
- return -1;
+ struct ppc_hw_breakpoint bp_info; /* ptrace breakpoint info */
+ int bp_id; /* Breakpoint handle of ptrace watchpoint */
+ int perf_fd; /* File descriptor of perf performance counter */
+ u64 perf_count; /* Most recently fetched perf performance counter value */
+ pid_t pid; /* PID of child process */
+ void *pc; /* Most recently fetched child PC value */
+ int status; /* Stop status of child after waitpid */
+ unsigned long value; /* Dummy value to be read/written to by child */
+ int err;
+
+ err = ptrace_fork_child(&pid);
+ if (err)
+ return err;
+
+ if (!pid) {
+ same_watch_addr_child(&value);
+ exit(1);
+ }
- perf_fd = perf_thread_cpu_event_open(child_pid, 0, (__u64)perf_data1, sizeof(*perf_data1));
- if (perf_fd > 0 || errno != ENOSPC)
- ret = -1;
+ err = check_watchpoints(pid);
+ if (err)
+ return err;
- ptrace(PPC_PTRACE_DELHWDEBUG, child_pid, 0, ptrace_fd);
- return ret;
-}
+ /* Place a perf watchpoint counter on value */
+ perf_fd = perf_watchpoint_open(pid, &value, sizeof(value));
+ FAIL_IF_MSG(perf_fd < 0, "Failed to open perf performance counter");
-static int test13(pid_t child_pid)
-{
- int perf_fd;
- int ptrace_fd;
- int ret = 0;
-
- /* Test:
- * if (new per thread and per cpu event by perf)
- * if (existing thread event by ptrace on the same thread)
- * if (addr range does not overlap)
- * allow;
- */
- ptrace_fd = ptrace_open(child_pid, (__u64)ptrace_data2, sizeof(*ptrace_data2));
- if (ptrace_fd < 0)
- return -1;
+ /* Place a ptrace watchpoint on value */
+ ppc_ptrace_init_breakpoint(&bp_info, PPC_BREAKPOINT_TRIGGER_READ, &value, sizeof(value));
+ bp_id = ppc_ptrace_sethwdbg(pid, &bp_info);
+ FAIL_IF_MSG(bp_id < 0, "Failed to set ptrace watchpoint");
- perf_fd = perf_thread_cpu_event_open(child_pid, 0, (__u64)perf_data2, sizeof(*perf_data2));
- if (perf_fd < 0) {
- ret = -1;
- goto ptrace_close;
- }
- close(perf_fd);
+ /* Let the child run. It should stop on the ptrace watchpoint */
+ FAIL_IF_MSG(ptrace_cont(pid, 0), "Failed to continue child");
-ptrace_close:
- ptrace(PPC_PTRACE_DELHWDEBUG, child_pid, 0, ptrace_fd);
- return ret;
-}
+ FAIL_IF_MSG(waitpid(pid, &status, 0) == -1, "Failed to wait for child");
+ FAIL_IF_MSG(!WIFSTOPPED(status), "Child is not stopped");
+ FAIL_IF_MSG(ptrace_getreg_pc(pid, &pc), "Failed to get child PC");
+ FAIL_IF_MSG(pc != same_watch_addr_load, "Child did not stop on load instruction");
-static int test14(pid_t child_pid)
-{
- int perf_fd;
- int ptrace_fd;
- int cpid;
- int ret = 0;
-
- /* Test:
- * if (new per thread and per cpu event by perf)
- * if (existing thread event by ptrace on the other thread)
- * allow;
+ /*
+ * We stopped before executing the load, so perf should not have
+ * recorded any events yet
*/
- ptrace_fd = ptrace_open(child_pid, (__u64)ptrace_data1, sizeof(*ptrace_data1));
- if (ptrace_fd < 0)
- return -1;
-
- cpid = fork();
- if (!cpid) {
- /* Temporary Child */
- pause();
- exit(EXIT_SUCCESS);
- }
-
- perf_fd = perf_thread_cpu_event_open(cpid, 0, (__u64)perf_data1,
- sizeof(*perf_data1));
- if (perf_fd < 0) {
- ret = -1;
- goto kill_child;
- }
- close(perf_fd);
-
-kill_child:
- kill(cpid, SIGINT);
- ptrace(PPC_PTRACE_DELHWDEBUG, child_pid, 0, ptrace_fd);
- return ret;
-}
+ FAIL_IF_MSG(perf_read_counter(perf_fd, &perf_count), "Failed to read perf counter");
+ FAIL_IF_MSG(perf_count != 0, "perf recorded unexpected event");
+
+ /* Single stepping over the load should increment the perf counter */
+ FAIL_IF_MSG(ptrace_singlestep(pid, 0), "Failed to single step child");
+
+ FAIL_IF_MSG(waitpid(pid, &status, 0) == -1, "Failed to wait for child");
+ FAIL_IF_MSG(!WIFSTOPPED(status), "Child is not stopped");
+ FAIL_IF_MSG(ptrace_getreg_pc(pid, &pc), "Failed to get child PC");
+ FAIL_IF_MSG(pc != same_watch_addr_load + 4, "Failed to single step load instruction");
+ FAIL_IF_MSG(perf_read_counter(perf_fd, &perf_count), "Failed to read perf counter");
+ FAIL_IF_MSG(perf_count != 1, "perf counter did not increment");
+
+ /*
+ * Set up a ptrace watchpoint on the value again and trigger it.
+ * The perf counter should not have incremented because we do not
+ * execute the load yet.
+ */
+ FAIL_IF_MSG(ppc_ptrace_delhwdbg(pid, bp_id), "Failed to remove old ptrace watchpoint");
+ bp_id = ppc_ptrace_sethwdbg(pid, &bp_info);
+ FAIL_IF_MSG(bp_id < 0, "Failed to set ptrace watchpoint");
+ FAIL_IF_MSG(ptrace_setreg_pc(pid, same_watch_addr_load), "Failed to set child PC");
+ FAIL_IF_MSG(ptrace_cont(pid, 0), "Failed to continue child");
+
+ FAIL_IF_MSG(waitpid(pid, &status, 0) == -1, "Failed to wait for child");
+ FAIL_IF_MSG(!WIFSTOPPED(status), "Child is not stopped");
+ FAIL_IF_MSG(ptrace_getreg_pc(pid, &pc), "Failed to get child PC");
+ FAIL_IF_MSG(pc != same_watch_addr_load, "Child did not stop on load trap");
+ FAIL_IF_MSG(perf_read_counter(perf_fd, &perf_count), "Failed to read perf counter");
+ FAIL_IF_MSG(perf_count != 1, "perf counter should not have changed");
+
+ /* Continuing over the load should increment the perf counter */
+ FAIL_IF_MSG(ptrace_cont(pid, 0), "Failed to continue child");
+
+ FAIL_IF_MSG(waitpid(pid, &status, 0) == -1, "Failed to wait for child");
+ FAIL_IF_MSG(!WIFSTOPPED(status), "Child is not stopped");
+ FAIL_IF_MSG(ptrace_getreg_pc(pid, &pc), "Failed to get child PC");
+ FAIL_IF_MSG(pc != same_watch_addr_trap, "Child did not stop on end trap");
+ FAIL_IF_MSG(perf_read_counter(perf_fd, &perf_count), "Failed to read perf counter");
+ FAIL_IF_MSG(perf_count != 2, "perf counter did not increment");
+
+ /*
+ * If we set the child PC back to the load instruction, then continue,
+ * we should reach the end trap (because ptrace is one-shot) and have
+ * another perf event.
+ */
+ FAIL_IF_MSG(ptrace_setreg_pc(pid, same_watch_addr_load), "Failed to set child PC");
+ FAIL_IF_MSG(ptrace_cont(pid, 0), "Failed to continue child");
+
+ FAIL_IF_MSG(waitpid(pid, &status, 0) == -1, "Failed to wait for child");
+ FAIL_IF_MSG(!WIFSTOPPED(status), "Child is not stopped");
+ FAIL_IF_MSG(ptrace_getreg_pc(pid, &pc), "Failed to get child PC");
+ FAIL_IF_MSG(pc != same_watch_addr_trap, "Child did not stop on end trap");
+ FAIL_IF_MSG(perf_read_counter(perf_fd, &perf_count), "Failed to read perf counter");
+ FAIL_IF_MSG(perf_count != 3, "perf counter did not increment");
+
+ /*
+ * If we set the child PC back to the load instruction, set a ptrace
+ * watchpoint on the load, then continue, we should immediately get
+ * the ptrace trap without incrementing the perf counter
+ */
+ FAIL_IF_MSG(ppc_ptrace_delhwdbg(pid, bp_id), "Failed to remove old ptrace watchpoint");
+ bp_id = ppc_ptrace_sethwdbg(pid, &bp_info);
+ FAIL_IF_MSG(bp_id < 0, "Failed to set ptrace watchpoint");
+ FAIL_IF_MSG(ptrace_setreg_pc(pid, same_watch_addr_load), "Failed to set child PC");
+ FAIL_IF_MSG(ptrace_cont(pid, 0), "Failed to continue child");
+
+ FAIL_IF_MSG(waitpid(pid, &status, 0) == -1, "Failed to wait for child");
+ FAIL_IF_MSG(!WIFSTOPPED(status), "Child is not stopped");
+ FAIL_IF_MSG(ptrace_getreg_pc(pid, &pc), "Failed to get child PC");
+ FAIL_IF_MSG(pc != same_watch_addr_load, "Child did not stop on load instruction");
+ FAIL_IF_MSG(perf_read_counter(perf_fd, &perf_count), "Failed to read perf counter");
+ FAIL_IF_MSG(perf_count != 3, "perf counter should not have changed");
+
+ /*
+ * If we change the PC while stopped on the load instruction, we should
+ * not increment the perf counter (because ptrace is before-execute,
+ * perf is after-execute).
+ */
+ FAIL_IF_MSG(ptrace_setreg_pc(pid, same_watch_addr_load + 4), "Failed to set child PC");
+ FAIL_IF_MSG(ptrace_cont(pid, 0), "Failed to continue child");
-static int do_test(const char *msg, int (*fun)(pid_t arg), pid_t arg)
-{
- int ret;
+ FAIL_IF_MSG(waitpid(pid, &status, 0) == -1, "Failed to wait for child");
+ FAIL_IF_MSG(!WIFSTOPPED(status), "Child is not stopped");
+ FAIL_IF_MSG(ptrace_getreg_pc(pid, &pc), "Failed to get child PC");
+ FAIL_IF_MSG(pc != same_watch_addr_trap, "Child did not stop on end trap");
+ FAIL_IF_MSG(perf_read_counter(perf_fd, &perf_count), "Failed to read perf counter");
+ FAIL_IF_MSG(perf_count != 3, "perf counter should not have changed");
- ret = fun(arg);
- if (ret)
- printf("%s: Error\n", msg);
- else
- printf("%s: Ok\n", msg);
- return ret;
-}
+ /* Clean up child */
+ FAIL_IF_MSG(kill(pid, SIGKILL) != 0, "Failed to kill child");
-char *desc[14] = {
- "perf cpu event -> ptrace thread event (Overlapping)",
- "perf cpu event -> ptrace thread event (Non-overlapping)",
- "perf thread event -> ptrace same thread event (Overlapping)",
- "perf thread event -> ptrace same thread event (Non-overlapping)",
- "perf thread event -> ptrace other thread event",
- "ptrace thread event -> perf kernel event",
- "ptrace thread event -> perf same thread event (Overlapping)",
- "ptrace thread event -> perf same thread event (Non-overlapping)",
- "ptrace thread event -> perf other thread event",
- "ptrace thread event -> perf cpu event (Overlapping)",
- "ptrace thread event -> perf cpu event (Non-overlapping)",
- "ptrace thread event -> perf same thread & cpu event (Overlapping)",
- "ptrace thread event -> perf same thread & cpu event (Non-overlapping)",
- "ptrace thread event -> perf other thread & cpu event",
-};
-
-static int test(pid_t child_pid)
-{
- int ret = TEST_PASS;
-
- ret |= do_test(desc[0], test1, child_pid);
- ret |= do_test(desc[1], test2, child_pid);
- ret |= do_test(desc[2], test3, child_pid);
- ret |= do_test(desc[3], test4, child_pid);
- ret |= do_test(desc[4], test5, child_pid);
- ret |= do_test(desc[5], test6, child_pid);
- ret |= do_test(desc[6], test7, child_pid);
- ret |= do_test(desc[7], test8, child_pid);
- ret |= do_test(desc[8], test9, child_pid);
- ret |= do_test(desc[9], test10, child_pid);
- ret |= do_test(desc[10], test11, child_pid);
- ret |= do_test(desc[11], test12, child_pid);
- ret |= do_test(desc[12], test13, child_pid);
- ret |= do_test(desc[13], test14, child_pid);
-
- return ret;
+ return 0;
}
-static void get_dbginfo(pid_t child_pid, struct ppc_debug_info *dbginfo)
+/*
+ * Tests the interaction between ptrace and perf when:
+ * 1. perf watches a value
+ * 2. ptrace watches a different value
+ * 3. The perf value is read, then the ptrace value is read immediately after
+ *
+ * A breakpoint implementation may accidentally misattribute/skip one of
+ * the ptrace or perf handlers, as interrupt based work is done after perf
+ * and before ptrace.
+ *
+ * We expect the perf counter to increment before the ptrace watchpoint
+ * triggers.
+ */
+int perf_then_ptrace_test(void)
{
- if (ptrace(PPC_PTRACE_GETHWDBGINFO, child_pid, NULL, dbginfo)) {
- perror("Can't get breakpoint info");
- exit(-1);
+ struct ppc_hw_breakpoint bp_info; /* ptrace breakpoint info */
+ int bp_id; /* Breakpoint handle of ptrace watchpoint */
+ int perf_fd; /* File descriptor of perf performance counter */
+ u64 perf_count; /* Most recently fetched perf performance counter value */
+ pid_t pid; /* PID of child process */
+ void *pc; /* Most recently fetched child PC value */
+ int status; /* Stop status of child after waitpid */
+ unsigned long perf_value; /* Dummy value to be watched by perf */
+ unsigned long ptrace_value; /* Dummy value to be watched by ptrace */
+ int err;
+
+ err = ptrace_fork_child(&pid);
+ if (err)
+ return err;
+
+ /*
+ * If we are the child, run a subroutine that reads the perf value,
+ * then reads the ptrace value with consecutive load instructions
+ */
+ if (!pid) {
+ perf_then_ptrace_child(&perf_value, &ptrace_value);
+ exit(0);
}
-}
-static int ptrace_perf_hwbreak(void)
-{
- int ret;
- pid_t child_pid;
- struct ppc_debug_info dbginfo;
+ err = check_watchpoints(pid);
+ if (err)
+ return err;
- child_pid = fork();
- if (!child_pid)
- return child();
+ /* Place a perf watchpoint counter */
+ perf_fd = perf_watchpoint_open(pid, &perf_value, sizeof(perf_value));
+ FAIL_IF_MSG(perf_fd < 0, "Failed to open perf performance counter");
- /* parent */
- wait(NULL); /* <-- child (SIGUSR1) */
+ /* Place a ptrace watchpoint */
+ ppc_ptrace_init_breakpoint(&bp_info, PPC_BREAKPOINT_TRIGGER_READ,
+ &ptrace_value, sizeof(ptrace_value));
+ bp_id = ppc_ptrace_sethwdbg(pid, &bp_info);
+ FAIL_IF_MSG(bp_id < 0, "Failed to set ptrace watchpoint");
- get_dbginfo(child_pid, &dbginfo);
- SKIP_IF(dbginfo.num_data_bps <= 1);
+ /* Let the child run. It should stop on the ptrace watchpoint */
+ FAIL_IF_MSG(ptrace_cont(pid, 0), "Failed to continue child");
- ret = perf_cpu_event_open(0, (__u64)perf_data1, sizeof(*perf_data1));
- SKIP_IF(ret < 0);
- close(ret);
+ FAIL_IF_MSG(waitpid(pid, &status, 0) == -1, "Failed to wait for child");
+ FAIL_IF_MSG(!WIFSTOPPED(status), "Child is not stopped");
+ FAIL_IF_MSG(ptrace_getreg_pc(pid, &pc), "Failed to get child PC");
+ FAIL_IF_MSG(pc != perf_then_ptrace_load2, "Child did not stop on ptrace load");
- ret = test(child_pid);
+ /* perf should have recorded the first load */
+ FAIL_IF_MSG(perf_read_counter(perf_fd, &perf_count), "Failed to read perf counter");
+ FAIL_IF_MSG(perf_count != 1, "perf counter did not increment");
- ptrace(PTRACE_CONT, child_pid, NULL, 0);
- return ret;
+ /* Clean up child */
+ FAIL_IF_MSG(kill(pid, SIGKILL) != 0, "Failed to kill child");
+
+ return 0;
}
int main(int argc, char *argv[])
{
- return test_harness(ptrace_perf_hwbreak, "ptrace-perf-hwbreak");
+ int err = 0;
+
+ err |= test_harness(same_watch_addr_test, "same_watch_addr");
+ err |= test_harness(perf_then_ptrace_test, "perf_then_ptrace");
+
+ return err;
}
* to the child.
*/
ret = ptrace_read_regs(pid, NT_PPC_PKEY, regs, 3);
- PARENT_SKIP_IF_UNSUPPORTED(ret, &info->child_sync);
+ PARENT_SKIP_IF_UNSUPPORTED(ret, &info->child_sync, "PKEYs not supported");
PARENT_FAIL_IF(ret, &info->child_sync);
info->amr1 = info->amr2 = regs[0];
int ret, status;
// TAR was added in v2.07
- SKIP_IF(!have_hwcap2(PPC_FEATURE2_ARCH_2_07));
+ SKIP_IF_MSG(!have_hwcap2(PPC_FEATURE2_ARCH_2_07), "TAR requires ISA 2.07 compatible hardware");
shm_id = shmget(IPC_PRIVATE, sizeof(int) * 3, 0777|IPC_CREAT);
pid = fork();
pid_t pid;
int ret, status;
- SKIP_IF(!have_htm());
- SKIP_IF(htm_is_synthetic());
+ SKIP_IF_MSG(!have_htm(), "Don't have transactional memory");
+ SKIP_IF_MSG(htm_is_synthetic(), "Transactional memory is synthetic");
shm_id = shmget(IPC_PRIVATE, sizeof(int) * 2, 0777|IPC_CREAT);
pid = fork();
if (pid < 0) {
pid_t pid;
int ret, status;
- SKIP_IF(!have_htm());
- SKIP_IF(htm_is_synthetic());
+ SKIP_IF_MSG(!have_htm(), "Don't have transactional memory");
+ SKIP_IF_MSG(htm_is_synthetic(), "Transactional memory is synthetic");
shm_id = shmget(IPC_PRIVATE, sizeof(int) * 3, 0777|IPC_CREAT);
pid = fork();
if (pid < 0) {
pid_t pid;
int ret, status;
- SKIP_IF(!have_htm());
- SKIP_IF(htm_is_synthetic());
+ SKIP_IF_MSG(!have_htm(), "Don't have transactional memory");
+ SKIP_IF_MSG(htm_is_synthetic(), "Transactional memory is synthetic");
shm_id = shmget(IPC_PRIVATE, sizeof(int) * 3, 0777|IPC_CREAT);
pid = fork();
if (pid == 0)
pid_t pid;
int ret, status, i;
- SKIP_IF(!have_htm());
- SKIP_IF(htm_is_synthetic());
+ SKIP_IF_MSG(!have_htm(), "Don't have transactional memory");
+ SKIP_IF_MSG(htm_is_synthetic(), "Transactional memory is synthetic");
shm_id = shmget(IPC_PRIVATE, sizeof(int) * 3, 0777|IPC_CREAT);
for (i = 0; i < 128; i++) {
pid_t pid;
int ret, status;
- SKIP_IF(!have_htm());
- SKIP_IF(htm_is_synthetic());
+ SKIP_IF_MSG(!have_htm(), "Don't have transactional memory");
+ SKIP_IF_MSG(htm_is_synthetic(), "Transactional memory is synthetic");
shm_id = shmget(IPC_PRIVATE, sizeof(struct shared), 0777|IPC_CREAT);
shm_id1 = shmget(IPC_PRIVATE, sizeof(int), 0777|IPC_CREAT);
pid = fork();
pid_t pid;
int ret, status;
- SKIP_IF(!have_htm());
- SKIP_IF(htm_is_synthetic());
+ SKIP_IF_MSG(!have_htm(), "Don't have transactional memory");
+ SKIP_IF_MSG(htm_is_synthetic(), "Transactional memory is synthetic");
shm_id = shmget(IPC_PRIVATE, sizeof(int) * 2, 0777|IPC_CREAT);
pid = fork();
if (pid == 0)
pid_t pid;
int ret, status, i;
- SKIP_IF(!have_htm());
- SKIP_IF(htm_is_synthetic());
+ SKIP_IF_MSG(!have_htm(), "Don't have transactional memory");
+ SKIP_IF_MSG(htm_is_synthetic(), "Transactional memory is synthetic");
shm_id = shmget(IPC_PRIVATE, sizeof(int) * 2, 0777|IPC_CREAT);
for (i = 0; i < 128; i++) {
pid_t pid;
int ret, status, i;
- SKIP_IF(!have_hwcap(PPC_FEATURE_HAS_VSX));
+ SKIP_IF_MSG(!have_hwcap(PPC_FEATURE_HAS_VSX), "Don't have VSX");
shm_id = shmget(IPC_PRIVATE, sizeof(int) * 2, 0777|IPC_CREAT);
+++ /dev/null
-../../../../../../arch/powerpc/include/asm/lppaca.h
\ No newline at end of file
--- /dev/null
+../../../../../../arch/powerpc/include/asm/vphn.h
\ No newline at end of file
projects / platform / kernel / linux-rpi.git / commitdiff