Punit Agrawal [Tue, 22 Aug 2017 10:42:47 +0000 (11:42 +0100)]
arm64: hugetlb: Override set_huge_swap_pte_at() to support contiguous hugepages
The default implementation of set_huge_swap_pte_at() does not support
hugepages consisting of contiguous ptes. Override it to add support for
contiguous hugepages.
Signed-off-by: Punit Agrawal <punit.agrawal@arm.com>
Cc: David Woods <dwoods@mellanox.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Punit Agrawal [Tue, 22 Aug 2017 10:42:46 +0000 (11:42 +0100)]
arm64: hugetlb: Override huge_pte_clear() to support contiguous hugepages
The default huge_pte_clear() implementation does not clear contiguous
page table entries when it encounters contiguous hugepages that are
supported on arm64.
Fix this by overriding the default implementation to clear all the
entries associated with contiguous hugepages.
Signed-off-by: Punit Agrawal <punit.agrawal@arm.com>
Cc: David Woods <dwoods@mellanox.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Punit Agrawal [Tue, 22 Aug 2017 10:42:45 +0000 (11:42 +0100)]
arm64: hugetlb: Handle swap entries in huge_pte_offset() for contiguous hugepages
huge_pte_offset() was updated to correctly handle swap entries for
hugepages. With the addition of the size parameter, it is now possible
to disambiguate whether the request is for a regular hugepage or a
contiguous hugepage.
Fix huge_pte_offset() for contiguous hugepages by using the size to find
the correct page table entry.
Signed-off-by: Punit Agrawal <punit.agrawal@arm.com>
Cc: David Woods <dwoods@mellanox.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Steve Capper [Tue, 22 Aug 2017 10:42:44 +0000 (11:42 +0100)]
arm64: hugetlb: Add break-before-make logic for contiguous entries
It has become apparent that one has to take special care when modifying
attributes of memory mappings that employ the contiguous bit.
Both the requirement and the architecturally correct "Break-Before-Make"
technique of updating contiguous entries can be found described in:
ARM DDI 0487A.k_iss10775, "Misprogramming of the Contiguous bit",
page D4-1762.
The huge pte accessors currently replace the attributes of contiguous
pte entries in place thus can, on certain platforms, lead to TLB
conflict aborts or even erroneous results returned from TLB lookups.
This patch adds two helper functions -
* get_clear_flush(.) - clears a contiguous entry and returns the head
pte (whilst taking care to retain dirty bit information that could
have been modified by DBM).
* clear_flush(.) that clears a contiguous entry
A tlb invalidate is performed to then ensure that there is no
possibility of multiple tlb entries being present for the same region.
Cc: David Woods <dwoods@mellanox.com>
Signed-off-by: Steve Capper <steve.capper@arm.com>
(Added helper clear_flush(), updated commit log, and some cleanup)
Signed-off-by: Punit Agrawal <punit.agrawal@arm.com>
[catalin.marinas@arm.com: remove CONFIG_ARM64_HW_AFDBM check]
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Steve Capper [Tue, 22 Aug 2017 10:42:43 +0000 (11:42 +0100)]
arm64: hugetlb: Spring clean huge pte accessors
This patch aims to re-structure the huge pte accessors without affecting
their functionality. Control flow is changed to reduce indentation and
expanded use is made of post for loop variable modification.
It is then much easier to add break-before-make semantics in a subsequent
patch.
Cc: David Woods <dwoods@mellanox.com>
Signed-off-by: Steve Capper <steve.capper@arm.com>
Signed-off-by: Punit Agrawal <punit.agrawal@arm.com>
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Steve Capper [Tue, 22 Aug 2017 10:42:42 +0000 (11:42 +0100)]
arm64: hugetlb: Introduce pte_pgprot helper
Rather than xor pte bits in various places, use this helper function.
Cc: David Woods <dwoods@mellanox.com>
Signed-off-by: Steve Capper <steve.capper@arm.com>
Signed-off-by: Punit Agrawal <punit.agrawal@arm.com>
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Steve Capper [Tue, 22 Aug 2017 10:42:41 +0000 (11:42 +0100)]
arm64: hugetlb: set_huge_pte_at Add WARN_ON on !pte_present
This patch adds a WARN_ON to set_huge_pte_at as the accessor assumes
that entries to be written down are all present. (There are separate
accessors to clear huge ptes).
We will need to handle the !pte_present case where memory offlining
is used on hugetlb pages. swap and migration entries will be supplied
to set_huge_pte_at in this case.
Cc: David Woods <dwoods@mellanox.com>
Signed-off-by: Steve Capper <steve.capper@arm.com>
Signed-off-by: Punit Agrawal <punit.agrawal@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Hoeun Ryu [Thu, 17 Aug 2017 02:24:27 +0000 (11:24 +0900)]
arm64: kexec: have own crash_smp_send_stop() for crash dump for nonpanic cores
Commit 0ee5941 : (x86/panic: replace smp_send_stop() with kdump friendly
version in panic path) introduced crash_smp_send_stop() which is a weak
function and can be overridden by architecture codes to fix the side effect
caused by commit f06e515 : (kernel/panic.c: add "crash_kexec_post_
notifiers" option).
ARM64 architecture uses the weak version function and the problem is that
the weak function simply calls smp_send_stop() which makes other CPUs
offline and takes away the chance to save crash information for nonpanic
CPUs in machine_crash_shutdown() when crash_kexec_post_notifiers kernel
option is enabled.
Calling smp_send_crash_stop() in machine_crash_shutdown() is useless
because all nonpanic CPUs are already offline by smp_send_stop() in this
case and smp_send_crash_stop() only works against online CPUs.
The result is that secondary CPUs registers are not saved by
crash_save_cpu() and the vmcore file misreports these CPUs as being
offline.
crash_smp_send_stop() is implemented to fix this problem by replacing the
existing smp_send_crash_stop() and adding a check for multiple calling to
the function. The function (strong symbol version) saves crash information
for nonpanic CPUs and machine_crash_shutdown() tries to save crash
information for nonpanic CPUs only when crash_kexec_post_notifiers kernel
option is disabled.
* crash_kexec_post_notifiers : false
panic()
__crash_kexec()
machine_crash_shutdown()
crash_smp_send_stop() <= save crash dump for nonpanic cores
* crash_kexec_post_notifiers : true
panic()
crash_smp_send_stop() <= save crash dump for nonpanic cores
__crash_kexec()
machine_crash_shutdown()
crash_smp_send_stop() <= just return.
Signed-off-by: Hoeun Ryu <hoeun.ryu@gmail.com>
Reviewed-by: James Morse <james.morse@arm.com>
Tested-by: James Morse <james.morse@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Vladimir Murzin [Mon, 14 Aug 2017 08:55:47 +0000 (09:55 +0100)]
arm64: dma-mapping: Mark atomic_pool as __ro_after_init
atomic_pool is setup once while init stage and never changed after
that, so it is good candidate for __ro_after_init
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Vladimir Murzin [Mon, 14 Aug 2017 08:55:46 +0000 (09:55 +0100)]
arm64: dma-mapping: Do not pass data to gen_pool_set_algo()
gen_pool_first_fit_order_align() does not make use of additional data,
so pass plain NULL there.
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Catalin Marinas [Thu, 6 Jul 2017 10:53:08 +0000 (11:53 +0100)]
arm64: Remove the !CONFIG_ARM64_HW_AFDBM alternative code paths
Since the pte handling for hardware AF/DBM works even when the hardware
feature is not present, make the pte accessors implementation permanent
and remove the corresponding #ifdefs. The Kconfig option is kept as it
can still be used to disable the feature at the hardware level.
Reviewed-by: Will Deacon <will.deacon@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Catalin Marinas [Wed, 5 Jul 2017 09:59:42 +0000 (10:59 +0100)]
arm64: Ignore hardware dirty bit updates in ptep_set_wrprotect()
ptep_set_wrprotect() is only called on CoW mappings which are private
(!VM_SHARED) with the pte either read-only (!PTE_WRITE && PTE_RDONLY) or
writable and software-dirty (PTE_WRITE && !PTE_RDONLY && PTE_DIRTY).
There is no race with the hardware update of the dirty state: clearing
of PTE_RDONLY when PTE_WRITE (a.k.a. PTE_DBM) is set. This patch removes
the code setting the software PTE_DIRTY bit in ptep_set_wrprotect() as
superfluous. A VM_WARN_ONCE is introduced in case the above logic is
wrong or the core mm code changes its use of ptep_set_wrprotect().
Reviewed-by: Will Deacon <will.deacon@arm.com>
Acked-by: Steve Capper <steve.capper@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Catalin Marinas [Tue, 4 Jul 2017 18:04:18 +0000 (19:04 +0100)]
arm64: Move PTE_RDONLY bit handling out of set_pte_at()
Currently PTE_RDONLY is treated as a hardware only bit and not handled
by the pte_mkwrite(), pte_wrprotect() or the user PAGE_* definitions.
The set_pte_at() function is responsible for setting this bit based on
the write permission or dirty state. This patch moves the PTE_RDONLY
handling out of set_pte_at into the pte_mkwrite()/pte_wrprotect()
functions. The PAGE_* definitions to need to be updated to explicitly
include PTE_RDONLY when !PTE_WRITE.
The patch also removes the redundant PAGE_COPY(_EXEC) definitions as
they are identical to the corresponding PAGE_READONLY(_EXEC).
Reviewed-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Catalin Marinas [Thu, 6 Jul 2017 10:46:39 +0000 (11:46 +0100)]
kvm: arm64: Convert kvm_set_s2pte_readonly() from inline asm to cmpxchg()
To take advantage of the LSE atomic instructions and also make the code
cleaner, convert the kvm_set_s2pte_readonly() function to use the more
generic cmpxchg().
Cc: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Will Deacon <will.deacon@arm.com>
Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Acked-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Catalin Marinas [Mon, 26 Jun 2017 13:27:36 +0000 (14:27 +0100)]
arm64: Convert pte handling from inline asm to using (cmp)xchg
With the support for hardware updates of the access and dirty states,
the following pte handling functions had to be implemented using
exclusives: __ptep_test_and_clear_young(), ptep_get_and_clear(),
ptep_set_wrprotect() and ptep_set_access_flags(). To take advantage of
the LSE atomic instructions and also make the code cleaner, convert
these pte functions to use the more generic cmpxchg()/xchg().
Reviewed-by: Will Deacon <will.deacon@arm.com>
Acked-by: Mark Rutland <mark.rutland@arm.com>
Acked-by: Steve Capper <steve.capper@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Catalin Marinas [Fri, 18 Aug 2017 17:32:50 +0000 (18:32 +0100)]
Merge branch 'for-next/kernel-mode-neon' into for-next/core
* for-next/kernel-mode-neon:
arm64: neon/efi: Make EFI fpsimd save/restore variables static
arm64: neon: Forbid when irqs are disabled
arm64: neon: Export kernel_neon_busy to loadable modules
arm64: neon: Temporarily add a kernel_mode_begin_partial() definition
arm64: neon: Remove support for nested or hardirq kernel-mode NEON
arm64: neon: Allow EFI runtime services to use FPSIMD in irq context
arm64: fpsimd: Consistently use __this_cpu_ ops where appropriate
arm64: neon: Add missing header guard in <asm/neon.h>
arm64: neon: replace generic definition of may_use_simd()
Catalin Marinas [Fri, 18 Aug 2017 17:30:30 +0000 (18:30 +0100)]
Merge branch 'for-next/perf' of git://git./linux/kernel/git/will/linux into for-next/core
* 'for-next/perf' of git://git.kernel.org/pub/scm/linux/kernel/git/will/linux:
arm64: perf: add support for Cortex-A35
arm64: perf: add support for Cortex-A73
arm64: perf: Remove redundant entries from CPU-specific event maps
arm64: perf: Connect additional events to pmu counters
arm64: perf: Allow standard PMUv3 events to be extended by the CPU type
perf: xgene: Remove unnecessary managed resources cleanup
arm64: perf: Allow more than one cycle counter to be used
Dave Martin [Fri, 18 Aug 2017 13:53:47 +0000 (14:53 +0100)]
arm64: neon/efi: Make EFI fpsimd save/restore variables static
The percpu variables efi_fpsimd_state and efi_fpsimd_state_used,
used by the FPSIMD save/restore routines for EFI calls, are
unintentionally global.
There's no reason for anything outside fpsimd.c to touch these, so
this patch makes them static (as they should have been in the first
place).
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Catalin Marinas [Tue, 15 Aug 2017 17:40:58 +0000 (18:40 +0100)]
Merge branch 'arm64/vmap-stack' of git://git./linux/kernel/git/mark/linux into for-next/core
* 'arm64/vmap-stack' of git://git.kernel.org/pub/scm/linux/kernel/git/mark/linux:
arm64: add VMAP_STACK overflow detection
arm64: add on_accessible_stack()
arm64: add basic VMAP_STACK support
arm64: use an irq stack pointer
arm64: assembler: allow adr_this_cpu to use the stack pointer
arm64: factor out entry stack manipulation
efi/arm64: add EFI_KIMG_ALIGN
arm64: move SEGMENT_ALIGN to <asm/memory.h>
arm64: clean up irq stack definitions
arm64: clean up THREAD_* definitions
arm64: factor out PAGE_* and CONT_* definitions
arm64: kernel: remove {THREAD,IRQ_STACK}_START_SP
fork: allow arch-override of VMAP stack alignment
arm64: remove __die()'s stack dump
Mark Rutland [Fri, 14 Jul 2017 19:30:35 +0000 (20:30 +0100)]
arm64: add VMAP_STACK overflow detection
This patch adds stack overflow detection to arm64, usable when vmap'd stacks
are in use.
Overflow is detected in a small preamble executed for each exception entry,
which checks whether there is enough space on the current stack for the general
purpose registers to be saved. If there is not enough space, the overflow
handler is invoked on a per-cpu overflow stack. This approach preserves the
original exception information in ESR_EL1 (and where appropriate, FAR_EL1).
Task and IRQ stacks are aligned to double their size, enabling overflow to be
detected with a single bit test. For example, a 16K stack is aligned to 32K,
ensuring that bit 14 of the SP must be zero. On an overflow (or underflow),
this bit is flipped. Thus, overflow (of less than the size of the stack) can be
detected by testing whether this bit is set.
The overflow check is performed before any attempt is made to access the
stack, avoiding recursive faults (and the loss of exception information
these would entail). As logical operations cannot be performed on the SP
directly, the SP is temporarily swapped with a general purpose register
using arithmetic operations to enable the test to be performed.
This gives us a useful error message on stack overflow, as can be trigger with
the LKDTM overflow test:
[ 305.388749] lkdtm: Performing direct entry OVERFLOW
[ 305.395444] Insufficient stack space to handle exception!
[ 305.395482] ESR: 0x96000047 -- DABT (current EL)
[ 305.399890] FAR: 0xffff00000a5e7f30
[ 305.401315] Task stack: [0xffff00000a5e8000..0xffff00000a5ec000]
[ 305.403815] IRQ stack: [0xffff000008000000..0xffff000008004000]
[ 305.407035] Overflow stack: [0xffff80003efce4e0..0xffff80003efcf4e0]
[ 305.409622] CPU: 0 PID: 1219 Comm: sh Not tainted 4.13.0-rc3-00021-g9636aea #5
[ 305.412785] Hardware name: linux,dummy-virt (DT)
[ 305.415756] task:
ffff80003d051c00 task.stack:
ffff00000a5e8000
[ 305.419221] PC is at recursive_loop+0x10/0x48
[ 305.421637] LR is at recursive_loop+0x38/0x48
[ 305.423768] pc : [<
ffff00000859f330>] lr : [<
ffff00000859f358>] pstate:
40000145
[ 305.428020] sp :
ffff00000a5e7f50
[ 305.430469] x29:
ffff00000a5e8350 x28:
ffff80003d051c00
[ 305.433191] x27:
ffff000008981000 x26:
ffff000008f80400
[ 305.439012] x25:
ffff00000a5ebeb8 x24:
ffff00000a5ebeb8
[ 305.440369] x23:
ffff000008f80138 x22:
0000000000000009
[ 305.442241] x21:
ffff80003ce65000 x20:
ffff000008f80188
[ 305.444552] x19:
0000000000000013 x18:
0000000000000006
[ 305.446032] x17:
0000ffffa2601280 x16:
ffff0000081fe0b8
[ 305.448252] x15:
ffff000008ff546d x14:
000000000047a4c8
[ 305.450246] x13:
ffff000008ff7872 x12:
0000000005f5e0ff
[ 305.452953] x11:
ffff000008ed2548 x10:
000000000005ee8d
[ 305.454824] x9 :
ffff000008545380 x8 :
ffff00000a5e8770
[ 305.457105] x7 :
1313131313131313 x6 :
00000000000000e1
[ 305.459285] x5 :
0000000000000000 x4 :
0000000000000000
[ 305.461781] x3 :
0000000000000000 x2 :
0000000000000400
[ 305.465119] x1 :
0000000000000013 x0 :
0000000000000012
[ 305.467724] Kernel panic - not syncing: kernel stack overflow
[ 305.470561] CPU: 0 PID: 1219 Comm: sh Not tainted 4.13.0-rc3-00021-g9636aea #5
[ 305.473325] Hardware name: linux,dummy-virt (DT)
[ 305.475070] Call trace:
[ 305.476116] [<
ffff000008088ad8>] dump_backtrace+0x0/0x378
[ 305.478991] [<
ffff000008088e64>] show_stack+0x14/0x20
[ 305.481237] [<
ffff00000895a178>] dump_stack+0x98/0xb8
[ 305.483294] [<
ffff0000080c3288>] panic+0x118/0x280
[ 305.485673] [<
ffff0000080c2e9c>] nmi_panic+0x6c/0x70
[ 305.486216] [<
ffff000008089710>] handle_bad_stack+0x118/0x128
[ 305.486612] Exception stack(0xffff80003efcf3a0 to 0xffff80003efcf4e0)
[ 305.487334] f3a0:
0000000000000012 0000000000000013 0000000000000400 0000000000000000
[ 305.488025] f3c0:
0000000000000000 0000000000000000 00000000000000e1 1313131313131313
[ 305.488908] f3e0:
ffff00000a5e8770 ffff000008545380 000000000005ee8d ffff000008ed2548
[ 305.489403] f400:
0000000005f5e0ff ffff000008ff7872 000000000047a4c8 ffff000008ff546d
[ 305.489759] f420:
ffff0000081fe0b8 0000ffffa2601280 0000000000000006 0000000000000013
[ 305.490256] f440:
ffff000008f80188 ffff80003ce65000 0000000000000009 ffff000008f80138
[ 305.490683] f460:
ffff00000a5ebeb8 ffff00000a5ebeb8 ffff000008f80400 ffff000008981000
[ 305.491051] f480:
ffff80003d051c00 ffff00000a5e8350 ffff00000859f358 ffff00000a5e7f50
[ 305.491444] f4a0:
ffff00000859f330 0000000040000145 0000000000000000 0000000000000000
[ 305.492008] f4c0:
0001000000000000 0000000000000000 ffff00000a5e8350 ffff00000859f330
[ 305.493063] [<
ffff00000808205c>] __bad_stack+0x88/0x8c
[ 305.493396] [<
ffff00000859f330>] recursive_loop+0x10/0x48
[ 305.493731] [<
ffff00000859f358>] recursive_loop+0x38/0x48
[ 305.494088] [<
ffff00000859f358>] recursive_loop+0x38/0x48
[ 305.494425] [<
ffff00000859f358>] recursive_loop+0x38/0x48
[ 305.494649] [<
ffff00000859f358>] recursive_loop+0x38/0x48
[ 305.494898] [<
ffff00000859f358>] recursive_loop+0x38/0x48
[ 305.495205] [<
ffff00000859f358>] recursive_loop+0x38/0x48
[ 305.495453] [<
ffff00000859f358>] recursive_loop+0x38/0x48
[ 305.495708] [<
ffff00000859f358>] recursive_loop+0x38/0x48
[ 305.496000] [<
ffff00000859f358>] recursive_loop+0x38/0x48
[ 305.496302] [<
ffff00000859f358>] recursive_loop+0x38/0x48
[ 305.496644] [<
ffff00000859f358>] recursive_loop+0x38/0x48
[ 305.496894] [<
ffff00000859f358>] recursive_loop+0x38/0x48
[ 305.497138] [<
ffff00000859f358>] recursive_loop+0x38/0x48
[ 305.497325] [<
ffff00000859f3dc>] lkdtm_OVERFLOW+0x14/0x20
[ 305.497506] [<
ffff00000859f314>] lkdtm_do_action+0x1c/0x28
[ 305.497786] [<
ffff00000859f178>] direct_entry+0xe0/0x170
[ 305.498095] [<
ffff000008345568>] full_proxy_write+0x60/0xa8
[ 305.498387] [<
ffff0000081fb7f4>] __vfs_write+0x1c/0x128
[ 305.498679] [<
ffff0000081fcc68>] vfs_write+0xa0/0x1b0
[ 305.498926] [<
ffff0000081fe0fc>] SyS_write+0x44/0xa0
[ 305.499182] Exception stack(0xffff00000a5ebec0 to 0xffff00000a5ec000)
[ 305.499429] bec0:
0000000000000001 000000001c4cf5e0 0000000000000009 000000001c4cf5e0
[ 305.499674] bee0:
574f4c465245564f 0000000000000000 0000000000000000 8000000080808080
[ 305.499904] bf00:
0000000000000040 0000000000000038 fefefeff1b4bc2ff 7f7f7f7f7f7fff7f
[ 305.500189] bf20:
0101010101010101 0000000000000000 000000000047a4c8 0000000000000038
[ 305.500712] bf40:
0000000000000000 0000ffffa2601280 0000ffffc63f6068 00000000004b5000
[ 305.501241] bf60:
0000000000000001 000000001c4cf5e0 0000000000000009 000000001c4cf5e0
[ 305.501791] bf80:
0000000000000020 0000000000000000 00000000004b5000 000000001c4cc458
[ 305.502314] bfa0:
0000000000000000 0000ffffc63f7950 000000000040a3c4 0000ffffc63f70e0
[ 305.502762] bfc0:
0000ffffa2601268 0000000080000000 0000000000000001 0000000000000040
[ 305.503207] bfe0:
0000000000000000 0000000000000000 0000000000000000 0000000000000000
[ 305.503680] [<
ffff000008082fb0>] el0_svc_naked+0x24/0x28
[ 305.504720] Kernel Offset: disabled
[ 305.505189] CPU features: 0x002082
[ 305.505473] Memory Limit: none
[ 305.506181] ---[ end Kernel panic - not syncing: kernel stack overflow
This patch was co-authored by Ard Biesheuvel and Mark Rutland.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Will Deacon <will.deacon@arm.com>
Tested-by: Laura Abbott <labbott@redhat.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Mark Rutland [Tue, 1 Aug 2017 17:51:15 +0000 (18:51 +0100)]
arm64: add on_accessible_stack()
Both unwind_frame() and dump_backtrace() try to check whether a stack
address is sane to access, with very similar logic. Both will need
updating in order to handle overflow stacks.
Factor out this logic into a helper, so that we can avoid further
duplication when we add overflow stacks.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Will Deacon <will.deacon@arm.com>
Tested-by: Laura Abbott <labbott@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Mark Rutland [Fri, 21 Jul 2017 13:25:33 +0000 (14:25 +0100)]
arm64: add basic VMAP_STACK support
This patch enables arm64 to be built with vmap'd task and IRQ stacks.
As vmap'd stacks are mapped at page granularity, stacks must be a multiple of
PAGE_SIZE. This means that a 64K page kernel must use stacks of at least 64K in
size.
To minimize the increase in Image size, IRQ stacks are dynamically allocated at
boot time, rather than embedding the boot CPU's IRQ stack in the kernel image.
This patch was co-authored by Ard Biesheuvel and Mark Rutland.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Will Deacon <will.deacon@arm.com>
Tested-by: Laura Abbott <labbott@redhat.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Mark Rutland [Mon, 31 Jul 2017 20:17:03 +0000 (21:17 +0100)]
arm64: use an irq stack pointer
We allocate our IRQ stacks using a percpu array. This allows us to generate our
IRQ stack pointers with adr_this_cpu, but bloats the kernel Image with the boot
CPU's IRQ stack. Additionally, these are packed with other percpu variables,
and aren't guaranteed to have guard pages.
When we enable VMAP_STACK we'll want to vmap our IRQ stacks also, in order to
provide guard pages and to permit more stringent alignment requirements. Doing
so will require that we use a percpu pointer to each IRQ stack, rather than
allocating a percpu IRQ stack in the kernel image.
This patch updates our IRQ stack code to use a percpu pointer to the base of
each IRQ stack. This will allow us to change the way the stack is allocated
with minimal changes elsewhere. In some cases we may try to backtrace before
the IRQ stack pointers are initialised, so on_irq_stack() is updated to account
for this.
In testing with cyclictest, there was no measureable difference between using
adr_this_cpu (for irq_stack) and ldr_this_cpu (for irq_stack_ptr) in the IRQ
entry path.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Will Deacon <will.deacon@arm.com>
Tested-by: Laura Abbott <labbott@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Ard Biesheuvel [Sat, 15 Jul 2017 16:23:13 +0000 (17:23 +0100)]
arm64: assembler: allow adr_this_cpu to use the stack pointer
Given that adr_this_cpu already requires a temp register in addition
to the destination register, tweak the instruction sequence so that sp
may be used as well.
This will simplify switching to per-cpu stacks in subsequent patches. While
this limits the range of adr_this_cpu, to +/-4GiB, we don't currently use
adr_this_cpu in modules, and this is not problematic for the main kernel image.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
[Mark: add more commit text]
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Will Deacon <will.deacon@arm.com>
Tested-by: Laura Abbott <labbott@redhat.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Mark Rutland [Wed, 19 Jul 2017 16:24:49 +0000 (17:24 +0100)]
arm64: factor out entry stack manipulation
In subsequent patches, we will detect stack overflow in our exception
entry code, by verifying the SP after it has been decremented to make
space for the exception regs.
This verification code is small, and we can minimize its impact by
placing it directly in the vectors. To avoid redundant modification of
the SP, we also need to move the initial decrement of the SP into the
vectors.
As a preparatory step, this patch introduces kernel_ventry, which
performs this decrement, and updates the entry code accordingly.
Subsequent patches will fold SP verification into kernel_ventry.
There should be no functional change as a result of this patch.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
[Mark: turn into prep patch, expand commit msg]
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Will Deacon <will.deacon@arm.com>
Tested-by: Laura Abbott <labbott@redhat.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Mark Rutland [Fri, 14 Jul 2017 14:54:36 +0000 (15:54 +0100)]
efi/arm64: add EFI_KIMG_ALIGN
The EFI stub is intimately coupled with the kernel, and takes advantage
of this by relocating the kernel at a weaker alignment than the
documented boot protocol mandates.
However, it does so by assuming it can align the kernel to the segment
alignment, and assumes that this is 64K. In subsequent patches, we'll
have to consider other details to determine this de-facto alignment
constraint.
This patch adds a new EFI_KIMG_ALIGN definition that will track the
kernel's de-facto alignment requirements. Subsequent patches will modify
this as required.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Will Deacon <will.deacon@arm.com>
Tested-by: Laura Abbott <labbott@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Cc: Matt Fleming <matt@codeblueprint.co.uk>
Mark Rutland [Fri, 14 Jul 2017 14:38:43 +0000 (15:38 +0100)]
arm64: move SEGMENT_ALIGN to <asm/memory.h>
Currently we define SEGMENT_ALIGN directly in our vmlinux.lds.S.
This is unfortunate, as the EFI stub currently open-codes the same
number, and in future we'll want to fiddle with this.
This patch moves the definition to our <asm/memory.h>, where it can be
used by both vmlinux.lds.S and the EFI stub code.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Will Deacon <will.deacon@arm.com>
Tested-by: Laura Abbott <labbott@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Mark Rutland [Thu, 20 Jul 2017 11:26:48 +0000 (12:26 +0100)]
arm64: clean up irq stack definitions
Before we add yet another stack to the kernel, it would be nice to
ensure that we consistently organise stack definitions and related
helper functions.
This patch moves the basic IRQ stack defintions to <asm/memory.h> to
live with their task stack counterparts. Helpers used for unwinding are
moved into <asm/stacktrace.h>, where subsequent patches will add helpers
for other stacks. Includes are fixed up accordingly.
This patch is a pure refactoring -- there should be no functional
changes as a result of this patch.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Will Deacon <will.deacon@arm.com>
Tested-by: Laura Abbott <labbott@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Mark Rutland [Fri, 14 Jul 2017 15:39:21 +0000 (16:39 +0100)]
arm64: clean up THREAD_* definitions
Currently we define THREAD_SIZE and THREAD_SIZE_ORDER separately, with
the latter dependent on particular CONFIG_ARM64_*K_PAGES definitions.
This is somewhat opaque, and will get in the way of future modifications
to THREAD_SIZE.
This patch cleans this up, defining both in terms of a common
THREAD_SHIFT, and using PAGE_SHIFT to calculate THREAD_SIZE_ORDER,
rather than using a number of definitions dependent on config symbols.
Subsequent patches will make use of this to alter the stack size used in
some configurations.
At the same time, these are moved into <asm/memory.h>, which will avoid
circular include issues in subsequent patches. To ensure that existing
code isn't adversely affected, <asm/thread_info.h> is updated to
transitively include these definitions.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Will Deacon <will.deacon@arm.com>
Tested-by: Laura Abbott <labbott@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Mark Rutland [Fri, 14 Jul 2017 18:43:56 +0000 (19:43 +0100)]
arm64: factor out PAGE_* and CONT_* definitions
Some headers rely on PAGE_* definitions from <asm/page.h>, but cannot
include this due to potential circular includes. For example, a number
of definitions in <asm/memory.h> rely on PAGE_SHIFT, and <asm/page.h>
includes <asm/memory.h>.
This requires users of these definitions to include both headers, which
is fragile and error-prone.
This patch ameliorates matters by moving the basic definitions out to a
new header, <asm/page-def.h>. Both <asm/page.h> and <asm/memory.h> are
updated to include this, avoiding this fragility, and avoiding the
possibility of circular include dependencies.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Will Deacon <will.deacon@arm.com>
Tested-by: Laura Abbott <labbott@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Ard Biesheuvel [Thu, 20 Jul 2017 16:15:45 +0000 (17:15 +0100)]
arm64: kernel: remove {THREAD,IRQ_STACK}_START_SP
For historical reasons, we leave the top 16 bytes of our task and IRQ
stacks unused, a practice used to ensure that the SP can always be
masked to find the base of the current stack (historically, where
thread_info could be found).
However, this is not necessary, as:
* When an exception is taken from a task stack, we decrement the SP by
S_FRAME_SIZE and stash the exception registers before we compare the
SP against the task stack. In such cases, the SP must be at least
S_FRAME_SIZE below the limit, and can be safely masked to determine
whether the task stack is in use.
* When transitioning to an IRQ stack, we'll place a dummy frame onto the
IRQ stack before enabling asynchronous exceptions, or executing code
we expect to trigger faults. Thus, if an exception is taken from the
IRQ stack, the SP must be at least 16 bytes below the limit.
* We no longer mask the SP to find the thread_info, which is now found
via sp_el0. Note that historically, the offset was critical to ensure
that cpu_switch_to() found the correct stack for new threads that
hadn't yet executed ret_from_fork().
Given that, this initial offset serves no purpose, and can be removed.
This brings us in-line with other architectures (e.g. x86) which do not
rely on this masking.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
[Mark: rebase, kill THREAD_START_SP, commit msg additions]
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Will Deacon <will.deacon@arm.com>
Tested-by: Laura Abbott <labbott@redhat.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Mark Rutland [Fri, 14 Jul 2017 11:23:09 +0000 (12:23 +0100)]
fork: allow arch-override of VMAP stack alignment
In some cases, an architecture might wish its stacks to be aligned to a
boundary larger than THREAD_SIZE. For example, using an alignment of
double THREAD_SIZE can allow for stack overflows smaller than
THREAD_SIZE to be detected by checking a single bit of the stack
pointer.
This patch allows architectures to override the alignment of VMAP'd
stacks, by defining THREAD_ALIGN. Where not defined, this defaults to
THREAD_SIZE, as is the case today.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Will Deacon <will.deacon@arm.com>
Tested-by: Laura Abbott <labbott@redhat.com>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Cc: linux-kernel@vger.kernel.org
Mark Rutland [Mon, 7 Aug 2017 11:10:51 +0000 (12:10 +0100)]
arm64: remove __die()'s stack dump
Our __die() implementation tries to dump the stack memory, in addition
to a backtrace, which is problematic.
For contemporary 16K stacks, this can be a lot of data, which can take a
long time to dump, and can push other useful context out of the kernel's
printk ringbuffer (and/or a user's scrollback buffer on an attached
console).
Additionally, the code implicitly assumes that the SP is on the task's
stack, and tries to dump everything between the SP and the highest task
stack address. When the SP points at an IRQ stack (or is corrupted),
this makes the kernel attempt to dump vast amounts of VA space. With
vmap'd stacks, this may result in erroneous accesses to peripherals.
This patch removes the memory dump, leaving us to rely on the backtrace,
and other means of dumping stack memory such as kdump.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Will Deacon <will.deacon@arm.com>
Tested-by: Laura Abbott <labbott@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Dou Liyang [Wed, 26 Jul 2017 13:34:26 +0000 (21:34 +0800)]
arm64: numa: Remove the unused parent_node() macro
Commit
a7be6e5a7f8d ("mm: drop useless local parameters of
__register_one_node()") removes the last user of parent_node().
The parent_node() macro in ARM64 platform is unnecessary.
Remove it for cleanup.
Reported-by: Michael Ellerman <mpe@ellerman.id.au>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Dou Liyang <douly.fnst@cn.fujitsu.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: linux-arm-kernel@lists.infradead.org
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Kevin Brodsky [Fri, 4 Aug 2017 17:17:00 +0000 (10:17 -0700)]
arm64: compat: Remove leftover variable declaration
Commit
a1d5ebaf8ccd ("arm64: big-endian: don't treat code as data when
copying sigret code") moved the 32-bit sigreturn trampoline code from
the aarch32_sigret_code array to kuser32.S. The commit removed the
array definition from signal32.c, but not its declaration in
signal32.h. Remove the leftover declaration.
Signed-off-by: Kevin Brodsky <kevin.brodsky@arm.com>
Signed-off-by: Mark Salyzyn <salyzyn@android.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Arnd Bergmann [Thu, 10 Aug 2017 16:45:22 +0000 (17:45 +0100)]
ACPI/IORT: Fix build regression without IOMMU
A recent change reintroduced a bug that had previously been
fixed by commit
d49f2dedf33b ("ACPI/IORT: Fix CONFIG_IOMMU_API
dependency"):
drivers/acpi/arm64/iort.c: In function 'iort_iommu_configure':
drivers/acpi/arm64/iort.c:829:26: error: 'struct iommu_fwspec' has no member named 'ops'
Replace a direct reference to iommu_fwspec->ops with a helper function
call to fix the issue.
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Arnd Bergmann [Thu, 10 Aug 2017 14:52:31 +0000 (16:52 +0200)]
arm64: fix pmem interface definition
Defining the two functions as 'static inline' and exporting them
leads to the interesting case where we can use the interface
from loadable modules, but not from built-in drivers, as shown
in this link failure:
vers/nvdimm/claim.o: In function `nsio_rw_bytes':
claim.c:(.text+0x1b8): undefined reference to `arch_invalidate_pmem'
drivers/nvdimm/pmem.o: In function `pmem_dax_flush':
pmem.c:(.text+0x11c): undefined reference to `arch_wb_cache_pmem'
drivers/nvdimm/pmem.o: In function `pmem_make_request':
pmem.c:(.text+0x5a4): undefined reference to `arch_invalidate_pmem'
pmem.c:(.text+0x650): undefined reference to `arch_invalidate_pmem'
pmem.c:(.text+0x6d4): undefined reference to `arch_invalidate_pmem'
This removes the bogus 'static inline'.
Fixes:
d50e071fdaa3 ("arm64: Implement pmem API support")
Acked-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Julien Thierry [Wed, 9 Aug 2017 16:46:39 +0000 (17:46 +0100)]
arm64: perf: add support for Cortex-A35
The Cortex-A35 uses some implementation defined perf events.
The Cortex-A35 derives from the Cortex-A53 core, using the same event mapings
based on Cortex-A35 TRM r0p2, section C2.3 - Performance monitoring events
(pages C2-562 to C2-565).
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Julien Thierry [Wed, 9 Aug 2017 16:46:38 +0000 (17:46 +0100)]
arm64: perf: add support for Cortex-A73
The Cortex-A73 uses some implementation defined perf events.
This patch sets up the necessary mapping for Cortex-A73.
Mappings are based on Cortex-A73 TRM r0p2, section 11.9 Events
(pages 11-457 to 11-460).
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Will Deacon [Tue, 8 Aug 2017 16:11:27 +0000 (17:11 +0100)]
arm64: perf: Remove redundant entries from CPU-specific event maps
Now that the event mapping code always looks into the PMUv3 events
before any extended mappings, the extended mappings can be reduced to
only those events that are not discoverable through the PMCEID registers.
Signed-off-by: Will Deacon <will.deacon@arm.com>
Julien Thierry [Tue, 25 Jul 2017 16:27:36 +0000 (17:27 +0100)]
arm64: perf: Connect additional events to pmu counters
Last level caches and node events were almost never connected in current
supported cores.
We connect last level caches to the actual last level within the core and
node events are connected to bus accesses.
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Robin Murphy [Thu, 10 Aug 2017 09:49:21 +0000 (10:49 +0100)]
arm64: uaccess: Add the uaccess_flushcache.c file
The uaccess_flushcache.c file was inadvertently dropped by the
maintainer in a previous commit. Add it back.
Fixes:
5d7bdeb1eeb2 ("arm64: uaccess: Implement *_flushcache variants")
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Ard Biesheuvel [Thu, 13 Jul 2017 17:16:01 +0000 (18:16 +0100)]
md/raid6: implement recovery using ARM NEON intrinsics
Provide a NEON accelerated implementation of the recovery algorithm,
which supersedes the default byte-by-byte one.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Ard Biesheuvel [Thu, 13 Jul 2017 17:16:00 +0000 (18:16 +0100)]
md/raid6: use faster multiplication for ARM NEON delta syndrome
The P/Q left side optimization in the delta syndrome simply involves
repeatedly multiplying a value by polynomial 'x' in GF(2^8). Given
that 'x * x * x * x' equals 'x^4' even in the polynomial world, we
can accelerate this substantially by performing up to 4 such operations
at once, using the NEON instructions for polynomial multiplication.
Results on a Cortex-A57 running in 64-bit mode:
Before:
-------
raid6: neonx1 xor() 1680 MB/s
raid6: neonx2 xor() 2286 MB/s
raid6: neonx4 xor() 3162 MB/s
raid6: neonx8 xor() 3389 MB/s
After:
------
raid6: neonx1 xor() 2281 MB/s
raid6: neonx2 xor() 3362 MB/s
raid6: neonx4 xor() 3787 MB/s
raid6: neonx8 xor() 4239 MB/s
While we're at it, simplify MASK() by using a signed shift rather than
a vector compare involving a temp register.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Catalin Marinas [Wed, 9 Aug 2017 14:39:09 +0000 (15:39 +0100)]
Merge tag 'arm64-iort-for-v4.14' of git://git./linux/kernel/git/lpieralisi/linux into for-next/core
Two patches for the 4.14 release merge window:
- IORT PCI aliases detection improvement to cater for systems with
complex PCI topologies that current code mishandles (R.Murphy)
- IORT SMMUv3 proximity domain (ie NUMA) handling (respective ACPICA
changes will be merged separately) (G. Kulkarni)
* tag 'arm64-iort-for-v4.14' of git://git.kernel.org/pub/scm/linux/kernel/git/lpieralisi/linux:
ACPI/IORT: numa: Add numa node mapping for smmuv3 devices
ACPI/IORT: Handle PCI aliases properly for IOMMUs
Catalin Marinas [Wed, 9 Aug 2017 14:37:49 +0000 (15:37 +0100)]
Merge branch 'arm64/exception-stack' of git://git./linux/kernel/git/mark/linux into for-next/core
* 'arm64/exception-stack' of git://git.kernel.org/pub/scm/linux/kernel/git/mark/linux:
arm64: unwind: remove sp from struct stackframe
arm64: unwind: reference pt_regs via embedded stack frame
arm64: unwind: disregard frame.sp when validating frame pointer
arm64: unwind: avoid percpu indirection for irq stack
arm64: move non-entry code out of .entry.text
arm64: consistently use bl for C exception entry
arm64: Add ASM_BUG()
Dave Martin [Wed, 9 Aug 2017 10:43:28 +0000 (11:43 +0100)]
arm64: neon: Forbid when irqs are disabled
Currently, may_use_simd() can return true if IRQs are disabled. If
the caller goes ahead and calls kernel_neon_begin(), this can
result in use of local_bh_enable() in an unsafe context.
In particular, __efi_fpsimd_begin() may do this when calling EFI as
part of system shutdown.
This patch ensures that callers don't think they can use
kernel_neon_begin() in such a context.
Acked-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Ard Biesheuvel [Sun, 23 Jul 2017 08:05:38 +0000 (09:05 +0100)]
arm64: unwind: remove sp from struct stackframe
The unwind code sets the sp member of struct stackframe to
'frame pointer + 0x10' unconditionally, without regard for whether
doing so produces a legal value. So let's simply remove it now that
we have stopped using it anyway.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Ard Biesheuvel [Sat, 22 Jul 2017 17:45:33 +0000 (18:45 +0100)]
arm64: unwind: reference pt_regs via embedded stack frame
As it turns out, the unwind code is slightly broken, and probably has
been for a while. The problem is in the dumping of the exception stack,
which is intended to dump the contents of the pt_regs struct at each
level in the call stack where an exception was taken and routed to a
routine marked as __exception (which means its stack frame is right
below the pt_regs struct on the stack).
'Right below the pt_regs struct' is ill defined, though: the unwind
code assigns 'frame pointer + 0x10' to the .sp member of the stackframe
struct at each level, and dump_backtrace() happily dereferences that as
the pt_regs pointer when encountering an __exception routine. However,
the actual size of the stack frame created by this routine (which could
be one of many __exception routines we have in the kernel) is not known,
and so frame.sp is pretty useless to figure out where struct pt_regs
really is.
So it seems the only way to ensure that we can find our struct pt_regs
when walking the stack frames is to put it at a known fixed offset of
the stack frame pointer that is passed to such __exception routines.
The simplest way to do that is to put it inside pt_regs itself, which is
the main change implemented by this patch. As a bonus, doing this allows
us to get rid of a fair amount of cruft related to walking from one stack
to the other, which is especially nice since we intend to introduce yet
another stack for overflow handling once we add support for vmapped
stacks. It also fixes an inconsistency where we only add a stack frame
pointing to ELR_EL1 if we are executing from the IRQ stack but not when
we are executing from the task stack.
To consistly identify exceptions regs even in the presence of exceptions
taken from entry code, we must check whether the next frame was created
by entry text, rather than whether the current frame was crated by
exception text.
To avoid backtracing using PCs that fall in the idmap, or are controlled
by userspace, we must explcitly zero the FP and LR in startup paths, and
must ensure that the frame embedded in pt_regs is zeroed upon entry from
EL0. To avoid these NULL entries showin in the backtrace, unwind_frame()
is updated to avoid them.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
[Mark: compare current frame against .entry.text, avoid bogus PCs]
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Dmitry Safonov [Wed, 26 Jul 2017 17:07:37 +0000 (20:07 +0300)]
arm64/vdso: Support mremap() for vDSO
vDSO VMA address is saved in mm_context for the purpose of using
restorer from vDSO page to return to userspace after signal handling.
In Checkpoint Restore in Userspace (CRIU) project we place vDSO VMA
on restore back to the place where it was on the dump.
With the exception for x86 (where there is API to map vDSO with
arch_prctl()), we move vDSO inherited from CRIU task to restoree
position by mremap().
CRIU does support arm64 architecture, but kernel doesn't update
context.vdso pointer after mremap(). Which results in translation
fault after signal handling on restored application:
https://github.com/xemul/criu/issues/288
Make vDSO code track the VMA address by supplying .mremap() fops
the same way it's done for x86 and arm32 by:
commit
b059a453b1cf ("x86/vdso: Add mremap hook to vm_special_mapping")
commit
280e87e98c09 ("ARM: 8683/1: ARM32: Support mremap() for sigpage/vDSO").
Cc: Russell King <rmk+kernel@armlinux.org.uk>
Cc: linux-arm-kernel@lists.infradead.org
Cc: Cyrill Gorcunov <gorcunov@openvz.org>
Cc: Pavel Emelyanov <xemul@virtuozzo.com>
Cc: Christopher Covington <cov@codeaurora.org>
Reviewed-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Dmitry Safonov <dsafonov@virtuozzo.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Robin Murphy [Tue, 25 Jul 2017 10:55:43 +0000 (11:55 +0100)]
arm64: uaccess: Implement *_flushcache variants
Implement the set of copy functions with guarantees of a clean cache
upon completion necessary to support the pmem driver.
Reviewed-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Robin Murphy [Tue, 25 Jul 2017 10:55:42 +0000 (11:55 +0100)]
arm64: Implement pmem API support
Add a clean-to-point-of-persistence cache maintenance helper, and wire
up the basic architectural support for the pmem driver based on it.
Reviewed-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
[catalin.marinas@arm.com: move arch_*_pmem() functions to arch/arm64/mm/flush.c]
[catalin.marinas@arm.com: change dmb(sy) to dmb(osh)]
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Robin Murphy [Tue, 25 Jul 2017 10:55:41 +0000 (11:55 +0100)]
arm64: Handle trapped DC CVAP
Cache clean to PoP is subject to the same access controls as to PoC, so
if we are trapping userspace cache maintenance with SCTLR_EL1.UCI, we
need to be prepared to handle it. To avoid getting into complicated
fights with binutils about ARMv8.2 options, we'll just cheat and use the
raw SYS instruction rather than the 'proper' DC alias.
Reviewed-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Robin Murphy [Tue, 25 Jul 2017 10:55:40 +0000 (11:55 +0100)]
arm64: Expose DC CVAP to userspace
The ARMv8.2-DCPoP feature introduces persistent memory support to the
architecture, by defining a point of persistence in the memory
hierarchy, and a corresponding cache maintenance operation, DC CVAP.
Expose the support via HWCAP and MRS emulation.
Reviewed-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Robin Murphy [Tue, 25 Jul 2017 10:55:39 +0000 (11:55 +0100)]
arm64: Convert __inval_cache_range() to area-based
__inval_cache_range() is already the odd one out among our data cache
maintenance routines as the only remaining range-based one; as we're
going to want an invalidation routine to call from C code for the pmem
API, let's tweak the prototype and name to bring it in line with the
clean operations, and to make its relationship with __dma_inv_area()
neatly mirror that of __clean_dcache_area_poc() and __dma_clean_area().
The loop clearing the early page tables gets mildly massaged in the
process for the sake of consistency.
Reviewed-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Robin Murphy [Tue, 25 Jul 2017 10:55:38 +0000 (11:55 +0100)]
arm64: mm: Fix set_memory_valid() declaration
Clearly, set_memory_valid() has never been seen in the same room as its
declaration... Whilst the type mismatch is such that kexec probably
wasn't broken in practice, fix it to match the definition as it should.
Fixes:
9b0aa14e3155 ("arm64: mm: add set_memory_valid()")
Reviewed-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Will Deacon [Tue, 8 Aug 2017 15:58:33 +0000 (16:58 +0100)]
arm64: perf: Allow standard PMUv3 events to be extended by the CPU type
Rather than continue adding CPU-specific event maps, instead look up by
default in the PMUv3 event map and only fallback to the CPU-specific maps
if either the event isn't described by PMUv3, or it is described but
the PMCEID registers say that it is unsupported by the current CPU.
Signed-off-by: Will Deacon <will.deacon@arm.com>
Ganapatrao Kulkarni [Wed, 2 Aug 2017 17:58:25 +0000 (10:58 -0700)]
ACPI/IORT: numa: Add numa node mapping for smmuv3 devices
ARM IORT specification(rev. C) has added provision to define proximity
domain in SMMUv3 IORT table. Adding required code to parse Proximity
domain and set numa_node of smmv3 platform devices.
Signed-off-by: Ganapatrao Kulkarni <ganapatrao.kulkarni@cavium.com>
[lorenzo.pieralisi@arm.com: update pr_info()/commit log]
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Ard Biesheuvel [Sat, 22 Jul 2017 11:48:34 +0000 (12:48 +0100)]
arm64: unwind: disregard frame.sp when validating frame pointer
Currently, when unwinding the call stack, we validate the frame pointer
of each frame against frame.sp, whose value is not clearly defined, and
which makes it more difficult to link stack frames together across
different stacks. It is far better to simply check whether the frame
pointer itself points into a valid stack.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Mark Rutland [Thu, 20 Jul 2017 13:01:01 +0000 (14:01 +0100)]
arm64: unwind: avoid percpu indirection for irq stack
Our IRQ_STACK_PTR() and on_irq_stack() helpers both take a cpu argument,
used to generate a percpu address. In all cases, they are passed
{raw_,}smp_processor_id(), so this parameter is redundant.
Since {raw_,}smp_processor_id() use a percpu variable internally, this
approach means we generate a percpu offset to find the current cpu, then
use this to index an array of percpu offsets, which we then use to find
the current CPU's IRQ stack pointer. Thus, most of the work is
redundant.
Instead, we can consistently use raw_cpu_ptr() to generate the CPU's
irq_stack pointer by simply adding the percpu offset to the irq_stack
address, which is simpler in both respects.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Mark Rutland [Wed, 26 Jul 2017 15:05:20 +0000 (16:05 +0100)]
arm64: move non-entry code out of .entry.text
Currently, cpu_switch_to and ret_from_fork both live in .entry.text,
though neither form the critical path for an exception entry.
In subsequent patches, we will require that code in .entry.text is part
of the critical path for exception entry, for which we can assume
certain properties (e.g. the presence of exception regs on the stack).
Neither cpu_switch_to nor ret_from_fork will meet these requirements, so
we must move them out of .entry.text. To ensure that neither are kprobed
after being moved out of .entry.text, we must explicitly blacklist them,
requiring a new NOKPROBE() asm helper.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Mark Rutland [Wed, 26 Jul 2017 10:14:53 +0000 (11:14 +0100)]
arm64: consistently use bl for C exception entry
In most cases, our exception entry assembly branches to C handlers with
a BL instruction, but in cases where we do not expect to return, we use
B instead.
While this is correct today, it means that backtraces for fatal
exceptions miss the entry assembly (as the LR is stale at the point we
call C code), while non-fatal exceptions have the entry assembly in the
LR. In subsequent patches, we will need the LR to be set in these cases
in order to backtrace reliably.
This patch updates these sites to use a BL, ensuring consistency, and
preparing for backtrace rework. An ASM_BUG() is added after each of
these new BLs, which both catches unexpected returns, and ensures that
the LR value doesn't point to another function label.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Mark Rutland [Wed, 26 Jul 2017 13:41:40 +0000 (14:41 +0100)]
arm64: Add ASM_BUG()
Currently. we can only use BUG() from C code, though there are
situations where we would like an equivalent mechanism in assembly code.
This patch refactors our BUG() definition such that it can be used in
either C or assembly, in the form of a new ASM_BUG().
The refactoring requires the removal of escape sequences, such as '\n'
and '\t', but these aren't strictly necessary as we can use ';' to
terminate assembler statements.
The low-level assembly is factored out into <asm/asm-bug.h>, with
<asm/bug.h> retained as the C wrapper.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dave Martin <dave.martin@arm.com>
Cc: James Morse <james.morse@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Robin Murphy [Fri, 4 Aug 2017 16:42:06 +0000 (17:42 +0100)]
ACPI/IORT: Handle PCI aliases properly for IOMMUs
When a PCI device has DMA quirks, we need to ensure that an upstream
IOMMU knows about all possible aliases, since the presence of a DMA
quirk does not preclude the device still also emitting transactions
(e.g. MSIs) on its 'real' RID. Similarly, the rules for bridge aliasing
are relatively complex, and some bridges may only take ownership of
transactions under particular transient circumstances, leading again to
multiple RIDs potentially being seen at the IOMMU for the given device.
Take all this into account in iort_iommu_configure() by mapping every
RID produced by the alias walk, not just whichever one comes out last.
Since adding any more internal PTR_ERR() juggling would have confused me
no end, a bit of refactoring happens in the process - we know where to
find the ops if everything succeeded, so we're free to just pass regular
error codes around up until then.
CC: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
CC: Hanjun Guo <hanjun.guo@linaro.org>
CC: Sudeep Holla <sudeep.holla@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
[lorenzo.pieralisi@arm.com: tagged __get_pci_rid __maybe_unused]
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Tai Nguyen [Thu, 13 Jul 2017 18:19:08 +0000 (11:19 -0700)]
perf: xgene: Remove unnecessary managed resources cleanup
Managed resources in the driver should be automatically cleaned up on
driver detach. It's unnecessary to manually free/unmmap these resources.
One of the manual cleanup causes static checkers to complain.
The bug is reported by Dan Carpenter <dan.carpenter@oracle.com> in [1]
[1] https://www.spinics.net/lists/arm-kernel/msg593012.html
This patch gets rid of all the unnecessary manual cleanup and properly
unregister all the registered PMU devices by the driver on driver detach.
Signed-off-by: Tai Nguyen <ttnguyen@apm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Pratyush Anand [Sat, 1 Jul 2017 06:33:35 +0000 (12:03 +0530)]
arm64: perf: Allow more than one cycle counter to be used
Currently:
$ perf stat -e cycles:u -e cycles:k true
Performance counter stats for 'true':
2,24,699 cycles:u
<not counted> cycles:k (0.00%)
0.
000788087 seconds time elapsed
We can not count more than one cycle counter in one instance,because we
allow to map cycle counter into PMCCNTR_EL0 only. However, if I did not
miss anything then specification do not prohibit to use PMEVCNTR<n>_EL0
for cycle count as well.
Modify the code so that it still prefers to use PMCCNTR_EL0 for cycle
counter, however allow to use PMEVCNTR<n>_EL0 if PMCCNTR_EL0 is already
in use.
After this patch:
$ perf stat -e cycles:u -e cycles:k true
Performance counter stats for 'true':
2,17,310 cycles:u
7,40,009 cycles:k
0.
000764149 seconds time elapsed
Signed-off-by: Pratyush Anand <panand@redhat.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Catalin Marinas [Mon, 7 Aug 2017 11:36:35 +0000 (12:36 +0100)]
arm64: neon: Export kernel_neon_busy to loadable modules
may_use_simd() can be invoked from loadable modules and it accesses
kernel_neon_busy. Make sure that the latter is exported.
Fixes:
cb84d11e1625 ("arm64: neon: Remove support for nested or hardirq kernel-mode NEON")
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Julien Thierry [Fri, 4 Aug 2017 08:31:42 +0000 (09:31 +0100)]
arm64: Decode information from ESR upon mem faults
When receiving unhandled faults from the CPU, description is very sparse.
Adding information about faults decoded from ESR.
Added defines to esr.h corresponding ESR fields. Values are based on ARM
Archtecture Reference Manual (DDI 0487B.a), section D7.2.28 ESR_ELx, Exception
Syndrome Register (ELx) (pages D7-2275 to D7-2280).
New output is of the form:
[ 77.818059] Mem abort info:
[ 77.820826] Exception class = DABT (current EL), IL = 32 bits
[ 77.826706] SET = 0, FnV = 0
[ 77.829742] EA = 0, S1PTW = 0
[ 77.832849] Data abort info:
[ 77.835713] ISV = 0, ISS = 0x00000070
[ 77.839522] CM = 0, WnR = 1
Signed-off-by: Julien Thierry <julien.thierry@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
[catalin.marinas@arm.com: fix "%lu" in a pr_alert() call]
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Dave Martin [Tue, 1 Aug 2017 14:35:54 +0000 (15:35 +0100)]
arm64: Abstract syscallno manipulation
The -1 "no syscall" value is written in various ways, shared with
the user ABI in some places, and generally obscure.
This patch attempts to make things a little more consistent and
readable by replacing all these uses with a single #define. A
couple of symbolic helpers are provided to clarify the intent
further.
Because the in-syscall check in do_signal() is changed from >= 0 to
!= NO_SYSCALL by this patch, different behaviour may be observable
if syscallno is set to values less than -1 by a tracer. However,
this is not different from the behaviour that is already observable
if a tracer sets syscallno to a value >= __NR_(compat_)syscalls.
It appears that this can cause spurious syscall restarting, but
that is not a new behaviour either, and does not appear harmful.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Dave Martin [Tue, 1 Aug 2017 14:35:53 +0000 (15:35 +0100)]
arm64: syscallno is secretly an int, make it official
The upper 32 bits of the syscallno field in thread_struct are
handled inconsistently, being sometimes zero extended and sometimes
sign-extended. In fact, only the lower 32 bits seem to have any
real significance for the behaviour of the code: it's been OK to
handle the upper bits inconsistently because they don't matter.
Currently, the only place I can find where those bits are
significant is in calling trace_sys_enter(), which may be
unintentional: for example, if a compat tracer attempts to cancel a
syscall by passing -1 to (COMPAT_)PTRACE_SET_SYSCALL at the
syscall-enter-stop, it will be traced as syscall
4294967295
rather than -1 as might be expected (and as occurs for a native
tracer doing the same thing). Elsewhere, reads of syscallno cast
it to an int or truncate it.
There's also a conspicuous amount of code and casting to bodge
around the fact that although semantically an int, syscallno is
stored as a u64.
Let's not pretend any more.
In order to preserve the stp x instruction that stores the syscall
number in entry.S, this patch special-cases the layout of struct
pt_regs for big endian so that the newly 32-bit syscallno field
maps onto the low bits of the stored value. This is not beautiful,
but benchmarking of the getpid syscall on Juno suggests indicates a
minor slowdown if the stp is split into an stp x and stp w.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Linus Torvalds [Mon, 7 Aug 2017 01:44:49 +0000 (18:44 -0700)]
Linux 4.13-rc4
Linus Torvalds [Sun, 6 Aug 2017 23:11:34 +0000 (16:11 -0700)]
Merge tag 'platform-drivers-x86-v4.13-4' of git://git.infradead.org/linux-platform-drivers-x86
Pull x86 platform driver fix from Darren Hart:
"Fix loop preventing some platforms from waking up via the power button
in s2idle:
- intel-vbtn: match power button on press rather than release"
* tag 'platform-drivers-x86-v4.13-4' of git://git.infradead.org/linux-platform-drivers-x86:
platform/x86: intel-vbtn: match power button on press rather than release
Linus Torvalds [Sun, 6 Aug 2017 19:31:17 +0000 (12:31 -0700)]
Merge tag 'ext4_for_linus_stable' of git://git./linux/kernel/git/tytso/ext4
Pull ext4 fixes from Ted Ts'o:
"A large number of ext4 bug fixes and cleanups for v4.13"
* tag 'ext4_for_linus_stable' of git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4:
ext4: fix copy paste error in ext4_swap_extents()
ext4: fix overflow caused by missing cast in ext4_resize_fs()
ext4, project: expand inode extra size if possible
ext4: cleanup ext4_expand_extra_isize_ea()
ext4: restructure ext4_expand_extra_isize
ext4: fix forgetten xattr lock protection in ext4_expand_extra_isize
ext4: make xattr inode reads faster
ext4: inplace xattr block update fails to deduplicate blocks
ext4: remove unused mode parameter
ext4: fix warning about stack corruption
ext4: fix dir_nlink behaviour
ext4: silence array overflow warning
ext4: fix SEEK_HOLE/SEEK_DATA for blocksize < pagesize
ext4: release discard bio after sending discard commands
ext4: convert swap_inode_data() over to use swap() on most of the fields
ext4: error should be cleared if ea_inode isn't added to the cache
ext4: Don't clear SGID when inheriting ACLs
ext4: preserve i_mode if __ext4_set_acl() fails
ext4: remove unused metadata accounting variables
ext4: correct comment references to ext4_ext_direct_IO()
Linus Torvalds [Sun, 6 Aug 2017 18:52:01 +0000 (11:52 -0700)]
Merge branch 'upstream' of git://git.linux-mips.org/ralf/upstream-linus
Pull MIPS fixes from Ralf Baechle:
"This fixes two build issues for ralink platforms, both due to missing
#includes which used to be included indirectly via other headers"
* 'upstream' of git://git.linux-mips.org/pub/scm/ralf/upstream-linus:
MIPS: ralink: mt7620: Add missing header
MIPS: ralink: Fix build error due to missing header
Dmitry V. Levin [Sat, 5 Aug 2017 20:00:50 +0000 (23:00 +0300)]
Fix compat_sys_sigpending breakage
The latest change of compat_sys_sigpending in commit
8f13621abced
("sigpending(): move compat to native") has broken it in two ways.
First, it tries to write 4 bytes more than userspace expects:
sizeof(old_sigset_t) == sizeof(long) == 8 instead of
sizeof(compat_old_sigset_t) == sizeof(u32) == 4.
Second, on big endian architectures these bytes are being written in the
wrong order.
This bug was found by strace test suite.
Reported-by: Anatoly Pugachev <matorola@gmail.com>
Inspired-by: Eugene Syromyatnikov <evgsyr@gmail.com>
Fixes:
8f13621abced ("sigpending(): move compat to native")
Signed-off-by: Dmitry V. Levin <ldv@altlinux.org>
Acked-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Maninder Singh [Sun, 6 Aug 2017 05:33:07 +0000 (01:33 -0400)]
ext4: fix copy paste error in ext4_swap_extents()
This bug was found by a static code checker tool for copy paste
problems.
Signed-off-by: Maninder Singh <maninder1.s@samsung.com>
Signed-off-by: Vaneet Narang <v.narang@samsung.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Jerry Lee [Sun, 6 Aug 2017 05:18:31 +0000 (01:18 -0400)]
ext4: fix overflow caused by missing cast in ext4_resize_fs()
On a 32-bit platform, the value of n_blcoks_count may be wrong during
the file system is resized to size larger than 2^32 blocks. This may
caused the superblock being corrupted with zero blocks count.
Fixes:
1c6bd7173d66
Signed-off-by: Jerry Lee <jerrylee@qnap.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Cc: stable@vger.kernel.org # 3.7+
Miao Xie [Sun, 6 Aug 2017 05:00:49 +0000 (01:00 -0400)]
ext4, project: expand inode extra size if possible
When upgrading from old format, try to set project id
to old file first time, it will return EOVERFLOW, but if
that file is dirtied(touch etc), changing project id will
be allowed, this might be confusing for users, we could
try to expand @i_extra_isize here too.
Reported-by: Zhang Yi <yi.zhang@huawei.com>
Signed-off-by: Miao Xie <miaoxie@huawei.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Miao Xie [Sun, 6 Aug 2017 04:55:48 +0000 (00:55 -0400)]
ext4: cleanup ext4_expand_extra_isize_ea()
Clean up some goto statement, make ext4_expand_extra_isize_ea() clearer.
Signed-off-by: Miao Xie <miaoxie@huawei.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Miao Xie [Sun, 6 Aug 2017 04:40:01 +0000 (00:40 -0400)]
ext4: restructure ext4_expand_extra_isize
Current ext4_expand_extra_isize just tries to expand extra isize, if
someone is holding xattr lock or some check fails, it will give up.
So rename its name to ext4_try_to_expand_extra_isize.
Besides that, we clean up unnecessary check and move some relative checks
into it.
Signed-off-by: Miao Xie <miaoxie@huawei.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Miao Xie [Sun, 6 Aug 2017 04:27:38 +0000 (00:27 -0400)]
ext4: fix forgetten xattr lock protection in ext4_expand_extra_isize
We should avoid the contention between the i_extra_isize update and
the inline data insertion, so move the xattr trylock in front of
i_extra_isize update.
Signed-off-by: Miao Xie <miaoxie@huawei.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tahsin Erdogan [Sun, 6 Aug 2017 04:07:01 +0000 (00:07 -0400)]
ext4: make xattr inode reads faster
ext4_xattr_inode_read() currently reads each block sequentially while
waiting for io operation to complete before moving on to the next
block. This prevents request merging in block layer.
Add a ext4_bread_batch() function that starts reads for all blocks
then optionally waits for them to complete. A similar logic is used
in ext4_find_entry(), so update that code to use the new function.
Signed-off-by: Tahsin Erdogan <tahsin@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Tahsin Erdogan [Sun, 6 Aug 2017 02:41:42 +0000 (22:41 -0400)]
ext4: inplace xattr block update fails to deduplicate blocks
When an xattr block has a single reference, block is updated inplace
and it is reinserted to the cache. Later, a cache lookup is performed
to see whether an existing block has the same contents. This cache
lookup will most of the time return the just inserted entry so
deduplication is not achieved.
Running the following test script will produce two xattr blocks which
can be observed in "File ACL: " line of debugfs output:
mke2fs -b 1024 -I 128 -F -O extent /dev/sdb 1G
mount /dev/sdb /mnt/sdb
touch /mnt/sdb/{x,y}
setfattr -n user.1 -v aaa /mnt/sdb/x
setfattr -n user.2 -v bbb /mnt/sdb/x
setfattr -n user.1 -v aaa /mnt/sdb/y
setfattr -n user.2 -v bbb /mnt/sdb/y
debugfs -R 'stat x' /dev/sdb | cat
debugfs -R 'stat y' /dev/sdb | cat
This patch defers the reinsertion to the cache so that we can locate
other blocks with the same contents.
Signed-off-by: Tahsin Erdogan <tahsin@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Andreas Dilger <adilger@dilger.ca>
Tahsin Erdogan [Sun, 6 Aug 2017 02:15:45 +0000 (22:15 -0400)]
ext4: remove unused mode parameter
ext4_alloc_file_blocks() does not use its mode parameter. Remove it.
Signed-off-by: Tahsin Erdogan <tahsin@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Arnd Bergmann [Sun, 6 Aug 2017 01:57:46 +0000 (21:57 -0400)]
ext4: fix warning about stack corruption
After commit
62d1034f53e3 ("fortify: use WARN instead of BUG for now"),
we get a warning about possible stack overflow from a memcpy that
was not strictly bounded to the size of the local variable:
inlined from 'ext4_mb_seq_groups_show' at fs/ext4/mballoc.c:2322:2:
include/linux/string.h:309:9: error: '__builtin_memcpy': writing between 161 and 1116 bytes into a region of size 160 overflows the destination [-Werror=stringop-overflow=]
We actually had a bug here that would have been found by the warning,
but it was already fixed last year in commit
30a9d7afe70e ("ext4: fix
stack memory corruption with 64k block size").
This replaces the fixed-length structure on the stack with a variable-length
structure, using the correct upper bound that tells the compiler that
everything is really fine here. I also change the loop count to check
for the same upper bound for consistency, but the existing code is
already correct here.
Note that while clang won't allow certain kinds of variable-length arrays
in structures, this particular instance is fine, as the array is at the
end of the structure, and the size is strictly bounded.
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Sat, 5 Aug 2017 23:47:34 +0000 (19:47 -0400)]
ext4: fix dir_nlink behaviour
The dir_nlink feature has been enabled by default for new ext4
filesystems since e2fsprogs-1.41 in 2008, and was automatically
enabled by the kernel for older ext4 filesystems since the
dir_nlink feature was added with ext4 in kernel 2.6.28+ when
the subdirectory count exceeded EXT4_LINK_MAX-1.
Automatically adding the file system features such as dir_nlink is
generally frowned upon, since it could cause the file system to not be
mountable on older kernel, thus preventing the administrator from
rolling back to an older kernel if necessary.
In this case, the administrator might also want to disable the feature
because glibc's fts_read() function does not correctly optimize
directory traversal for directories that use st_nlinks field of 1 to
indicate that the number of links in the directory are not tracked by
the file system, and could fail to traverse the full directory
hierarchy. Fortunately, in the past ten years very few users have
complained about incomplete file system traversal by glibc's
fts_read().
This commit also changes ext4_inc_count() to allow i_nlinks to reach
the full EXT4_LINK_MAX links on the parent directory (including "."
and "..") before changing i_links_count to be 1.
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=196405
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Dan Carpenter [Sat, 5 Aug 2017 23:00:31 +0000 (19:00 -0400)]
ext4: silence array overflow warning
I get a static checker warning:
fs/ext4/ext4.h:3091 ext4_set_de_type()
error: buffer overflow 'ext4_type_by_mode' 15 <= 15
It seems unlikely that we would hit this read overflow in real life, but
it's also simple enough to make the array 16 bytes instead of 15.
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Jan Kara [Sat, 5 Aug 2017 21:43:24 +0000 (17:43 -0400)]
ext4: fix SEEK_HOLE/SEEK_DATA for blocksize < pagesize
ext4_find_unwritten_pgoff() does not properly handle a situation when
starting index is in the middle of a page and blocksize < pagesize. The
following command shows the bug on filesystem with 1k blocksize:
xfs_io -f -c "falloc 0 4k" \
-c "pwrite 1k 1k" \
-c "pwrite 3k 1k" \
-c "seek -a -r 0" foo
In this example, neither lseek(fd, 1024, SEEK_HOLE) nor lseek(fd, 2048,
SEEK_DATA) will return the correct result.
Fix the problem by neglecting buffers in a page before starting offset.
Reported-by: Andreas Gruenbacher <agruenba@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Signed-off-by: Jan Kara <jack@suse.cz>
CC: stable@vger.kernel.org # 3.8+
Mario Limonciello [Fri, 4 Aug 2017 17:00:06 +0000 (12:00 -0500)]
platform/x86: intel-vbtn: match power button on press rather than release
This fixes a problem where the system gets stuck in a loop
unable to wakeup via power button in s2idle.
The problem happens because:
- press power button:
- system emits 0xc0 (power press), event ignored
- system emits 0xc1 (power release), event processed,
emited as KEY_POWER
- set wakeup_mode to true
- system goes to s2idle
- press power button
- system emits 0xc0 (power press), wakeup_mode is true,
system wakes
- system emits 0xc1 (power release), event processed,
emited as KEY_POWER
- system goes to s2idle again
To avoid this situation, process the presses (which matches what
intel-hid does too).
Verified on an Dell XPS 9365
Signed-off-by: Mario Limonciello <mario.limonciello@dell.com>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Darren Hart (VMware) <dvhart@infradead.org>
Linus Torvalds [Sat, 5 Aug 2017 21:09:26 +0000 (14:09 -0700)]
Merge tag 'media/v4.13-2' of git://git./linux/kernel/git/mchehab/linux-media
Pull media fixes from Mauro Carvalho Chehab:
"This series is larger than I would like to submit for -rc4. My
original intent were to sent it to either -rc2 or -rc3. Unfortunately,
due to my vacations, I got a lot of pending stuff after my return, and
had to do some biz trips, with prevented me to send this earlier.
Several fixes:
- some fixes at atomisp staging driver
- several gcc 7 warning fixes
- cleanup media SVG files, in order to fix PDF build on some distros
- fix random Kconfig build of venus driver
- some fixes for the venus driver
- some changes from semaphone to mutex in ngene's driver
- some locking fixes at dib0700 driver
- several fixes on ngene's driver and frontends to make it properly
support some new boards added on Kernel 4.13
- some fixes to CEC drivers
- omap_vout: vrfb: convert to dmaengine
- docs-rst: document EBUSY for VIDIOC_S_FMT
Please notice that the big diffstat changes here are at the SVG files.
Visually, the images look the same, but the file size is now a lot
smaller than before, and they don't use some XML tags that would cause
them to be badly parsed by some ImageMagick versions, or to require a
lot of memory by TeTex, with would break PDF output on some
distributions"
* tag 'media/v4.13-2' of git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-media: (68 commits)
media: atomisp2: array underflow in imx_enum_frame_size()
media: atomisp2: array underflow in ap1302_enum_frame_size()
media: atomisp2: Array underflow in atomisp_enum_input()
media: platform: davinci: drop VPFE_CMD_S_CCDC_RAW_PARAMS
media: platform: davinci: return -EINVAL for VPFE_CMD_S_CCDC_RAW_PARAMS ioctl
media: venus: don't abuse dma_alloc for non-DMA allocations
media: venus: hfi: fix error handling in hfi_sys_init_done()
media: venus: fix compile-test build on non-qcom ARM platform
media: venus: mark PM functions as __maybe_unused
media: cec-notifier: small improvements
media: pulse8-cec: persistent_config should be off by default
media: cec: cec_transmit_attempt_done: ignore CEC_TX_STATUS_MAX_RETRIES
media: staging: atomisp: array underflow in ioctl
media: lirc: LIRC_GET_REC_RESOLUTION should return microseconds
media: svg: avoid too long lines
media: svg files: simplify files
media: selection.svg: simplify the SVG file
media: vimc: set id_table for platform drivers
media: staging: atomisp: disable warnings with cc-disable-warning
media: davinci: variable 'common' set but not used
...
Daeho Jeong [Sat, 5 Aug 2017 17:11:57 +0000 (13:11 -0400)]
ext4: release discard bio after sending discard commands
We've changed the discard command handling into parallel manner.
But, in this change, I forgot decreasing the usage count of the bio
which was used to send discard request. I'm sorry about that.
Fixes:
a015434480dc ("ext4: send parallel discards on commit completions")
Signed-off-by: Daeho Jeong <daeho.jeong@samsung.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Jan Kara <jack@suse.cz>
Linus Torvalds [Sat, 5 Aug 2017 13:55:13 +0000 (06:55 -0700)]
Merge tag 'gpio-v4.13-2' of git://git./linux/kernel/git/linusw/linux-gpio
Pull GPIO fixes from Linus Walleij:
- LP87565: set the proper output level for direction_output.
- stm32: fix the kernel build by selecting the hierarchical irqdomain
symbol properly - this happens to be done in the pin control
framework but whatever, it had dependencies to GPIO so we need to
apply it here.
- Select the hierarchical IRQ domain also for Xgene.
- Fix wakeups to work on MXC.
- Fix up the device tree binding on Exar that went astray, also add the
right bindings.
- Fix the unwanted events for edges from the library.
- Fix the unbalanced chanined IRQ on the Tegra.
* tag 'gpio-v4.13-2' of git://git.kernel.org/pub/scm/linux/kernel/git/linusw/linux-gpio:
gpio: tegra: fix unbalanced chained_irq_enter/exit
gpiolib: skip unwanted events, don't convert them to opposite edge
gpio: exar: Use correct property prefix and document bindings
gpio: gpio-mxc: Fix: higher 16 GPIOs usable as wake source
gpio: xgene-sb: select IRQ_DOMAIN_HIERARCHY
pinctrl: stm32: select IRQ_DOMAIN_HIERARCHY instead of depends on
gpio: lp87565: Set proper output level and direction for direction_output
MAINTAINERS: Add entry for Whiskey Cove PMIC GPIO driver
Linus Torvalds [Fri, 4 Aug 2017 23:45:29 +0000 (16:45 -0700)]
Merge tag 'clk-fixes-for-linus' of git://git./linux/kernel/git/clk/linux
Pull clk fixes from Stephen Boyd:
"A handful of critical fixes for changes introduce this merge window.
- The TI sci_clk_get() API was pretty broken and nobody noticed.
- There were some CPUfreq crashes on C.H.I.P devices because we
failed to propagate rates up the clk tree.
- Also, the Intel Atom PMC clk driver needs to mark a clk critical if
the firmware has it enabled already so that audio doesn't get
killed on Baytrail.
- Gemini devices have a dead serial console because the reset control
usage in the serial driver assume one method of reset that gemini
doesn't support (this will be fixed in the next version in the
reset framework so this is the small fix for -rc series).
- Finally we have two rate calculation fixes, one for Exynos and one
for Meson SoCs, that fix rate inconsistencies"
* tag 'clk-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/clk/linux:
clk: keystone: sci-clk: Fix sci_clk_get
clk: meson: mpll: fix mpll0 fractional part ignored
clk: samsung: exynos5420: The EPLL rate table corrections
clk: sunxi-ng: sun5i: Add clk_set_rate_parent to the CPU clock
clk: x86: Do not gate clocks enabled by the firmware
clk: gemini: Fix reset regression
Linus Torvalds [Fri, 4 Aug 2017 22:18:27 +0000 (15:18 -0700)]
Merge tag 'for-linus' of git://git./virt/kvm/kvm
Pull KVM fixes from Radim Krčmář:
"ARM:
- Yet another race with VM destruction plugged
- A set of small vgic fixes
x86:
- Preserve pending INIT
- RCU fixes in paravirtual async pf, VM teardown, and VMXOFF
emulation
- nVMX interrupt injection and dirty tracking fixes
- initialize to make UBSAN happy"
* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm:
KVM: arm/arm64: vgic: Use READ_ONCE fo cmpxchg
KVM: nVMX: Fix interrupt window request with "Acknowledge interrupt on exit"
KVM: nVMX: mark vmcs12 pages dirty on L2 exit
kvm: nVMX: don't flush VMCS12 during VMXOFF or VCPU teardown
KVM: nVMX: do not pin the VMCS12
KVM: avoid using rcu_dereference_protected
KVM: X86: init irq->level in kvm_pv_kick_cpu_op
KVM: X86: Fix loss of pending INIT due to race
KVM: async_pf: make rcu irq exit if not triggered from idle task
KVM: nVMX: fixes to nested virt interrupt injection
KVM: nVMX: do not fill vm_exit_intr_error_code in prepare_vmcs12
KVM: arm/arm64: Handle hva aging while destroying the vm
KVM: arm/arm64: PMU: Fix overflow interrupt injection
KVM: arm/arm64: Fix bug in advertising KVM_CAP_MSI_DEVID capability
Linus Torvalds [Fri, 4 Aug 2017 22:16:09 +0000 (15:16 -0700)]
Merge branch 'x86-urgent-for-linus' of git://git./linux/kernel/git/tip/tip
Pull x86 fix from Thomas Gleixner:
"The recent irq core changes unearthed API abuse in the HPET code,
which manifested itself in a suspend/resume regression.
The fix replaces the cruft with the proper function calls and cures
the regression"
* 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
x86/hpet: Cure interface abuse in the resume path
Linus Torvalds [Fri, 4 Aug 2017 22:14:09 +0000 (15:14 -0700)]
Merge branch 'timers-urgent-for-linus' of git://git./linux/kernel/git/tip/tip
Pull timer fix from Thomas Gleixner:
"A single fix for a multiplication overflow in the timer code on 32bit
systems"
* 'timers-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
timers: Fix overflow in get_next_timer_interrupt
Linus Torvalds [Fri, 4 Aug 2017 22:12:15 +0000 (15:12 -0700)]
Merge tag 'armsoc-fixes' of git://git./linux/kernel/git/arm/arm-soc
Pull ARM SoC fixes from Arnd Bergmann:
"This comes a bit later than I planned, and as a consequence is a
larger than it should be.
Most of the changes are devicetree fixes, across lots of platforms:
Renesas, Samsung Exynos, Marvell EBU, TI OMAP, Rockchips, Amlogic
Meson, Sigma Desings Tango, Allwinner SUNxi and TI Davinci.
Also across many platforms, I applied an older series of simple
randconfig build fixes. This includes making the CONFIG_MTD_XIP option
compile again, which had been broken for many years and probably has
not been missed, but it felt wrong to just remove it completely.
The only other changes are:
- We enable HWSPINLOCK in defconfig to get some Qualcomm boards to
work out of the box.
- A few regression fixes for Texas Instruments OMAP2+.
- A boot regression fix for the Renesas regulator quirk.
- A suspend/resume fix for Uniphier SoCs, fixing the resume of the
system bus"
* tag 'armsoc-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc: (43 commits)
ARM: dts: tango4: Request RGMII RX and TX clock delays
bus: uniphier-system-bus: set up registers when resuming
ARM64: dts: marvell: armada-37xx: Fix the number of GPIO on south bridge
ARM: shmobile: rcar-gen2: Fix deadlock in regulator quirk
arm64: defconfig: enable missing HWSPINLOCK
ARM: pxa: select both FB and FB_W100 for eseries
ARM: ixp4xx: fix ioport_unmap definition
ARM: ep93xx: use ARM_PATCH_PHYS_VIRT correctly
ARM: mmp: mark usb_dma_mask as __maybe_unused
ARM: omap2: mark unused functions as __maybe_unused
ARM: omap1: avoid unused variable warning
ARM: sirf: mark sirfsoc_init_late as __maybe_unused
ARM: ixp4xx: use normal prototype for {read,write}s{b,w,l}
ARM: omap1/ams-delta: warn about failed regulator enable
ARM: rpc: rename RAM_SIZE macro
ARM: w90x900: normalize clk API
ARM: ep93xx: normalize clk API
ARM: dts: sun8i: a83t: Switch to CCU device tree binding macros
arm64: allwinner: sun50i-a64: Correct emac register size
ARM: dts: sunxi: h3/h5: Correct emac register size
...
Linus Torvalds [Fri, 4 Aug 2017 19:11:48 +0000 (12:11 -0700)]
Merge tag 'arm64-fixes' of git://git./linux/kernel/git/arm64/linux
Pull arm64 fixes from Will Deacon:
"Here are some more arm64 fixes for 4.13. The main one is the PTE race
with the hardware walker, but there are a couple of other things too.
- Report correct timer frequency to userspace when trapping
CNTFRQ_EL0
- Fix race with hardware page table updates when updating access
flags
- Silence clang overflow warning in VA_START and PAGE_OFFSET
calculations"
* tag 'arm64-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux:
arm64: avoid overflow in VA_START and PAGE_OFFSET
arm64: Fix potential race with hardware DBM in ptep_set_access_flags()
arm64: Use arch_timer_get_rate when trapping CNTFRQ_EL0
Linus Torvalds [Fri, 4 Aug 2017 17:17:45 +0000 (10:17 -0700)]
Merge git://git./linux/kernel/git/davem/sparc
Pull sparc fixes from David Miller:
- block interrupts properly across the entire MMU context change (both
the hw MMU context change and the TSB table change) so that we don't
get a perf event interrupt in the middle. From Rob Gardner.
- be sure to register hugepages early enough, from Nitin Gupta.
- UltraSPARC-III user copy exception handling would return garbage for
the copied length in some circumstances.
* git://git.kernel.org/pub/scm/linux/kernel/git/davem/sparc:
sparc64: Fix exception handling in UltraSPARC-III memcpy.
sbus: Convert to using %pOF instead of full_name
sparc: defconfig: Cleanup from old Kconfig options
sparc64: Register hugepages during arch init
sparc64: Prevent perf from running during super critical sections
Linus Torvalds [Fri, 4 Aug 2017 17:15:11 +0000 (10:15 -0700)]
Merge tag 'ceph-for-4.13-rc4' of git://github.com/ceph/ceph-client
Pull ceph fixes from Ilya Dryomov:
"A bunch of fixes and follow-ups for -rc1 Luminous patches: issues with
->reencode_message() and last minute RADOS semantic changes in
v12.1.2"
* tag 'ceph-for-4.13-rc4' of git://github.com/ceph/ceph-client:
libceph: make RECOVERY_DELETES feature create a new interval
libceph: upmap semantic changes
crush: assume weight_set != null imples weight_set_size > 0
libceph: fallback for when there isn't a pool-specific choose_arg
libceph: don't call ->reencode_message() more than once per message
libceph: make encode_request_*() work with r_mempool requests