mm: avoid data corruption on CoW fault into PFN-mapped VMA
authorKirill A. Shutemov <kirill@shutemov.name>
Fri, 6 Mar 2020 06:28:32 +0000 (22:28 -0800)
committerLinus Torvalds <torvalds@linux-foundation.org>
Fri, 6 Mar 2020 13:06:09 +0000 (07:06 -0600)
commitc3e5ea6ee574ae5e845a40ac8198de1fb63bb3ab
treeec7eb9d9a887d28057702901810fcd74fecfb679
parent8a8683ad9ba48b4b52a57f013513d1635c1ca5c4
mm: avoid data corruption on CoW fault into PFN-mapped VMA

Jeff Moyer has reported that one of xfstests triggers a warning when run
on DAX-enabled filesystem:

WARNING: CPU: 76 PID: 51024 at mm/memory.c:2317 wp_page_copy+0xc40/0xd50
...
wp_page_copy+0x98c/0xd50 (unreliable)
do_wp_page+0xd8/0xad0
__handle_mm_fault+0x748/0x1b90
handle_mm_fault+0x120/0x1f0
__do_page_fault+0x240/0xd70
do_page_fault+0x38/0xd0
handle_page_fault+0x10/0x30

The warning happens on failed __copy_from_user_inatomic() which tries to
copy data into a CoW page.

This happens because of race between MADV_DONTNEED and CoW page fault:

CPU0 CPU1
 handle_mm_fault()
   do_wp_page()
     wp_page_copy()
       do_wp_page()
madvise(MADV_DONTNEED)
  zap_page_range()
    zap_pte_range()
      ptep_get_and_clear_full()
      <TLB flush>
 __copy_from_user_inatomic()
 sees empty PTE and fails
 WARN_ON_ONCE(1)
 clear_page()

The solution is to re-try __copy_from_user_inatomic() under PTL after
checking that PTE is matches the orig_pte.

The second copy attempt can still fail, like due to non-readable PTE, but
there's nothing reasonable we can do about, except clearing the CoW page.

Reported-by: Jeff Moyer <jmoyer@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Tested-by: Jeff Moyer <jmoyer@redhat.com>
Cc: <stable@vger.kernel.org>
Cc: Justin He <Justin.He@arm.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Link: http://lkml.kernel.org/r/20200218154151.13349-1-kirill.shutemov@linux.intel.com
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
mm/memory.c