mm: thp: Avoid mapping a 64kB THP when anonymous COW-ed pages exist

For the given read-only file there might exist copy-on-write anonymous
pages (e.g. when application has modified something in its .data
section).

Presence of such pages causes that the retract_page_tables() function in
mm/khugepaged.c skips retracting their PTE entries, even if the original
(unmodified) file pages have been replaced by the khugepaged with 64kB
THP. Those COW-ed pages remain mapped with 4kB entries.

When read-fault happens on the adjacent page, kernel checks if it can be
mapped with 64kB entry. Without a check for non-NULL vma->anon_vma the
4kB entries for the COW-ed pages will be replaced by the 64kB entries for
the unmodiffied pages of the original file.

This issue has not been observed for 2MB THPs, because PMD-fault (related
to 2MB THP size) doesn't happen if there are any pages still mapped with
4kB TLB entries.

This fixes the random crashes when CONFIG_THP_CONSERVATIVE mode is
enabled.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Change-Id: Ic1d87b887f589894732e1c1066123aa88f4f582d

diff --git a/arch/arm64/include/asm/huge_mm.h b/arch/arm64/include/asm/huge_mm.h
index cc44800..c2fbc68 100644 (file)
--- a/arch/arm64/include/asm/huge_mm.h
+++ b/arch/arm64/include/asm/huge_mm.h
@@ -45,6 +45,13 @@ static inline bool transhuge_adv_vma_suitable(struct vm_area_struct *vma,
                if (((vma->vm_start >> PAGE_SHIFT) & HPAGE_CONT_PTE_CACHE_INDEX_MASK)
                        != (vma->vm_pgoff & HPAGE_CONT_PTE_CACHE_INDEX_MASK))
                        return false;
+
+               /*
+                * Do not create THPs if anonymous COW-ed 4k pages exist, see
+                * comment in mm/khugepaged.c:retract_page_tables() function.
+                */
+               if (vma->anon_vma)
+                       return false;
        }
 
        if (haddr < vma->vm_start || haddr + HPAGE_CONT_PTE_SIZE >= vma->vm_end)