s390/cmma: fix handling of swapper_pg_dir and invalid_pg_dir

[ Upstream commit 84bb41d5df48868055d159d9247b80927f1f70f9 ]

If the cmma no-dat feature is available the kernel page tables are walked
to identify and mark all pages which are used for address translation (all
region, segment, and page tables). In a subsequent loop all other pages are
marked as "no-dat" pages with the ESSA instruction.

This information is visible to the hypervisor, so that the hypervisor can
optimize purging of guest TLB entries. All pages used for swapper_pg_dir
and invalid_pg_dir are incorrectly marked as no-dat, which in turn can
result in incorrect guest TLB flushes.

Fix this by marking those pages correctly as being used for DAT.

Cc: <stable@vger.kernel.org>
Reviewed-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>

diff --git a/arch/s390/mm/page-states.c b/arch/s390/mm/page-states.c
index 00e7b08..79a037f 100644 (file)
--- a/arch/s390/mm/page-states.c
+++ b/arch/s390/mm/page-states.c
@@ -181,6 +181,12 @@ void __init cmma_init_nodat(void)
                return;
        /* Mark pages used in kernel page tables */
        mark_kernel_pgd();
+       page = virt_to_page(&swapper_pg_dir);
+       for (i = 0; i < 4; i++)
+               set_bit(PG_arch_1, &page[i].flags);
+       page = virt_to_page(&invalid_pg_dir);
+       for (i = 0; i < 4; i++)
+               set_bit(PG_arch_1, &page[i].flags);
 
        /* Set all kernel pages not used for page tables to stable/no-dat */
        for_each_mem_pfn_range(i, MAX_NUMNODES, &start, &end, NULL) {