x86/mm: Use INVPCID for __native_flush_tlb_single()
authorDave Hansen <dave.hansen@linux.intel.com>
Mon, 4 Dec 2017 14:08:01 +0000 (15:08 +0100)
committerGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Tue, 2 Jan 2018 19:31:01 +0000 (20:31 +0100)
commitc5548af97ae98f9d9e6ae5a9a005e605bd3c06b5
treee3eef3c5b701bfd8ef8fa05e6cb0c7cc8cbaab39
parent36a72ab52c8d969a7a302082f52731c1be0e9ada
x86/mm: Use INVPCID for __native_flush_tlb_single()

commit 6cff64b86aaaa07f89f50498055a20e45754b0c1 upstream.

This uses INVPCID to shoot down individual lines of the user mapping
instead of marking the entire user map as invalid. This
could/might/possibly be faster.

This for sure needs tlb_single_page_flush_ceiling to be redetermined;
esp. since INVPCID is _slow_.

A detailed performance analysis is available here:

  https://lkml.kernel.org/r/3062e486-3539-8a1f-5724-16199420be71@intel.com

[ Peterz: Split out from big combo patch ]

Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: Eduardo Valentin <eduval@amazon.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Juergen Gross <jgross@suse.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Will Deacon <will.deacon@arm.com>
Cc: aliguori@amazon.com
Cc: daniel.gruss@iaik.tugraz.at
Cc: hughd@google.com
Cc: keescook@google.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
arch/x86/include/asm/cpufeatures.h
arch/x86/include/asm/tlbflush.h
arch/x86/mm/init.c