KVM: x86/mmu: Track tail count in pte_list_desc to optimize guest fork()
authorLai Jiangshan <jiangshan.ljs@antgroup.com>
Fri, 13 Jan 2023 12:29:10 +0000 (20:29 +0800)
committerSean Christopherson <seanjc@google.com>
Fri, 17 Mar 2023 02:07:37 +0000 (19:07 -0700)
commit141705b78381ab1dcb52c84ebb396e434e86b624
tree14efa91e9e6e25bee23736dd22aaed8ad2b783da
parent19ace7d6ca15a4395dd294286fe253a233bbf20a
KVM: x86/mmu: Track tail count in pte_list_desc to optimize guest fork()

Rework "struct pte_list_desc" and pte_list_{add|remove} to track the tail
count, i.e. number of PTEs in non-head descriptors, and to always keep all
tail descriptors full so that adding a new entry and counting the number
of entries is done in constant time instead of linear time.

No visible performace is changed in tests.  But pte_list_add() is no longer
shown in the perf result for the COWed pages even the guest forks millions
of tasks.

Signed-off-by: Lai Jiangshan <jiangshan.ljs@antgroup.com>
Link: https://lore.kernel.org/r/20230113122910.672417-1-jiangshanlai@gmail.com
[sean: reword shortlog, tweak changelog, add lots of comments, add BUG_ON()]
Signed-off-by: Sean Christopherson <seanjc@google.com>
arch/x86/kvm/mmu/mmu.c