mm, slub: consider rest of partial list if acquire_slab() fails
authorJann Horn <jannh@google.com>
Tue, 12 Jan 2021 23:49:04 +0000 (15:49 -0800)
committerGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Tue, 19 Jan 2021 17:27:32 +0000 (18:27 +0100)
commitc8c01da728ef65ad20d26a338819639a2ee25a13
tree13252ec1280d2f2dbbf37c0574e2a64588f4928e
parentecca0c675bdecebdeb2f2eb76fb33520c441dacf
mm, slub: consider rest of partial list if acquire_slab() fails

commit 8ff60eb052eeba95cfb3efe16b08c9199f8121cf upstream.

acquire_slab() fails if there is contention on the freelist of the page
(probably because some other CPU is concurrently freeing an object from
the page).  In that case, it might make sense to look for a different page
(since there might be more remote frees to the page from other CPUs, and
we don't want contention on struct page).

However, the current code accidentally stops looking at the partial list
completely in that case.  Especially on kernels without CONFIG_NUMA set,
this means that get_partial() fails and new_slab_objects() falls back to
new_slab(), allocating new pages.  This could lead to an unnecessary
increase in memory fragmentation.

Link: https://lkml.kernel.org/r/20201228130853.1871516-1-jannh@google.com
Fixes: 7ced37197196 ("slub: Acquire_slab() avoid loop")
Signed-off-by: Jann Horn <jannh@google.com>
Acked-by: David Rientjes <rientjes@google.com>
Acked-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Pekka Enberg <penberg@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
mm/slub.c