Merge branch 'page_pool-allow-caching-from-safely-localized-napi'
Jakub Kicinski says:
====================
page_pool: allow caching from safely localized NAPI
I went back to the explicit "are we in NAPI method", mostly
because I don't like having both around :( (even tho I maintain
that in_softirq() && !in_hardirq() is as safe, as softirqs do
not nest).
Still returning the skbs to a CPU, tho, not to the NAPI instance.
I reckon we could create a small refcounted struct per NAPI instance
which would allow sockets and other users so hold a persisent
and safe reference. But that's a bigger change, and I get 90+%
recycling thru the cache with just these patches (for RR and
streaming tests with 100% CPU use it's almost 100%).
Some numbers for streaming test with 100% CPU use (from previous version,
but really they perform the same):
HW-GRO page=page
before after before after
recycle:
cached: 0
138669686 0
150197505
cache_full: 0 223391 0 74582
ring:
138551933 9997191 149299454 0
ring_full: 0 488 3154 127590
released_refcnt: 0 0 0 0
alloc:
fast:
136491361 148615710 146969587 150322859
slow: 1772 1799 144 105
slow_high_order: 0 0 0 0
empty: 1772 1799 144 105
refill:
2165245 156302
2332880 2128
waive: 0 0 0 0
v1: https://lore.kernel.org/all/
20230411201800.596103-1-kuba@kernel.org/
rfcv2: https://lore.kernel.org/all/
20230405232100.103392-1-kuba@kernel.org/
====================
Link: https://lore.kernel.org/r/20230413042605.895677-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>