zink: move batch decl to top of draw_vbo

this may still change during descriptor updating, but having the pointer
earlier is helpful

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11408>

zink: remove unnecessary draw checks

I don't know how/why these got here but they aren't needed

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11408>

zink: use u_live_shader_cache

let the duplicated shader madness end

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11403>

zink: unify gfx shader create callbacks

these are identical

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11403>

zink: mark bind_stage() as inline

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11403>

zink: simplify zink_program_has_descriptors()

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11407>

zink: remove return types from program update functions during draw

also remove some null checks; these add overhead and the app is going to
explode anyway if they fail

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11406>

zink: slightly refactor program updating during draw

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11406>

zink: destroy lazy descriptor pools during batch reset when unused

this should keep resource usage a bit lower

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11404>

zink: track active use counts for descriptor layouts

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11404>

zink: cache descriptor update templates along with layout

this simplifies a bunch of code and just makes more sense given that the
lifetimes should match

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11404>

crocus: disable Z16

This is disabled on i965 for all pre-gen8, so do the same here.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11409>

crocus: fixup render aux usage function.

This got misaligned somewhere in development, should fix the
regressions vs i965 in tesseract as seen on phoronix benchmarks.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11409>

zink: make batch_usage_matches take a batch state param

no functional changes, just some unwinding in some cases

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11399>

zink: make batch_usage_set take a batch state param

no functional changes

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11399>

zink: remove unnecessary conditionals in resource batch tracking

this is no longer an atomic op

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11399>

zink: unset program batch usage on state reset

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11399>

zink: make batch_usage_unset take a batch state param

no functional changes

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11399>

zink: remove atomic from batch usage setting

this shouldn't be necessary since usage can only be set from the context thread
and only needs to be accessed atomically for the cmpxchg when unsetting

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11399>

zink: move batch usage functions to static inlines

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11399>

zink: optimize zink_tc_fence struct packing

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11400>

zink: reorder has_barriers flag in batch state struct

better struct packing

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11400>

zink: split batch state work_count into separate vars

this has better struct packing

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11400>

zink: remove zink_batch_state::descs_used

this is no longer used or needed

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11400>

crocus: fixed some missing WM dirtys.

This fixes misrendering in ET: legacy.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11390>

zink: add ZINK_DESCRIPTORS env var to explicitly set a mode

currently this supports 3 modes, with the default being a hybrid between
caching and lazy

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: unblock last_set cached descriptor reuse when safe to do so

if no changes at all have occurred to a given set since the last use,
and if the program hasn't changed, then there is no possiblility for
the set to have been invalidated, and so it can immediately be reused

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: skip hash updates for descriptor types which aren't used

this is pointless and hurts drawoverhead perf

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: add oob asserts for descriptor set ref setting

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: enable templated descriptor updates in cache mode

this leverages the template infrastructure from the lazy manager for the
cached mode, alternatively generating a pseudo-template for updates
if real templates aren't available in order to retain 1.0 compatibility

Acked-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: move samplerview descset refs to base objects

this further extends the lifetimes for sets

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: add funcs for descriptor_surface refs

this abstracts more code for the cache backend

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: move shader image descriptor set refs to underlying type

this was kinda useless since it meant that the set was invalidated any
time the shader image was unbound

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: update null sampler/image descriptor surface with is_buffer during hashing

this is the only time it might need to be used, so it can be updated naturally here

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: add is_buffer flag to union zink_descriptor_surface

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: run lazy batch descriptor functions in cache mode

Acked-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: unify cached descriptor update code

this is all pretty much standardized now, so it can be combined into something
simpler

Acked-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: move ubo range assert to update_descriptor_state()

Acked-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: remove sorting for dynamic ubo offset updating

if the offsets are assigned in the right order, they don't need to be
ordered later

Acked-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: modernize cached image descriptor updating

use the auto-updated descriptor info

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: modernize cached ssbo descriptor updating

use the already-updated bufferinfo structs

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: modernize cached ubo descriptor updating

use the already-updated bufferinfo structs

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: modernize cached push ubo descriptor updating

use the already-updated bufferinfo structs

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: match lazy descriptor set layout in cache mode

now cached descriptors also use a push set for ubo0, also reusing the
ZINK_DESCRIPTOR_TYPES for this set for enum purposes

Acked-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: split lazy sets based on descriptor type

this is now closer to the cached descriptor set layout, but with
the push set as the zero-indexed set (passed as ZINK_DESCRIPTOR_TYPES
for enum purposes)

Acked-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: add a function for creating descriptor layouts for push sets

push sets contain a single ubo at index 0 per shader stage in the set

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

v3d: ci: Add KHR-GLES31.core.shader_image_load_store.basic-glsl-earlyFragTests to flakes

Closes: #4934
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11386>

venus: add debug info for experimental features during init

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11387>

anv: Agressively no-op Flush/InvalidateMappedMemoryRanges

This has two steps.  First, for each range we look at the memory object
and see if it actually needs flushing before we start throwing CLFLUSH
instructions.  Second, we look at the whole list of types on device
initialization and decide whether or not we need CLFLUSH at all.  The
first part should speed up atom chips a bit since we're currently
CLFLUSHing everything even when we don't need to.  The second isn't
needed on most of today's parts because we base it on !has_llc but it is
needed for discrete parts.  It's also over-all cleaner.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Tested-by: Felix DeGrood <felix.j.degrood@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11364>

freedreno: Be more strict about QUERY_AVAILABLE to simplify the code.

ARB_oq doesn't just say "polling in a loop will make it complete
eventually", it says "querying will make it complete in finite time."

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11368>

freedreno: Drop a bit of indirection around the batch cache flush path.

Checking the flag to call separate functions which then call the same
helper with the flag is a bit silly.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11368>

freedreno: Make a bunch of the batch cache take ctx as the arg.

As we move the batch cache from screen to context, it cleans up the diff a
ton to separate this mechanical change out, and makes the API generally
more consistent (since you usually have to pass the ctx anyway).

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11368>

freedreno: Drop batch-cache orphan tracking.

Let's let valgrind/asan handle leak detection.  This means you can use the
flag even on non-DEBUG builds.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11368>

pan/bi: Fuse abs into FCMP/FMIN/FMAX.v2f16

Needs to be aware of an encoding restriction here.

No shader-db changes.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11327>

pan/bi: Schedule FCMP.v2f16 with abs modifier

So many encoding restrictions, because Bifrost.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11327>

pan/bi: Fuse fclamp_pos and fsat_signed

Mostly interesting now that we fuse clamps.

total instructions in shared programs: 149562 -> 148927 (-0.42%)
instructions in affected programs: 37195 -> 36560 (-1.71%)
helped: 233
HURT: 0
helped stats (abs) min: 1 max: 10 x̄: 2.73 x̃: 2
helped stats (rel) min: 0.22% max: 17.39% x̄: 2.19% x̃: 1.72%
95% mean confidence interval for instructions value: -2.97 -2.48
95% mean confidence interval for instructions %-change: -2.44% -1.94%
Instructions are helped.

total tuples in shared programs: 130487 -> 130122 (-0.28%)
tuples in affected programs: 25693 -> 25328 (-1.42%)
helped: 179
HURT: 12
helped stats (abs) min: 1 max: 10 x̄: 2.11 x̃: 2
helped stats (rel) min: 0.26% max: 25.00% x̄: 2.52% x̃: 1.70%
HURT stats (abs)   min: 1 max: 1 x̄: 1.00 x̃: 1
HURT stats (rel)   min: 0.61% max: 0.68% x̄: 0.65% x̃: 0.66%
95% mean confidence interval for tuples value: -2.15 -1.68
95% mean confidence interval for tuples %-change: -2.74% -1.91%
Tuples are helped.

total clauses in shared programs: 27827 -> 27761 (-0.24%)
clauses in affected programs: 1407 -> 1341 (-4.69%)
helped: 56
HURT: 0
helped stats (abs) min: 1 max: 4 x̄: 1.18 x̃: 1
helped stats (rel) min: 1.79% max: 14.29% x̄: 6.11% x̃: 5.26%
95% mean confidence interval for clauses value: -1.32 -1.03
95% mean confidence interval for clauses %-change: -7.00% -5.23%
Clauses are helped.

total quadwords in shared programs: 118563 -> 118248 (-0.27%)
quadwords in affected programs: 19859 -> 19544 (-1.59%)
helped: 144
HURT: 16
helped stats (abs) min: 1 max: 10 x̄: 2.30 x̃: 2
helped stats (rel) min: 0.29% max: 16.00% x̄: 2.65% x̃: 2.13%
HURT stats (abs)   min: 1 max: 1 x̄: 1.00 x̃: 1
HURT stats (rel)   min: 0.47% max: 2.00% x̄: 0.80% x̃: 0.76%
95% mean confidence interval for quadwords value: -2.27 -1.66
95% mean confidence interval for quadwords %-change: -2.67% -1.93%
Quadwords are helped.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11327>

pan/bi: Add back custom algebraic opts

Right now just do a trivial one to test the infrastructure. In the next
commit we'll use this for a more interesting optimization that's a bit
painful in BIR but trivial with nir_search.

total instructions in shared programs: 149566 -> 149562 (<.01%)
instructions in affected programs: 502 -> 498 (-0.80%)
helped: 3
HURT: 0
helped stats (abs) min: 1 max: 2 x̄: 1.33 x̃: 1
helped stats (rel) min: 0.38% max: 1.30% x̄: 0.97% x̃: 1.21%

total tuples in shared programs: 130957 -> 130487 (-0.36%)
tuples in affected programs: 54752 -> 54282 (-0.86%)
helped: 303
HURT: 2
helped stats (abs) min: 1 max: 29 x̄: 1.56 x̃: 1
helped stats (rel) min: 0.13% max: 7.14% x̄: 1.08% x̃: 0.92%
HURT stats (abs)   min: 1 max: 2 x̄: 1.50 x̃: 1
HURT stats (rel)   min: 1.89% max: 2.99% x̄: 2.44% x̃: 2.44%
95% mean confidence interval for tuples value: -1.79 -1.30
95% mean confidence interval for tuples %-change: -1.17% -0.95%
Tuples are helped.

total clauses in shared programs: 27877 -> 27827 (-0.18%)
clauses in affected programs: 1556 -> 1506 (-3.21%)
helped: 45
HURT: 0
helped stats (abs) min: 1 max: 2 x̄: 1.11 x̃: 1
helped stats (rel) min: 1.43% max: 9.52% x̄: 3.88% x̃: 3.57%
95% mean confidence interval for clauses value: -1.21 -1.02
95% mean confidence interval for clauses %-change: -4.38% -3.39%
Clauses are helped.

total quadwords in shared programs: 119058 -> 118563 (-0.42%)
quadwords in affected programs: 33777 -> 33282 (-1.47%)
helped: 250
HURT: 2
helped stats (abs) min: 1 max: 29 x̄: 1.99 x̃: 1
helped stats (rel) min: 0.23% max: 11.11% x̄: 1.67% x̃: 1.40%
HURT stats (abs)   min: 1 max: 1 x̄: 1.00 x̃: 1
HURT stats (rel)   min: 1.64% max: 2.00% x̄: 1.82% x̃: 1.82%
95% mean confidence interval for quadwords value: -2.27 -1.66
95% mean confidence interval for quadwords %-change: -1.80% -1.49%
Quadwords are helped.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11327>

pan/bi: Propagate fabs/neg/sat

Initial support for modifier propagation. Bifrost makes this
unreasonably hard.

total instructions in shared programs: 151604 -> 150761 (-0.56%)
instructions in affected programs: 48773 -> 47930 (-1.73%)
helped: 212
HURT: 0
helped stats (abs) min: 1 max: 28 x̄: 3.98 x̃: 1
helped stats (rel) min: 0.29% max: 12.70% x̄: 1.75% x̃: 1.26%
95% mean confidence interval for instructions value: -4.71 -3.25
95% mean confidence interval for instructions %-change: -1.97% -1.53%
Instructions are helped.

total tuples in shared programs: 131876 -> 131560 (-0.24%)
tuples in affected programs: 25393 -> 25077 (-1.24%)
helped: 104
HURT: 3
helped stats (abs) min: 1 max: 28 x̄: 3.08 x̃: 2
helped stats (rel) min: 0.34% max: 8.57% x̄: 1.55% x̃: 1.04%
HURT stats (abs)   min: 1 max: 2 x̄: 1.33 x̃: 1
HURT stats (rel)   min: 0.51% max: 2.86% x̄: 1.30% x̃: 0.53%
95% mean confidence interval for tuples value: -3.63 -2.28
95% mean confidence interval for tuples %-change: -1.73% -1.21%
Tuples are helped.

total clauses in shared programs: 28122 -> 28032 (-0.32%)
clauses in affected programs: 2720 -> 2630 (-3.31%)
helped: 58
HURT: 1
helped stats (abs) min: 1 max: 6 x̄: 1.57 x̃: 1
helped stats (rel) min: 0.88% max: 14.29% x̄: 4.06% x̃: 3.67%
HURT stats (abs)   min: 1 max: 1 x̄: 1.00 x̃: 1
HURT stats (rel)   min: 7.69% max: 7.69% x̄: 7.69% x̃: 7.69%
95% mean confidence interval for clauses value: -1.85 -1.20
95% mean confidence interval for clauses %-change: -4.60% -3.13%
Clauses are helped.

total quadwords in shared programs: 119778 -> 119509 (-0.22%)
quadwords in affected programs: 20698 -> 20429 (-1.30%)
helped: 95
HURT: 1
helped stats (abs) min: 1 max: 28 x̄: 2.85 x̃: 2
helped stats (rel) min: 0.38% max: 7.14% x̄: 1.50% x̃: 1.13%
HURT stats (abs)   min: 2 max: 2 x̄: 2.00 x̃: 2
HURT stats (rel)   min: 3.23% max: 3.23% x̄: 3.23% x̃: 3.23%
95% mean confidence interval for quadwords value: -3.49 -2.11
95% mean confidence interval for quadwords %-change: -1.71% -1.20%
Quadwords are helped.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11327>

pan/bi: Report tuples, not nops, in shader-db

More useful in practice, I find.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11327>

pan/bi: Handle fsat_signed and fclamp_pos

Translates to Mali-specific clamps.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11327>

pan/bi: Track instruction size in opcode table

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11327>

pan/bi: Move typesize to common code

Useful for the opcode table.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11327>

pan/bi: Move bi_word_node to common code

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11327>

pan/bi: Include modifier info in opcode table

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11327>

util: Consider CPU affinity when detecting number of CPUs

A similar path can be used on at least FreeBSD using cpuset_getaffinity.
This is how Ninja determines the number of available CPUs on that
platform.  See the GetProcessorCount function in util.cc:

https://github.com/ninja-build/ninja/blob/master/src/util.cc

v2: Fix counting the number of available CPUs.  The CPU_COUNT API does
not work the way I thought it did. :face_palm: Noticed by Marek.

Reviewed-by: Adam Jackson <ajax@redhat.com> [v1]
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> [v1]
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11228>

util: Use maximum number of CPUs for determining cache topology

This prevents problems when some CPUs are offline.  In a four CPU
system, if CPUs 1 and 2 are offline, the cache topology code would
only examine CPUs 0 and 1... giving incorrect information.

The types are changed to int16_t so that the offset of num_L3_caches
does not change.  This triggered a STATIC_ASSERT failure:

STATIC_ASSERT(offsetof(struct util_cpu_caps_t, num_L3_caches) == 5 * sizeof(uint32_t));

I'm assuming there's some assembly code or something that depends on
this offset, and I don't feel like messing with it.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11228>

util: Set util_cpu_caps.num_cpu_mask_bits based on total CPUs in the system

In the current code, this prevents a very unlikely corner case.  More
importantly, it should prevent the next commit from breaking the
universe.

Imagine a system with 64 CPUs configured, but first 32 CPUs are offline.
_SC_NPROCESSORS_CONF will return 32.  All of the surrounding code will
interpret this as meaning CPUs 0 through 31, but all of those CPUs are
offline.  Nothing good can happen then.

The problem cases require systems with more than 32 CPUs because
util_cpu_caps.num_cpu_mask_bits is always rounded up to a multiple of
32.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11228>

util: Fix setting nr_cpus on some BSD variants

Linux, FreeBSD, and DragonFly should have _SC_NOPROCESSORS_ONLN.  NetBSD
and OpenBSD should have HW_NCPUONLINE.  This is what FFmpeg uses on
those platforms.

FreeBSD sysconf(3) manual page:

https://www.freebsd.org/cgi/man.cgi?query=sysconf&sektion=3&apropos=0&manpath=freebsd

The FFmpeg patch is at:

https://patchwork.ffmpeg.org/project/ffmpeg/patch/YGi4sJx3trG3Yn7c@humpty.home.comstyle.com/

OpenBSD sysctl(2) manual page:

https://man.openbsd.org/sysctl.2

Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11228>

util: Trivial cleanup in the BSD code of util_cpu_detect_once

This code is going to be replicated in future commits, so tidy up a bit
first.

Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11228>

util: Change order of PIPE_OS_UNIX code in util_cpu_detect_once

This makes it easier to add preferred vs. fallback paths later.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11228>

util: Zero out all of mask in util_set_thread_affinity

memset operates in bytes, and there are 8-bits in a byte.  This is a
very easy to miss typo. :(

Fixes: 9758b1d416a1 ("util: add util_set_thread_affinity helpers including Windows support")

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11228>

radeonsi: Use util_cpu_caps to detect number of CPUs

The next patch will change the way the number of CPUs is detected, and
this change prevents some code duplication.

v2: Add missing #include.  Noticed by Pierre-Eric... not sure how I
missed it. :(

Reviewed-by: Adam Jackson <ajax@redhat.com> [v1]
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11228>

venus: moves GPU rendering off CPU timeline for Android WSI

When globalFencing is supported, we can export a native sync fd for
presentation to move rendering off CPU timeline.

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11342>

venus: refactor vn_AcquireImageANDROID with globalFencing

This patch refactors to use vn_Import*FdKHR for Android WSI native sync
fence import when globalFencing is supported. Currently there's no perf
win from this, but will move the composer release fence waiting to the
GPU device side automatically when the entire Venus fencing support is
improved.

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11342>

venus: silence a build warning

Address -Wsometimes-uninitialized

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11342>

aux/indices: include provoking vertex check in prim type conversion

Include provoking vertex checking in u_index_prim_type_convert() to make sure
the right primitive type is returned from the generator function.

Fixes google earth running on VMware svga device on hw version 11.

Fixes: cad2026b72cf ("aux/indices: break out primitive type conversion to separate function")

Reviewed-by: Charmaine Lee <charmainel@vmware.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11367>

freedreno: Fallback to sw for copy_image with compressed

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11371>

freedreno: Flush batches on shadow/uncompress

Normally when we shadow a resource (whether it is changing the modifier
or not) we do not need to flush existing batches, since they reference
the original version of the resource.  There is a special case for
resources that are referenced by a batches framebuffer state, because
this state is emitted when the batch is flushed.  Because of this, we
need those batches to be flushed before we shadow the resource.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11371>

freedreno/a6xx: Also validate format in blitter path

Since we can be blitting using a format that is different from the
resource's native format, we also need to validate and demote if
necessary, similar to sampler-views, image-views, and fb state.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11371>

freedreno/a6xx: Handle R8G8 sharp edges in validate_format()

Because R8G8 has a different layout from R16, we not only need to demote
to uncompressed to (for example) sample R8G8 as R16 (or visa versa) but
we also need to demote further to linear.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11371>

freedreno: Avoid recursive re-entry of u_blitter

Normally when demotion to uncompressed is required, it is handled when
various state is attached (sampler-view, framebuffer, etc).  But in this
one path into u_blitter we need to handle it up front, to avoid
recursing back into u_blitter for a decompress blit.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11371>

freedreno: Add debugging for blitter fallback recursion

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11371>

freedreno: Don't try staging blit for non-renderable formats

If a format is not supported as a render target, there is no point in
trying a staging blit, as it will end up in a CPU copy fallback.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11371>

freedreno: Drop obsolete comment

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11371>

venus: be verbose about which physical devices are skipped

Signed-off-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11369>

venus: fix compatibility with older host drivers

Unlike on Android, KHR_swapchain may be advertised even when the host
driver does not support EXT_image_drm_format_modifier and
EXT_queue_family_foreign.  Do not request those extensions when they are
not supported.

Signed-off-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11369>

venus: clean up vn_physical_device_get_native_extensions

Separate Android and non-Android paths.  No real change.

Signed-off-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11369>

nir/cse: resize the instruction set

ministat (CSE only):
Difference at 95.0% confidence
-3357.54 +/- 32.5177
-25.267% +/- 0.24098%
(Student's t, pooled s = 33.909)

ministat (entire run):
Difference at 95.0% confidence
-3414.27 +/- 270.628
-2.76477% +/- 0.217647%
(Student's t, pooled s = 282.207)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6390>

nir: use a single set during CSE

Use a single set and ensure dominance by checking after a equivalent
instruction is found.

Besides removing the need to copy a set, this also lets us resize the set
at the start of the pass in the next commit.

ministat (CSE only):
Difference at 95.0% confidence
-984.956 +/- 28.8559
-6.90075% +/- 0.190231%
(Student's t, pooled s = 26.9052)

ministat (entire run):
Difference at 95.0% confidence
-1246.1 +/- 257.253
-0.998972% +/- 0.205094%
(Student's t, pooled s = 239.863)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Co-authored-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6390>

freedreno/registers: define REG_DSI_CPHY_MODE_CTRL

For use by the kernel driver.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11381>

iris: fix error message on I915_GEM_[GS]ET_TILING failure

The first problem was noticed by Coverity (CID 1485970).

Fixes: ab49063f449 ("iris: Only use SET/GET_TILING when exporting/importing BOs")
Signed-off-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11357>

zink: use correct type for u_bit_scan

Since we're going to iterate over each bit of this variable using
u_bit_scan, we should make sure we use an unsigned int for the variable.

While this works on GCC, it's not guaranteed to work, and produces a
warning on MSVC.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11358>

anv: Replace DC Flush with HDC Pipeline Flush

HDC Pipeline Flush is the correct method for flushing HDC
pipeline on Gfx12+ HW. Continue using DC Flush for earlier HW.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9834>

anv: Add ANV_PIPE_HDC_PIPELINE_FLUSH_BIT

Gfx12+ PIPE_CONTROL bit for flushing HDC cache and memory
transactions to L3 cache.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9834>

anv: Only flush Tile Cache on VK_ACCESS_HOST_R/W

Tile Cache flush flushes all Color/Depth values from L3 cache
to memory in Unified Cache mode. This is only required when
CPU access is required.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9834>

anv: remove unnecessary Tile Cache flushes

On Gfx12+, flushing tile cache ensures color/depth values are
globally visible, but that's expensive.  Most operations only
need values to be GT-visible which can be achieved with depth
or rt flush.  Remove a bunch of unnecessary Tile Cache flushes.
Fast clears and slow depth clears still require Tile Cache flush.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9834>

anv: Remove Tile Cache flush from SBA, Pipe Select

Tile Cache flushing not required for State Base Address or
Pipe Select instructions.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9834>

anv: Clear all pending stall after pipe flush

Was only clearing CS stalls after emitting pending pipe
controls.  Need to clear all stalls.

Cc: mesa-stable
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9834>

anv: Add debug messages for DEBUG_PIPE_CONTROL

Enable with INTEL_DEBUG=pc.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9834>

anv: Cache VB/IB in L3$ for Gfx12

Gfx12 enables caching of Vertex and Index Buffers in L3.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9834>