zink: add funcs for descriptor_surface refs

this abstracts more code for the cache backend

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: move shader image descriptor set refs to underlying type

this was kinda useless since it meant that the set was invalidated any
time the shader image was unbound

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: update null sampler/image descriptor surface with is_buffer during hashing

this is the only time it might need to be used, so it can be updated naturally here

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: add is_buffer flag to union zink_descriptor_surface

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: run lazy batch descriptor functions in cache mode

Acked-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: unify cached descriptor update code

this is all pretty much standardized now, so it can be combined into something
simpler

Acked-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: move ubo range assert to update_descriptor_state()

Acked-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: remove sorting for dynamic ubo offset updating

if the offsets are assigned in the right order, they don't need to be
ordered later

Acked-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: modernize cached image descriptor updating

use the auto-updated descriptor info

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: modernize cached ssbo descriptor updating

use the already-updated bufferinfo structs

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: modernize cached ubo descriptor updating

use the already-updated bufferinfo structs

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: modernize cached push ubo descriptor updating

use the already-updated bufferinfo structs

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: match lazy descriptor set layout in cache mode

now cached descriptors also use a push set for ubo0, also reusing the
ZINK_DESCRIPTOR_TYPES for this set for enum purposes

Acked-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: split lazy sets based on descriptor type

this is now closer to the cached descriptor set layout, but with
the push set as the zero-indexed set (passed as ZINK_DESCRIPTOR_TYPES
for enum purposes)

Acked-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

zink: add a function for creating descriptor layouts for push sets

push sets contain a single ubo at index 0 per shader stage in the set

Reviewed-by: Hoe Hao Cheng <haochengho12907@gmail.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11188>

v3d: ci: Add KHR-GLES31.core.shader_image_load_store.basic-glsl-earlyFragTests to flakes

Closes: #4934
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11386>

venus: add debug info for experimental features during init

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11387>

anv: Agressively no-op Flush/InvalidateMappedMemoryRanges

This has two steps.  First, for each range we look at the memory object
and see if it actually needs flushing before we start throwing CLFLUSH
instructions.  Second, we look at the whole list of types on device
initialization and decide whether or not we need CLFLUSH at all.  The
first part should speed up atom chips a bit since we're currently
CLFLUSHing everything even when we don't need to.  The second isn't
needed on most of today's parts because we base it on !has_llc but it is
needed for discrete parts.  It's also over-all cleaner.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Tested-by: Felix DeGrood <felix.j.degrood@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11364>

freedreno: Be more strict about QUERY_AVAILABLE to simplify the code.

ARB_oq doesn't just say "polling in a loop will make it complete
eventually", it says "querying will make it complete in finite time."

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11368>

freedreno: Drop a bit of indirection around the batch cache flush path.

Checking the flag to call separate functions which then call the same
helper with the flag is a bit silly.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11368>

freedreno: Make a bunch of the batch cache take ctx as the arg.

As we move the batch cache from screen to context, it cleans up the diff a
ton to separate this mechanical change out, and makes the API generally
more consistent (since you usually have to pass the ctx anyway).

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11368>

freedreno: Drop batch-cache orphan tracking.

Let's let valgrind/asan handle leak detection.  This means you can use the
flag even on non-DEBUG builds.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11368>

pan/bi: Fuse abs into FCMP/FMIN/FMAX.v2f16

Needs to be aware of an encoding restriction here.

No shader-db changes.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11327>

pan/bi: Schedule FCMP.v2f16 with abs modifier

So many encoding restrictions, because Bifrost.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11327>

pan/bi: Fuse fclamp_pos and fsat_signed

Mostly interesting now that we fuse clamps.

total instructions in shared programs: 149562 -> 148927 (-0.42%)
instructions in affected programs: 37195 -> 36560 (-1.71%)
helped: 233
HURT: 0
helped stats (abs) min: 1 max: 10 x̄: 2.73 x̃: 2
helped stats (rel) min: 0.22% max: 17.39% x̄: 2.19% x̃: 1.72%
95% mean confidence interval for instructions value: -2.97 -2.48
95% mean confidence interval for instructions %-change: -2.44% -1.94%
Instructions are helped.

total tuples in shared programs: 130487 -> 130122 (-0.28%)
tuples in affected programs: 25693 -> 25328 (-1.42%)
helped: 179
HURT: 12
helped stats (abs) min: 1 max: 10 x̄: 2.11 x̃: 2
helped stats (rel) min: 0.26% max: 25.00% x̄: 2.52% x̃: 1.70%
HURT stats (abs)   min: 1 max: 1 x̄: 1.00 x̃: 1
HURT stats (rel)   min: 0.61% max: 0.68% x̄: 0.65% x̃: 0.66%
95% mean confidence interval for tuples value: -2.15 -1.68
95% mean confidence interval for tuples %-change: -2.74% -1.91%
Tuples are helped.

total clauses in shared programs: 27827 -> 27761 (-0.24%)
clauses in affected programs: 1407 -> 1341 (-4.69%)
helped: 56
HURT: 0
helped stats (abs) min: 1 max: 4 x̄: 1.18 x̃: 1
helped stats (rel) min: 1.79% max: 14.29% x̄: 6.11% x̃: 5.26%
95% mean confidence interval for clauses value: -1.32 -1.03
95% mean confidence interval for clauses %-change: -7.00% -5.23%
Clauses are helped.

total quadwords in shared programs: 118563 -> 118248 (-0.27%)
quadwords in affected programs: 19859 -> 19544 (-1.59%)
helped: 144
HURT: 16
helped stats (abs) min: 1 max: 10 x̄: 2.30 x̃: 2
helped stats (rel) min: 0.29% max: 16.00% x̄: 2.65% x̃: 2.13%
HURT stats (abs)   min: 1 max: 1 x̄: 1.00 x̃: 1
HURT stats (rel)   min: 0.47% max: 2.00% x̄: 0.80% x̃: 0.76%
95% mean confidence interval for quadwords value: -2.27 -1.66
95% mean confidence interval for quadwords %-change: -2.67% -1.93%
Quadwords are helped.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11327>

pan/bi: Add back custom algebraic opts

Right now just do a trivial one to test the infrastructure. In the next
commit we'll use this for a more interesting optimization that's a bit
painful in BIR but trivial with nir_search.

total instructions in shared programs: 149566 -> 149562 (<.01%)
instructions in affected programs: 502 -> 498 (-0.80%)
helped: 3
HURT: 0
helped stats (abs) min: 1 max: 2 x̄: 1.33 x̃: 1
helped stats (rel) min: 0.38% max: 1.30% x̄: 0.97% x̃: 1.21%

total tuples in shared programs: 130957 -> 130487 (-0.36%)
tuples in affected programs: 54752 -> 54282 (-0.86%)
helped: 303
HURT: 2
helped stats (abs) min: 1 max: 29 x̄: 1.56 x̃: 1
helped stats (rel) min: 0.13% max: 7.14% x̄: 1.08% x̃: 0.92%
HURT stats (abs)   min: 1 max: 2 x̄: 1.50 x̃: 1
HURT stats (rel)   min: 1.89% max: 2.99% x̄: 2.44% x̃: 2.44%
95% mean confidence interval for tuples value: -1.79 -1.30
95% mean confidence interval for tuples %-change: -1.17% -0.95%
Tuples are helped.

total clauses in shared programs: 27877 -> 27827 (-0.18%)
clauses in affected programs: 1556 -> 1506 (-3.21%)
helped: 45
HURT: 0
helped stats (abs) min: 1 max: 2 x̄: 1.11 x̃: 1
helped stats (rel) min: 1.43% max: 9.52% x̄: 3.88% x̃: 3.57%
95% mean confidence interval for clauses value: -1.21 -1.02
95% mean confidence interval for clauses %-change: -4.38% -3.39%
Clauses are helped.

total quadwords in shared programs: 119058 -> 118563 (-0.42%)
quadwords in affected programs: 33777 -> 33282 (-1.47%)
helped: 250
HURT: 2
helped stats (abs) min: 1 max: 29 x̄: 1.99 x̃: 1
helped stats (rel) min: 0.23% max: 11.11% x̄: 1.67% x̃: 1.40%
HURT stats (abs)   min: 1 max: 1 x̄: 1.00 x̃: 1
HURT stats (rel)   min: 1.64% max: 2.00% x̄: 1.82% x̃: 1.82%
95% mean confidence interval for quadwords value: -2.27 -1.66
95% mean confidence interval for quadwords %-change: -1.80% -1.49%
Quadwords are helped.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11327>

pan/bi: Propagate fabs/neg/sat

Initial support for modifier propagation. Bifrost makes this
unreasonably hard.

total instructions in shared programs: 151604 -> 150761 (-0.56%)
instructions in affected programs: 48773 -> 47930 (-1.73%)
helped: 212
HURT: 0
helped stats (abs) min: 1 max: 28 x̄: 3.98 x̃: 1
helped stats (rel) min: 0.29% max: 12.70% x̄: 1.75% x̃: 1.26%
95% mean confidence interval for instructions value: -4.71 -3.25
95% mean confidence interval for instructions %-change: -1.97% -1.53%
Instructions are helped.

total tuples in shared programs: 131876 -> 131560 (-0.24%)
tuples in affected programs: 25393 -> 25077 (-1.24%)
helped: 104
HURT: 3
helped stats (abs) min: 1 max: 28 x̄: 3.08 x̃: 2
helped stats (rel) min: 0.34% max: 8.57% x̄: 1.55% x̃: 1.04%
HURT stats (abs)   min: 1 max: 2 x̄: 1.33 x̃: 1
HURT stats (rel)   min: 0.51% max: 2.86% x̄: 1.30% x̃: 0.53%
95% mean confidence interval for tuples value: -3.63 -2.28
95% mean confidence interval for tuples %-change: -1.73% -1.21%
Tuples are helped.

total clauses in shared programs: 28122 -> 28032 (-0.32%)
clauses in affected programs: 2720 -> 2630 (-3.31%)
helped: 58
HURT: 1
helped stats (abs) min: 1 max: 6 x̄: 1.57 x̃: 1
helped stats (rel) min: 0.88% max: 14.29% x̄: 4.06% x̃: 3.67%
HURT stats (abs)   min: 1 max: 1 x̄: 1.00 x̃: 1
HURT stats (rel)   min: 7.69% max: 7.69% x̄: 7.69% x̃: 7.69%
95% mean confidence interval for clauses value: -1.85 -1.20
95% mean confidence interval for clauses %-change: -4.60% -3.13%
Clauses are helped.

total quadwords in shared programs: 119778 -> 119509 (-0.22%)
quadwords in affected programs: 20698 -> 20429 (-1.30%)
helped: 95
HURT: 1
helped stats (abs) min: 1 max: 28 x̄: 2.85 x̃: 2
helped stats (rel) min: 0.38% max: 7.14% x̄: 1.50% x̃: 1.13%
HURT stats (abs)   min: 2 max: 2 x̄: 2.00 x̃: 2
HURT stats (rel)   min: 3.23% max: 3.23% x̄: 3.23% x̃: 3.23%
95% mean confidence interval for quadwords value: -3.49 -2.11
95% mean confidence interval for quadwords %-change: -1.71% -1.20%
Quadwords are helped.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11327>

pan/bi: Report tuples, not nops, in shader-db

More useful in practice, I find.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11327>

pan/bi: Handle fsat_signed and fclamp_pos

Translates to Mali-specific clamps.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11327>

pan/bi: Track instruction size in opcode table

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11327>

pan/bi: Move typesize to common code

Useful for the opcode table.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11327>

pan/bi: Move bi_word_node to common code

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11327>

pan/bi: Include modifier info in opcode table

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11327>

util: Consider CPU affinity when detecting number of CPUs

A similar path can be used on at least FreeBSD using cpuset_getaffinity.
This is how Ninja determines the number of available CPUs on that
platform.  See the GetProcessorCount function in util.cc:

https://github.com/ninja-build/ninja/blob/master/src/util.cc

v2: Fix counting the number of available CPUs.  The CPU_COUNT API does
not work the way I thought it did. :face_palm: Noticed by Marek.

Reviewed-by: Adam Jackson <ajax@redhat.com> [v1]
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> [v1]
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11228>

util: Use maximum number of CPUs for determining cache topology

This prevents problems when some CPUs are offline.  In a four CPU
system, if CPUs 1 and 2 are offline, the cache topology code would
only examine CPUs 0 and 1... giving incorrect information.

The types are changed to int16_t so that the offset of num_L3_caches
does not change.  This triggered a STATIC_ASSERT failure:

STATIC_ASSERT(offsetof(struct util_cpu_caps_t, num_L3_caches) == 5 * sizeof(uint32_t));

I'm assuming there's some assembly code or something that depends on
this offset, and I don't feel like messing with it.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11228>

util: Set util_cpu_caps.num_cpu_mask_bits based on total CPUs in the system

In the current code, this prevents a very unlikely corner case.  More
importantly, it should prevent the next commit from breaking the
universe.

Imagine a system with 64 CPUs configured, but first 32 CPUs are offline.
_SC_NPROCESSORS_CONF will return 32.  All of the surrounding code will
interpret this as meaning CPUs 0 through 31, but all of those CPUs are
offline.  Nothing good can happen then.

The problem cases require systems with more than 32 CPUs because
util_cpu_caps.num_cpu_mask_bits is always rounded up to a multiple of
32.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11228>

util: Fix setting nr_cpus on some BSD variants

Linux, FreeBSD, and DragonFly should have _SC_NOPROCESSORS_ONLN.  NetBSD
and OpenBSD should have HW_NCPUONLINE.  This is what FFmpeg uses on
those platforms.

FreeBSD sysconf(3) manual page:

https://www.freebsd.org/cgi/man.cgi?query=sysconf&sektion=3&apropos=0&manpath=freebsd

The FFmpeg patch is at:

https://patchwork.ffmpeg.org/project/ffmpeg/patch/YGi4sJx3trG3Yn7c@humpty.home.comstyle.com/

OpenBSD sysctl(2) manual page:

https://man.openbsd.org/sysctl.2

Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11228>

util: Trivial cleanup in the BSD code of util_cpu_detect_once

This code is going to be replicated in future commits, so tidy up a bit
first.

Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11228>

util: Change order of PIPE_OS_UNIX code in util_cpu_detect_once

This makes it easier to add preferred vs. fallback paths later.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11228>

util: Zero out all of mask in util_set_thread_affinity

memset operates in bytes, and there are 8-bits in a byte.  This is a
very easy to miss typo. :(

Fixes: 9758b1d416a1 ("util: add util_set_thread_affinity helpers including Windows support")

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11228>

radeonsi: Use util_cpu_caps to detect number of CPUs

The next patch will change the way the number of CPUs is detected, and
this change prevents some code duplication.

v2: Add missing #include.  Noticed by Pierre-Eric... not sure how I
missed it. :(

Reviewed-by: Adam Jackson <ajax@redhat.com> [v1]
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11228>

venus: moves GPU rendering off CPU timeline for Android WSI

When globalFencing is supported, we can export a native sync fd for
presentation to move rendering off CPU timeline.

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11342>

venus: refactor vn_AcquireImageANDROID with globalFencing

This patch refactors to use vn_Import*FdKHR for Android WSI native sync
fence import when globalFencing is supported. Currently there's no perf
win from this, but will move the composer release fence waiting to the
GPU device side automatically when the entire Venus fencing support is
improved.

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11342>

venus: silence a build warning

Address -Wsometimes-uninitialized

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11342>

aux/indices: include provoking vertex check in prim type conversion

Include provoking vertex checking in u_index_prim_type_convert() to make sure
the right primitive type is returned from the generator function.

Fixes google earth running on VMware svga device on hw version 11.

Fixes: cad2026b72cf ("aux/indices: break out primitive type conversion to separate function")

Reviewed-by: Charmaine Lee <charmainel@vmware.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11367>

freedreno: Fallback to sw for copy_image with compressed

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11371>

freedreno: Flush batches on shadow/uncompress

Normally when we shadow a resource (whether it is changing the modifier
or not) we do not need to flush existing batches, since they reference
the original version of the resource.  There is a special case for
resources that are referenced by a batches framebuffer state, because
this state is emitted when the batch is flushed.  Because of this, we
need those batches to be flushed before we shadow the resource.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11371>

freedreno/a6xx: Also validate format in blitter path

Since we can be blitting using a format that is different from the
resource's native format, we also need to validate and demote if
necessary, similar to sampler-views, image-views, and fb state.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11371>

freedreno/a6xx: Handle R8G8 sharp edges in validate_format()

Because R8G8 has a different layout from R16, we not only need to demote
to uncompressed to (for example) sample R8G8 as R16 (or visa versa) but
we also need to demote further to linear.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11371>

freedreno: Avoid recursive re-entry of u_blitter

Normally when demotion to uncompressed is required, it is handled when
various state is attached (sampler-view, framebuffer, etc).  But in this
one path into u_blitter we need to handle it up front, to avoid
recursing back into u_blitter for a decompress blit.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11371>

freedreno: Add debugging for blitter fallback recursion

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11371>

freedreno: Don't try staging blit for non-renderable formats

If a format is not supported as a render target, there is no point in
trying a staging blit, as it will end up in a CPU copy fallback.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11371>

freedreno: Drop obsolete comment

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11371>

venus: be verbose about which physical devices are skipped

Signed-off-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11369>

venus: fix compatibility with older host drivers

Unlike on Android, KHR_swapchain may be advertised even when the host
driver does not support EXT_image_drm_format_modifier and
EXT_queue_family_foreign.  Do not request those extensions when they are
not supported.

Signed-off-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11369>

venus: clean up vn_physical_device_get_native_extensions

Separate Android and non-Android paths.  No real change.

Signed-off-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11369>

nir/cse: resize the instruction set

ministat (CSE only):
Difference at 95.0% confidence
-3357.54 +/- 32.5177
-25.267% +/- 0.24098%
(Student's t, pooled s = 33.909)

ministat (entire run):
Difference at 95.0% confidence
-3414.27 +/- 270.628
-2.76477% +/- 0.217647%
(Student's t, pooled s = 282.207)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6390>

nir: use a single set during CSE

Use a single set and ensure dominance by checking after a equivalent
instruction is found.

Besides removing the need to copy a set, this also lets us resize the set
at the start of the pass in the next commit.

ministat (CSE only):
Difference at 95.0% confidence
-984.956 +/- 28.8559
-6.90075% +/- 0.190231%
(Student's t, pooled s = 26.9052)

ministat (entire run):
Difference at 95.0% confidence
-1246.1 +/- 257.253
-0.998972% +/- 0.205094%
(Student's t, pooled s = 239.863)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Co-authored-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6390>

freedreno/registers: define REG_DSI_CPHY_MODE_CTRL

For use by the kernel driver.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11381>

iris: fix error message on I915_GEM_[GS]ET_TILING failure

The first problem was noticed by Coverity (CID 1485970).

Fixes: ab49063f449 ("iris: Only use SET/GET_TILING when exporting/importing BOs")
Signed-off-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11357>

zink: use correct type for u_bit_scan

Since we're going to iterate over each bit of this variable using
u_bit_scan, we should make sure we use an unsigned int for the variable.

While this works on GCC, it's not guaranteed to work, and produces a
warning on MSVC.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11358>

anv: Replace DC Flush with HDC Pipeline Flush

HDC Pipeline Flush is the correct method for flushing HDC
pipeline on Gfx12+ HW. Continue using DC Flush for earlier HW.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9834>

anv: Add ANV_PIPE_HDC_PIPELINE_FLUSH_BIT

Gfx12+ PIPE_CONTROL bit for flushing HDC cache and memory
transactions to L3 cache.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9834>

anv: Only flush Tile Cache on VK_ACCESS_HOST_R/W

Tile Cache flush flushes all Color/Depth values from L3 cache
to memory in Unified Cache mode. This is only required when
CPU access is required.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9834>

anv: remove unnecessary Tile Cache flushes

On Gfx12+, flushing tile cache ensures color/depth values are
globally visible, but that's expensive.  Most operations only
need values to be GT-visible which can be achieved with depth
or rt flush.  Remove a bunch of unnecessary Tile Cache flushes.
Fast clears and slow depth clears still require Tile Cache flush.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9834>

anv: Remove Tile Cache flush from SBA, Pipe Select

Tile Cache flushing not required for State Base Address or
Pipe Select instructions.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9834>

anv: Clear all pending stall after pipe flush

Was only clearing CS stalls after emitting pending pipe
controls.  Need to clear all stalls.

Cc: mesa-stable
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9834>

anv: Add debug messages for DEBUG_PIPE_CONTROL

Enable with INTEL_DEBUG=pc.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9834>

anv: Cache VB/IB in L3$ for Gfx12

Gfx12 enables caching of Vertex and Index Buffers in L3.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9834>

zink: standardize zero-init code style

quick grepping shows that the amount of {0} (there were 8) is higher
than { 0 } (only 5), so {0} it is!

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11376>

zink: zero-init structs with ISO C

zero-initing with empty braces is a GNU extension, MSVC does not like
it.

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11376>

ci/lava: Generate YAML from Python, not Jinja

This makes it much easier to use the common init scripts ... which we
also do here.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Acked-by: Martin Peres <martin.peres@mupuf.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>

ci/lava: Make kernel image type a normal argument

Just pass the actual parameter, rather than bare YAML.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Acked-by: Martin Peres <martin.peres@mupuf.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>

ci/lava: Drop bitrotten fastboot support

We don't have any fastboot devices in LAVA, and even if we did, the old
overlay path no longer applies.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Acked-by: Martin Peres <martin.peres@mupuf.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>

ci/lava: Use common stage-2 init

Now that our job-execution scripts look very similar for LAVA and
bare-metal, we can just tell LAVA to use the same stage-2 init we use
for bare-metal and delete a bunch of duplication.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Acked-by: Martin Peres <martin.peres@mupuf.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>

ci/lava: Pass MinIO path on the command line

This brings us much closer with what bare-metal does, and also allows us
to upload job data to a local instance rather than the primary fd.o one.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Acked-by: Martin Peres <martin.peres@mupuf.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>

ci/bare-metal: Set CPU and GPU governors to max, disable GPU runtime PM

Give us a bit more predictable performance by making sure we always run
at full tilt.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Acked-by: Martin Peres <martin.peres@mupuf.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>

ci: Unify {BARE_METAL,LAVA}_TEST_SCRIPT environment

Should also probably never have been different.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Acked-by: Martin Peres <martin.peres@mupuf.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>

ci: Unify {BM,LAVA}_START_XORG environment

Why were they ever different ... ?

Signed-off-by: Daniel Stone <daniels@collabora.com>
Acked-by: Martin Peres <martin.peres@mupuf.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>

ci: Consistent pass/fail result output

One less point of differentiation.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Acked-by: Martin Peres <martin.peres@mupuf.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>

ci/lava: Start using devcoredump captures

No reason not to.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Acked-by: Martin Peres <martin.peres@mupuf.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>

ci/bare-metal: Move devcoredump capture to CI common

Reusing this for LAVA sounds like a good idea!

Signed-off-by: Daniel Stone <daniels@collabora.com>
Acked-by: Martin Peres <martin.peres@mupuf.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>

ci/bare-metal: Split init script into two stages

Whilst we want to reuse the same init and job environment for LAVA and
bare-metal, LAVA needs to additionally inject wget and tar jobs, so we
can actually get our per-job environment, as the rootfs we run in is
just the container-generated base rootfs.

Split the init script into two stages, with the first stage doing very
base bringup of devices and networking, and the second stage setting the
job environment and running the jobs.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Acked-by: Martin Peres <martin.peres@mupuf.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>

ci/bare-metal: Consistently set library paths

Everything needs them, so might as well set it up front.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Acked-by: Martin Peres <martin.peres@mupuf.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>

ci: Be consistent about install path

Make both LAVA and bare-metal untar into $CI_PROJECT_DIR/install/, and
symlink /install/ to it during init.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Acked-by: Martin Peres <martin.peres@mupuf.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>

ci: Move bare-metal init script to common directory

Signed-off-by: Daniel Stone <daniels@collabora.com>
Acked-by: Martin Peres <martin.peres@mupuf.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>

ci/bare-metal: Reorder init so network comes first

Make sure that everything we need to ensure network access comes first,
so we can reuse this in LAVA which needs the network to pull the
per-pipeline build and the per-job environment overlays.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Acked-by: Martin Peres <martin.peres@mupuf.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>

ci/bare-metal: Try harder to do NTP

Same as LAVA does.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Acked-by: Martin Peres <martin.peres@mupuf.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>

ci/lava: Rename environment variable script

Make it line up with bare-metal.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Acked-by: Martin Peres <martin.peres@mupuf.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>

ci/lava: Use HWCI_KERNEL_MODULES to load modules

One fewer difference to bare-metal.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Acked-by: Martin Peres <martin.peres@mupuf.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>

ci/bare-metal: Add parens around shell command

Play safe and make sure we don't get bit by priority rules between the
|| and | operators.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Suggested-by: Martin Peres <martin.peres@mupuf.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>

ci/bare-metal: Rename BM_KERNEL_MODULES to HWCI_KERNEL_MODULES

To try to make init more common.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Acked-by: Martin Peres <martin.peres@mupuf.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>

ci/lava: Explicitly start Xorg for Iris EGL tests

These tests relies on Xorg being started, so let's explicitly declare
that.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Acked-by: Martin Peres <martin.peres@mupuf.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>

ci/lava: Set PIGLIT_NO_WINDOW

This got lost in the move away from hardcoded environment variables, and
fixes the Iris EGL tests.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Acked-by: Martin Peres <martin.peres@mupuf.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>

ci/lava: Always upload Piglit replay images to MinIO

This should probably be set in the trace-job environments, but the
inheritance is a bit of a mess between all the systems at the moment,
and this matches previous hardcoded behaviour at least.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Acked-by: Martin Peres <martin.peres@mupuf.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>

ci/piglit: Fix path to uploaded images

Missed this bit when I was reworking environment variables.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Reported-by: Emma Anholt <emma@anholt.net>
Ref: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11329#note_956187
Acked-by: Martin Peres <martin.peres@mupuf.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>

radeonsi: use si_nir_is_output_const_if_tex_is_const

When a blending mode producing "color = src * dst" is used and we
can determine that dst is 1, then the draw call can dropped completely.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10979>

radeonsi/nir: add si_nir_is_output_const_if_tex_is_const

Determine if a given shader write the same constant value to its output
if a specific input texture is replaced by constant load.

It's done by checking if the store_output intrinsics only depends on
constant and a texture. If it's true, the given texture is replaced by
a constant load in cloned shader and this clone is optimized.

Then the output is checked (= is it constant or not).

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10979>

radeonsi: use si_install_draw_wrapper for tmz handling

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10979>

radeonsi: add si_install_draw_wrapper

This allows to implement custom draw_vbo code-path without
touching si_draw_vbo.

As an example, skipped all draw calls with an odd new_draws
could be done like this:

   void mywrapper(...) {
       if (new_draws % 2)
          return;
       return sctx->real_draw_vbo(...);
   }

   if (some_condition_is_met)
      si_install_draw_wrapper(sctx, mywrapper);

Instead of having to add the "if ()" condition inside si_draw_vbo.

Note that a single wrapper may be installed so care must be taken
to not override an existing wrapper.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10979>