nir_to_tgsi: Add support for HW atomics.

Found missing when taking virgl on the nir-to-tgsi path.  Needs an SSBO
fix as well to pass all of the atomic_counters tests.

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11937>

pan/bi: Remove incorrect errata workaround

This worked around a symptom of the underlying issue worked around in
the previous commit. This workaround is wrong in the sense of failing to
correct some broken code sequences and needlessly rejecting some working
code sequences.

total tuples in shared programs: 123770 -> 123630 (-0.11%)
tuples in affected programs: 9548 -> 9408 (-1.47%)
helped: 133
HURT: 0
helped stats (abs) min: 1.0 max: 2.0 x̄: 1.05 x̃: 1
helped stats (rel) min: 0.42% max: 16.67% x̄: 4.07% x̃: 1.15%
95% mean confidence interval for tuples value: -1.09 -1.01
95% mean confidence interval for tuples %-change: -4.98% -3.17%
Tuples are helped.

total cycles in shared programs: 12114.83 -> 12114.50 (<.01%)
cycles in affected programs: 34.08 -> 33.75 (-0.98%)
helped: 9
HURT: 1
helped stats (abs) min: 0.04166599999999998 max: 0.04166700000000034 x̄: 0.04 x̃: 0
helped stats (rel) min: 0.72% max: 12.50% x̄: 2.99% x̃: 2.04%
HURT stats (abs)   min: 0.04166700000000034 max: 0.04166700000000034 x̄: 0.04 x̃: 0
HURT stats (rel)   min: 0.62% max: 0.62% x̄: 0.62% x̃: 0.62%
95% mean confidence interval for cycles value: -0.05 -0.01
95% mean confidence interval for cycles %-change: -5.27% <.01%
Inconclusive result (%-change mean confidence interval includes 0).

total arith in shared programs: 4603.42 -> 4601.54 (-0.04%)
arith in affected programs: 50.50 -> 48.62 (-3.71%)
helped: 41
HURT: 1
helped stats (abs) min: 0.04166599999999998 max: 0.08333299999999999 x̄: 0.05 x̃: 0
helped stats (rel) min: 0.72% max: 33.33% x̄: 17.23% x̃: 13.33%
HURT stats (abs)   min: 0.04166700000000034 max: 0.04166700000000034 x̄: 0.04 x̃: 0
HURT stats (rel)   min: 0.62% max: 0.62% x̄: 0.62% x̃: 0.62%
95% mean confidence interval for arith value: -0.05 -0.04
95% mean confidence interval for arith %-change: -20.93% -12.69%
Arith are helped.

total quadwords in shared programs: 110116 -> 110009 (-0.10%)
quadwords in affected programs: 7829 -> 7722 (-1.37%)
helped: 106
HURT: 0
helped stats (abs) min: 1.0 max: 2.0 x̄: 1.01 x̃: 1
helped stats (rel) min: 0.49% max: 7.14% x̄: 1.91% x̃: 1.35%
95% mean confidence interval for quadwords value: -1.03 -0.99
95% mean confidence interval for quadwords %-change: -2.23% -1.59%
Quadwords are helped.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12114>

pan/bi: Test restrictions on same-cycle temporaries

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12114>

pan/bi: Restrict swizzles on same cycle temporaries

Hand typed. We could generate this from the XML to avoid the repititon
but I think the cure is worse than the disease.

This fixes instruction encoding faults seen in conformance tests.

Only a single shader-db affected, and it was likely already broken...

quadwords HURT:   shaders/glmark/22-1.shader_test MESA_SHADER_FRAGMENT: 133 -> 135 (1.50%)

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12114>

lima: fail in get_handle(TYPE_KMS) without a scanout resource

The previous logic was returning a handle valid for the render-only
device if rsc->scanout was NULL. However the caller doesn't expect
this: the caller will use the handle with the KMS device.

Instead of returning a handle for the wrong device, fail if we don't
have one.

Signed-off-by: Simon Ser <contact@emersion.fr>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12074>

panfrost: fail in get_handle(TYPE_KMS) without a scanout resource

The previous logic was returning a handle valid for the render-only
device if rsc->scanout was NULL. However the caller doesn't expect
this: the caller will use the handle with the KMS device.

Instead of returning a handle for the wrong device, fail if we don't
have one.

Signed-off-by: Simon Ser <contact@emersion.fr>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12074>

freedreno: fail in get_handle(TYPE_KMS) without a scanout resource

The previous logic was returning a handle valid for the render-only
device if rsc->scanout was NULL. However the caller doesn't expect
this: the caller will use the handle with the KMS device.

Instead of returning a handle for the wrong device, fail if we don't
have one.

Signed-off-by: Simon Ser <contact@emersion.fr>
Reviewed-by: Emma Anholt <emma@anholt.net>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12074>

etnaviv: fail in get_handle(TYPE_KMS) without a scanout resource

The previous logic was returning a handle valid for the render-only
device if rsc->scanout was NULL. However the caller doesn't expect
this: the caller will use the handle with the KMS device.

Instead of returning a handle for the wrong device, fail if we don't
have one.

Signed-off-by: Simon Ser <contact@emersion.fr>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12074>

etnaviv: fix renderonly check in etna_resource_alloc

When the driver hasn't been initialized via renderonly, screen->ro
will be NULL. This fixes a crash when passing USE_SCANOUT to etnaviv
when it's missing renderonly.

Signed-off-by: Simon Ser <contact@emersion.fr>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12074>

nine: Fix assert in tx_src_param

A previous commit cleaned up the asserts but the last part of
this assert looks like it got mixed up. It should have allowed
param->rel for D3DSPR_INPUT if version is 3.0. Instead it does
&& on the enum value D3DSPR_ADDR which is of course always true,
with the version check. The result is that we miss input
validation with version 3.0.

Spotted by a compile warning

Fixes: 5974401a4ad ("st/nine: Regroup param->rel tests")
Reviewed-by: Axel Davy davyaxel0@gmail.com
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11880>

nir_to_tgsi: Add support for TXP.

This doesn't perfectly emit all the TXPs we might be able to (if you get
any non-TXP-able projectors, that whole sampler dim gets lowered), but it
should be a big boost to i915g.

softpipe shader-db:
total instructions in shared programs: 2953625 -> 2951201 (-0.08%)
instructions in affected programs: 108901 -> 106477 (-2.23%)
total temps in shared programs: 565869 -> 566632 (0.13%)
temps in affected programs: 8922 -> 9685 (8.55%)

i915g shader-db:
total instructions in shared programs: 372570 -> 370254 (-0.62%)
instructions in affected programs: 39825 -> 37509 (-5.82%)
total tex_indirect in shared programs: 11420 -> 11074 (-3.03%)
tex_indirect in affected programs: 913 -> 567 (-37.90%)
LOST:   0
GAINED: 12

Closes: #4984
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11912>

nir_to_tgsi: Pack our tex coords into vec4 nir_tex_src_backend[12].

For TGSI, we need the coordinate, comparator, bias, and LOD all together
in the first two vec4 args, and by doing it in the backend we were
generating extra MOVs.

softpipe shader-db results:
total instructions in shared programs: 2985416 -> 2953625 (-1.06%)
instructions in affected programs: 499937 -> 468146 (-6.36%)
total temps in shared programs: 544769 -> 565869 (3.87%)
temps in affected programs: 105469 -> 126569 (20.01%)

i915g shader-db:
total instructions in shared programs: 371625 -> 369594 (-0.55%)
instructions in affected programs: 24903 -> 22872 (-8.16%)
total tex_indirect in shared programs: 11381 -> 11365 (-0.14%)
tex_indirect in affected programs: 43 -> 27 (-37.21%)
LOST:   7
GAINED: 16

The temps increase is the pre-existing issue that we never release temps
for NIR regs, which doesn't matter much for softpipe (just memory/cache
footprint) but does for i915g as seen by shaders that no longer compile
(though overall we seem to win).

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11912>

i915g: Add support for shader-db.

I wanted to make sure that NIR-to-TGSI wouldn't regress this driver's code
generation, so make it possible to use standard shader-db on it.

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11912>

anv: fix submission batching with perf queries

If we have 2 command buffers back to back, one with a query pool, one
without, we don't want to retain the second query pool value (NULL).

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 0a7224f3ff7542 ("anv: group as many command buffers into a single execbuf")
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12107>

i965: Explicitly abort instead of exiting on batch failure

This avoids a deadlock condition when registered atexit handlers attempt to
acquire a mutex, but the program was going to exit with an error anyway

Reviewed-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Rohan Garg <rohan.garg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11813>

i965: Write a custom allocator for the intel memobj struct

This helps ensure we're not doing out of bounds access later.

Signed-off-by: Rohan Garg <rohan.garg@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12075>

i965: Take into account the offset when marking a valid data region

Signed-off-by: Rohan Garg <rohan.garg@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12085>

lavapipe: do not mark unsupported tests as crashing

These were fixed previously, but due to the CI not really running all
tests any more, I didn't notice these fixes. Let's bring the expected
results up to date.

Fixes: 2e29857bb65 ("llvmpipe: only report supported shader-image formats")
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12077>

broadcom: remove v3dv3 from neon library

No need to build the simulator with NEON; and also v3dv3 simulator
is not for VC4, so don't inherit v3dv3 requirement when building vc4
driver.

Fixes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5126
Fixes: d198e26a1eb ("broadcom/common: move v3d_tiling to common")
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12078>

spirv: set medium precision with RelaxedPrecision decorator

This allows the variables decorated with RelaxedPrecision to have the
proper precision. It is worth to note that the decorator can be
applied on other cases, but those would be handled on the future.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7614>

vulkan: add vk_spec_info_to_nir_spirv util method

All vulkan drivers have been copying anv's code to convert
VkSpecializationInfo into nir_spirv_specialization.

Recently there was a Vulkan spec change on allowed values for
VkSpecializationInfo, and all drivers got affected.

This commits creates a new helper, and uses it on all Vulkan Mesa
drivers.

v2: use (uint8_t*) castings, instead of void*, to avoid C2036 with
    MSVC (detected by the CI, inspired on what radv was doing)

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Juan A. Suarez <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12047>

microsoft/compiler: Correctly flag when using raw buffers

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12119>

microsoft/compiler: Add support for arrays to image_store

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12119>

nir_lower_readonly_images_to_tex: Fix typeo on image arrays

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12119>

glsl/standalone: Lower COMPUTE shader precision

Fragment isn't last for GLSL stages, compute is.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12082>

features: mark off timelines for lavapipe

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12071>

lavapipe: implement timeline semaphores

this uses a singly-linked list of timeline ids to compare against the device
queue's cmdbuf counter and determine which timeline id maps to which cmdbuf and
thus which fence can be waited on

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12071>

lavapipe: use consistent semaphore variable naming

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12071>

lavapipe: rework queue to use u_queue

this simplifies the entire queue mechanism and makes it more consistent:
previously some cases (e.g., null cmdbuf submission) would immediately
be marked as finished, which meant that fences could theoretically become
desynchronized, breaking application assumptions

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12071>

zink: simplify some dumb code in invalidate_buffer (v2)

the ref has to be preserved so that rebind doesn't destroy it prematurely

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12118>

Revert "zink: simplify some dumb code in invalidate_buffer"

This reverts commit 6bda555cc0e0b8c511b77989069402d6019452d2.

this needs to stay dumb in order to preserve the ref on the resource
without destroying it during rebind

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12118>

gallivm: Always take the per-pixel LOD path for cubemaps.

The VK spec requires that derivatives are mapped to the face, and given
that the face is per pixel that implies that we need per-pixel
derivatives.  This also seems to match hardware behavior.

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10219>

gallivm: Default brilinear filtering to off.

It's non-conformant for GL and Vulkan, and not expected to be a desired
quality/speed tradeoff for applications using llvmpipe.  The option is
left in place using GALLIVM_PERF=brilinear.

Causes minor rasterization changes in our traces:
- improved sharpness of the trees in CS
- more consistent mipmap filtering of the ground in CS:Source, STK and the
  RaytracedShadows demo.
- changing some aliasing on shadow maps in 0ad (not consistently
  visibly better or worse)

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10219>

nir/lower_shader_calls: adding missing stack offset alignment

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 8dfb240b1f0633 ("nir: Add raytracing shader call lowering pass.")
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12112>

venus: print warnings when stuck in busy waits

The first warning is printed after stuck in vn_relax for at least about
3.5s.  The actual time can be much longer depending on the
kernel/load/hw.

Signed-off-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Ryan Neph <ryanneph@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12115>

zink: force threadsafe mapping for query results when necessary

this would otherwise use the ctx-based slab allocator, which races
and crashes

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12015>

pan/bi: Teach meson about format pack tests

Now we have all our Bifrost unit tests under meson and can remove the
test entrypoint from bifrost_compiler. This does require a small
refactoring for our util_dynarray handling to make sure we don't leak
memory. Otherwise meson-arm64-asan complains.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12109>

pan/bi: Teach meson about Bifrost packing test

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12109>

pan/bi: Teach meson about scheduler predicate test

One step of 3 to getting all our tests in meson test.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12109>

pan/bi: Add BIT_ASSERT helper for unit testing

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12109>

pan/bi: Expose unit tested scheduler predicates

I want to move the tests to their own executable to integrate better
with meson.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12109>

pan/bi: Remove dated ASSERTED properties

This was used when we were only unit testing. Now that we have an actual
scheduler, it's pure noise.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12109>

pan/bi: Inline away bi_must_last

Totally trivial.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12109>

freedreno/ir3: Add build id to the disassembler test

This is required (at least for me on x86) to get the tool to pass it's
own test, otherwise it fails the build_id assertion.

Fixes: 1462b00391711ff86350e802c1a3e5075a3a1632
       ("freedreno/ir3: Add a unit test for our disassembler.")

Acked-by: Rob Clark <robclark@freedesktop.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12084>

zink: never use staging buffer for unsynchronized buffer maps

this is ultra broken, do not attempt

Fixes: 6bfbce0447b ("zink: rework buffer mapping")

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12111>

mesa/main: Check for fbo attachments when importing EGL images to textures

Fixes an assert when binding an fbo with a texture bound to one of its attachments,
if the texture was updated with an EGL image after it was bound.

Cc: mesa-stable
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11998>

iris: Split iris_upload_shader in two

Now the part that uploads the shader and the part that finishes the
creation of the shader are separated.  Each now has a more reasonable
number of parameters.

Suggested-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11229>

iris: Enable threaded shader compilation

There are a couple minor things that can be improved:

1. Eliminate (or reduce) the dynamic allocation of the
threaded_compile_job.

2. For apps like shader-db, improve the case where nr_threads=0.  Right
now this adds thread switching and mutex overhead.

3. Other performance improvements?  iris_uncompiled_shader::variants has
some special properties that make it ripe for replacement with a
lockless list.  Without gathering some data, it's hard to guess what
impact that could have.

v2: Fix whitespace and formatting issues.  Noticed by Ken.
s/threaded_compile_job/iris_threaded_compile_job/g.  Suggested by Ken.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11229>

iris: add sync_compile option

This can be useful to simplify debugging compiler issues.

Similar to 9445a4ab439 ("radeonsi: add radeonsi_sync_compile option").

v2: Actually query the driconf to set screen->driconf.sync_compile.
Noticed by Ken.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11229>

iris: Don't pass the shader key to iris_compile_[shader stage]

Use the copy of the key embedded in the compiled shader.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11229>

iris: Add the variant to the list as early as possible

I tried to find a way to break this into some smaller commits, but
everything is very intertwined. :(

When searching the variants list in the iris_uncompiled_shader, add the
new variant if it is not found.  This will be necessary for threaded
shader compilation.  This conceptually simple change had a bunch of
fallout.

Much of this was at least conceptually borrowed from radeonsi.

- Other threads might find a variant in the list before the variant has
  been compiled.  To accomdate this, add a fence.  Each thread will wait
  on the fence in the variant when searching the list.

- A variant in the list may fail compilation.  To accomodate this, add a
  flag.  All paths will examine iris_compiled_shader::compilation_failed
  before trying to use the variant.

- The race condition between multiple threads trying to create the same
  variant at the same time is handled *before* both thread spend the
  effort to compile the shader.  The means that iris_upload_shader
  cannot change shaders on the caller, so it does not need to return
  anything.

v2: Change "found" parameter of find_or_add_variant to "added."  This
inverts the values returned, and it probably makes uses of the returned
value more easily understood.  Always set the value in the called
function.  Suggested by Ken.

v3: Move shader->compilation_failed check to avoid shader != NULL test.
Rearrange some logic and add a comment in iris_update_compiled_tcs.
Suggested by Ken.  Don't call find_or_add_variant in
iris_create_shader_state.  See
https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11229#note_1000843
for more details.  Noticed by Ken.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11229>

iris: Allocate shader variant in caller of iris_upload_shader

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11229>

iris: Extract allocation bits from iris_upload_shader to iris_create_shader_variant

The added assertion in iris_create_shader_variant helped catch a bug in
the next commit.

v2: Drop (unnecessary) initialization of shader->assembly.res when
moving to iris_create_shader_variant.  Suggested by Ken.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11229>

iris: Ref count the uncompiled shaders

I tried /just/ ref counting the uncompiled shaders, but that is not
sufficient.  At the very least, it's a problem for blorp shaders that
only have variants (and no uncompiled shader).

This is in prepartion for using the live shader cache.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11229>

iris: Merge iris_create_[shader stage]_state funcs into iris_create_shader_state

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11229>

iris: Unify iris_create_[shader stage]_state functions

v2: Merge old iris_create_shader_state into the new function.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11229>

iris: Unify iris_delete_[shader stage]_state functions

This is in prepartion for using the live shader cache.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11229>

iris: Fix return type of iris_compile_*

Trivial.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11229>

iris: Add a comment for iris_uncompiled_shader::nir

This would have saved me a few minutes searching around.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11229>

util/queue: Don't crash in util_queue_destroy when init failed

This simplifies the error exit paths for drivers that use these queues.

v2: Move allocation of queue->jobs after initializing the mutxes and
condition variables.  Noticed by Ken.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11229>

st/texture: refactor get_src_format() to be more useful

take more params in order to enable better reuse

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11982>

mesa/st: add format-finding capabilities to pbo get_dst_format()

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11982>

mesa/st: enable calling st_choose_format() purely for translation

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11982>

mesa/st: break up st_choose_matching_format()

having a function that just returns the equivalent format is handy

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11982>

mesa/st: break up st_GetTexSubImage

this makes the function more readable and enables a lot of the code
to be reused for other purposes

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11982>

ir3: Preserve gl_ViewportIndex in the binning shader

Fixes dEQP-VK.draw.shader_viewport_index.* with TU_DEBUG=forcebin.

Fixes: efff7342204 ("turnip: multiViewport and VK_EXT_shader_viewport_index_layer")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12104>

gallium/tests: Fix warning calculating absdiff

With clang building tests:

../src/gallium/tests/trivial/compute.c:1215:29: warning: taking the absolute value of unsigned type 'unsigned int' has no effect [-Wabsolute-value]
                        if (abs(((uint32_t *)x)[j] -
                            ^
../src/gallium/tests/trivial/compute.c:1215:29: note: remove the call to 'abs' since unsigned values cannot be negative
                        if (abs(((uint32_t *)x)[j] -

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12087>

loader/dri3: create linear buffer with scanout support

If we have a different GPU dealing with display, we fallback to
exchanging linear buffers with the compositor. We should specify in
creating the linear buffer that this could be used for display.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4706
Cc: mesa-stable
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11942>

tu: delay decision of forcing sysmem due to subpass self-dependencies

DXVK always inserts vertex stage subpass self-dependency for every
subpass regardless of whether there actually would be a barrier.
This effectively disabled gmem rendering with DXVK.

Thus we delay the decision to disable gmem rendering until we
see a barrier with vertex stages.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12038>

docs/ci: Update http cache config to let Authorization headers pass through

When downloading traces from a restricted repository, the Authorization
header will be set. Previous to this change, the http proxy would drop
it.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6441>

ci/freedreno: Test with non-redistributable traces

Add a job to test with traces that we cannot redistribute, listed in a
separate file. Since those traces might not be accessible by everyone,
this job is created only when the pipeline is triggered by `marge-bot`.

This job is optional because otherwise it could be blocking a merge
request of someone who cannot really debug the issue due to lack of
access to these traces.

The documentation available under `docs/ci` goes into more details
explaining the rationale behind optional traces.

Signed-off-by: Antonio Caggiano <antonio.caggiano@collabora.com>
Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6441>

gallium/vl: don't leak fd in vl_dri3_screen_create

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12092>

radeonsi: preserve derivatives after discards for ARB shaders

Contrary to the GLSL spec, the ARB fp spec doesn't explicitely
states that derivatives are undefined after discards.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5018
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11972>

mesa: move gl_program::is_arb_asm to shader_info

This will be used from radeonsi in the next commit.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11972>

lavapipe: do not assert on more than 32 samplers

We can have more than 32 samplers, but the code below will assert in that
case. The return value is not used for samplers, so let's just return
zero early and be done with it.

Fixes: c18ff60087e ("lavapipe: emit correct textures_used for texture-arrays")
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11845>

vulkan/wsi/x11: do not inherit last_present_mode

Under XWayland, the first present after a window resize is sometimes
completed with COPY (seems to happen when the previous present with the
old size is pending; not really sure).  The following presents are
completed with FLIP.

When a swapchain is created with an old swapchain, and
old_chain->last_present_mode is FLIP, chain->last_present_mode is set to
FLIP as well.  This causes the new swapchain to be marked
VK_SUBOPTIMAL_KHR, which is sticky, if the first present is completed
with COPY.

Instead of inheriting, treat each swapchain as independent.  We will
miss the case where an old swapchain is flipping but a new swapchain is
copying.  But swapchain reallocation normally happens in response to
present engine state change.  If the newly allocated swapchain is
copying, another reallocation is unlikely to fix that.

Fixes: 61309c2a727 ("vulkan/wsi/x11: Return VK_SUBOPTIMAL_KHR for X11")
Signed-off-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-by: Louis-Francis Ratté-Boulianne <lfrb@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12030>

zink: rework buffer mapping

this collects a bunch of changes which need to be made but which can't
be split up incrementally without breaking things:
* move map offset to zink_transfer::offset for consistent handling
* fix stream uploader map rules to only apply for discard cases so
  the stream uploader doesn't explode
* fix the staging buffer map rules to apply to any non-staging readback
* stop making huge staging buffers for readback
* break out the DONTBLOCK case since this is only coming from qbo readback
* add explicit read-only sync for maps to stall on writes

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12090>

zink: use MAP_ONCE for qbo readback

this implies that the buffer can be unmapped

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12090>

pan/va: Allow floating-point swizzles on ATEST

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12026>

pan/bi: Align staging registers on Valhall

This handles the following from the Valhall specification (that I wrote):

    If multiple subsequent staging registers are accessed, the base must
    be aligned to 2. However, even if 4 registers are accessed, it is
    not necessary to align to 4, only to 2. This restriction allows the
    hardware to use a 64-bit data path without handling unaligned
    access, which is more efficient. This restriction does not apply if
    only a single register is accessed.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12026>

pan/bi: Clarify the logic of bi_reconverge_branches

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12026>

pan/bi: Clean up and export bi_reconverge_branches

Decides when we need "branch reconvergence" (canonical term), the
logical opposite of "back-to-back execution" (non-canonical term, this
is old code in Bifrost terms). So invert the return value, rename, and
export so we can use it when packing Valhall instructions.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12026>

pan/bi: Add branch_offset immediate

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12026>

pan/bi: Garbage collect stuff in bi_layout.c

Predates clause scheduling.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12026>

pan/bi: Fix UBO push with nir_opt_shrink_vectors

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12026>

pan/bi: Add constant folding unit test

I just played with the implementation, let's ensure I didn't break it.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12026>

pan/bi: Refactor constant folding for testability

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12026>

pan/bi: Use bi_apply_swizzle in constant folding

Much more legible now.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12026>

pan/bi: Add helper to swizzle a constant

Instead of open-coding special cases in multiple places, just keep a
canonical version that handles every case.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12026>

pan/bi: Add strip_index helper

Needed to correctly lower sources to moves, used in the FAU lowering.
Technically, the issue is already present on Bifrost, but it's hidden
because Bifrost packing doesn't validate the absense of unsupported
modifiers. Valhall packing adds more safety, which caught this issue.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12026>

pan/bi: Rename CLPER_V7 back to CLPER

v6 is really the oddball here. CLPER on v9 supports a superset of v7.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12026>

pan/bi: Rename NOP.i32 to NOP

Simpler and matches the syntax of Valhall I've picked out.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12026>

pan/bi: Fix typo in FAU enum

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12026>

pan/bi: Model Valhall special values as FAU

Not sure if this is "really" FAU but it's convenient to treat it as
such.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12026>

pan/bi: Model RSCALE for Valhall

Faster than FMA_RSCALE. We'll want to optimize this (e.g. for exp2).

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12026>

pan/bi: Model *ADD_IMM instructions in IR

Needed for Valhall.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12026>

pan/bi: Remove unused BIR_FAU_HI

Redundant with offset.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12026>

pan/bi: Add discard flag to bi_index

Needed to model Valhall instructions. Should also be useful to RA if we
ever get around to doing something SSA based.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12026>

pan/bi: DCE after bifrost_nir_lower_algebraic_late

Needed for sat_signed to fuse, since we run modifier prop before backend
DCE.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12026>

pan/bi: Constify BIR manipulation

For use in Valhall packing.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12026>