zink: ci updates

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15020>

zink: always invalidate streamout counter buffer if not resuming

this otherwise treates begin/end/begin the same as begin/pause/resume

cc: mesa-stable

fixes:
KHR-GL46.texture_view.view_classes
KHR-GL46.transform_feedback.capture_geometry_separate_test
KHR-GL46.transform_feedback.capture_vertex_separate_test
KHR-GL46.transform_feedback.query_geometry_separate_test
KHR-GL46.transform_feedback.query_vertex_separate_test

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15020>

zink: export PIPE_SHADER_CAP_INDIRECT_TEMP_ADDR

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15020>

zink: map R8G8B8X8_SRGB -> R8G8B8A8_SRGB

this fixes a weird texstore bug that seems specific to this format

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15020>

zink: ci updates

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15018>

zink: activate conditional render for compute dispatch when necessary

fixes:
KHR-GL46.compute_shader.conditional-dispatching

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15018>

zink: restart conditional render when crossing batch boundary

this will only happen when conditional render was being used outside
of a renderpass

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15018>

zink: always terminate conditional render when flushing a batch

we might not know whether conditional render is active,
so forcibly disable when necessary

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15018>

zink: track internal conditional render state

this allows no-oping redundant calls

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15018>

intel/compiler: remove what looks like a bad rebase

This bit in the compiler looks like it was added by accident on one of
the latest versions of the original commit, but it clearly doesn't
belong there.

Fixes: 03e1e19246d ("anv: Refactor descriptor copy")

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15016>

gallivm/st/lvp: add flags arg to get_query_result_resource api.

Currently this just has wait, but in order to get the right answer
for vulkan partial, lavapipe/llvmpipe need to pass a partial flag
through here in the future.

This just changes the API so that's possible.

v2: use an enum (zmike)

Acked-by: Ilia Mirkin <imirkin@alum.mit.edu>
Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15009>

nir_to_tgsi: Add support for FBFETCH.

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15014>

nir_to_tgsi: Don't vectorize 64-bit instructions, to keep virgl happy.

virglrenderer makes invalid shaders when faced with vector 64-bit
instructions, which GLSL-to-TGSI never produced.  While this doesn't fix
everything, it does get more tests running, and virgl probably the primary
consumer of 64-bit TGSI.  virgl may be deprecating its host 64-bit
support, at which point we can drop this workaround.

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15014>

virgl: Move double operands to a temp to avoid double-swizzling bugs.

virglrenderer applies the swizzle when packing doubles.  Since we use scan
now, we can just use the file count for texture detection.

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15014>

virgl: Move tex immediate operands to a temp to avoid virglrenderer bug.

Prior to the noted MR, virglrenderer encoded the tex operands in a
limited-size temp buffer which we could easily overflow with TGSI
immediates.  GLSL-to-TGSI always emitted an extra MOV, so keep that
behavior when nir-to-tgsi feeds us more optimized TGSI.

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15014>

virgl: Add workarounds for virglrenderer input/sv signedness bugs.

GLSL-to-TGSI would emit some MOVs that made things OK, but NTT
successfully copy-propagates the inputs and sysvals more, triggering bugs
in virglrenderer when the signedness of the input is different from that
of the ALU op.

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15014>

virgl: Apply TGSI transforms to compute shaders, too.

We need to do this for the upcoming virglrenderer workarounds.

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15014>

virgl: Add a workaround for virglrenderer output writemask bugs.

Various workaround paths in virglrenderer assume that outputs are written
with a full writemask, which is not required by TGSI.  To work around
that, store affected outputs in a temp and do a full write after each
writemasked write.

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15014>

virgl: Work around old virglrenderer's BARRIER counting bug.

One less regression from doing nir-to-tgsi on CI's virglrenderer.

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15014>

tgsi_translate: Make the procType public when translating.

This means that tgsi_translate users can check the PIPE_SHADER stage
without having to separately tgsi_scan().

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15014>

venus: Exposes VK_EXT_4444_formats extension

Allows venus to passthrough the VK_EXT_4444_formats extension to
the vulkan client.

And add code to the device initialization and feature query
functions.

Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-by: Yiwei Zhang <zzyiwei@chromium.org>
Signed-off-by: Igor Torrente <igor.torrente@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14954>

turnip: advertise VK_EXT_queue_family_foreign

Both Venus and Android AHB requires this extension.

Turnip ignores VK_SHARING_MODE_EXCLUSIVE so this is a no-op.

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Acked-by: Rob Clark <robdclark@chromium.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14836>

ir3: Limit the maximum imm offset in nir_opt_offset for shared vars

STL/LDL have 13 bits to store imm offset.

Fixes crash in CS compilation in Monster Hunter World.

Fixes: b024102d7c2959451bfef323432beaa4dca4dd88
("freedreno/ir3: Use nir_opt_offset for removing constant adds for shared vars.")

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14968>

intel/compiler: fix array & struct IO lowering in mesh shaders

We really need offsets to be in dwords, not in vec4s.

The bug manifests as random failure of func.mesh.clipdistance.5 crucible
test, where stores to gl_MeshVerticesNV[x].gl_ClipDistance[4+n] actually write to
gl_MeshVerticesNV[x].gl_ClipDistance[1+n].

Fixes: 1f438eb0337 ("intel/compiler: Implement Mesh Output")
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14997>

spirv: Create PRIMITIVE_INDICES for NV_mesh_shader on-demand.

The shader can have SpvOpWritePackedPrimitiveIndices4x8NV while the
output variable may not exist. This seems to be a defect in the
NV_mesh_shader SPIR-V spec, let's work around it by creating the
variable on-demand.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15005>

compiler: Extract num_mesh_vertices_per_primitive function.

Prevent code duplication.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15005>

radv: remove set but unused radv_buffer::shareable

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14937>

radv: remove useless NULL checks in vkBind{Buffer,Image}Memory2()

The memory object must be a valid vkDeviceMemory handle.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14937>

radv: fix finding shaders by PC

Shaders are allocated contiguously in memory for a pipeline and
the freelist.next pointer is a pointer to the pipeline now.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14950>

radv: make the trap handler shader BO resident

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14950>

radv: fix allocating/uploading the trap handler shader

Since shaders are allocated per pipeline, the trap handler shader
was not uploaded at all.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14950>

docs/features: Mark VK_KHR_ray_query in progress

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15000>

freedreno/ir3: split up load/store/atomic by generation

Some bits are slightly different on a4xx. Use the encodings that work.
Perhaps these can be combined at some point if we get a proper
understanding of what they mean.

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14789>

isaspec: add gen-based leaf bitset separation

This is necessary for some ops which have slightly different encoding on
a4xx/a5xx, but are otherwise identical. This helps keeping the compiler
from having to worry about these details and creating separate ops.

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14789>

isaspec: fix gen_max to be 2^32-1

The minus sign has higher preference than shift:

>>> 1 << 32 - 1
2147483648
>>> (1 << 32) - 1
4294967295

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14789>

panfrost: Flesh out the Shader Program Descriptor

Only breaking change since Bifrost is that the shader contains barrier? flag is
now fragment-only, meaning it is just a spawn helper threads flag. This affects
compute shaders slightly.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15003>

panfrost: Simplify Valhall preload descriptor

Honestly, we could stand to do the same to Bifrost...

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15003>

panfrost: Clarify unknowns in z/stencil descriptor

Depth culling and clamping.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15003>

panfrost: Add more fields to Attribute Descriptor

More XML

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15003>

panfrost: Update primitive descriptor for Valhall

Contains stuff needed for layered rendering. Unfortunately, there's no more
provoking vertex per draw -- ugh! That's fine for Vulkan (just don't set
provokingVertexModePerPipeline), but requires inserting extra flushes on desktop
OpenGL.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15003>

radv: Only wait on CS/PS to finish if we wait on a semaphore.

I think plain submission doesn't need it.

Reviewed-By: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14574>

radv: Fix preamble argument order.

Used the wrong cmdbuffer in the wrong situation. Oops.

Fixes: 915e9178faf ("radv: Split out commandbuffer submission.")
Reviewed-By: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14574>

radv: Use larger arena sizes.

For some games that take like 400 MiB of shader binaries, the
number of shader arenas ends up going >1500. Cut that down a bit
by using larger arenas.

8 MiB should still be decent with small BAR and should still cut
things down from ~1500 to ~50 buffers.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14591>

lima/ppir: refactor bitcopy to use unsigned char

This code does not work as expected when built with clang and
-fstrict-aliasing.
Redefine it in unsigned char operations so that it does not
violate strict aliasing rules.

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Cc: 22.0 <mesa-stable>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14894>

lima/ppir: initialize slots array for dummy/undef

Some functions in ppir iterate the ppir_op_info slots arrays looking
for the PPIR_INSTR_SLOT_END token. The dummy/undef internal ops may
appear in the scheduling code and their slots arrays did not contain
that token, which could result in invalid array reads.
Reported by gcc -fsanitize=address.

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Cc: 22.0 <mesa-stable>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14894>

lima/gpir: avoid invalid write in regalloc

Reported by gcc -fsanitize=address, sometimes gpir regalloc attempts to
handle an uninitialized node->value_reg (containing the value -1), which
results in an invalid array access.
Avoid it for now to prevent crashes, but more investigation may be
required later on.

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Cc: 22.0 <mesa-stable>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14894>

lima: remove an unneeded lima_job_get assignment

Reported by scan-build.

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14894>

lima: add some checks for potential null pointer dereference

scan-build complains about a potential null pointer dereference in
some places around the lima code.
None of those seem to be a real issue as of now, but let's add some
asserts to cover for that and clean up the warning list.

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14894>

lima: fix warning of garbage value access

scan-build complains that an access of reg[j+1] in this code might
return garbage.
Let's take the chance to clean this open coded sorting code up and
just use qsort.

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14894>

lima/ppir: initialize spill_costs array in regalloc

Static analysis complains that spill_costs might be accessed in
non-initialized positions.
It does not seem to be an issue with the current code which initializes
it for every relevant register index, but we can also just initialize it
to not have to worry about that.

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14894>

lima/ppir: avoid ppir_codegen_outmod implicit conversion

Fix some clang -Wenum-conversion warnings like:

  warning: implicit conversion from enumeration type 'ppir_outmod' to
  different enumeration type 'ppir_codegen_outmod' [-Wenum-conversion]

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14894>

lima/ppir: clean up override-init warnings

Define ppir_op_unsupported as 0 so that we don't have to do the
initialization to -1.

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14894>

lima/gpir: clean up override-init warnings

Define gpir_op_unsupported as 0 so that we don't have to do the
initialization to -1.

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14894>

venus: use 64KB alignment for suballocations

TGL CCS surface addresses must be aligned to 64KB.

Reviewed-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15001>

venus: no roundtrip needed for shmem backed by BLOB_MEM_HOST3D

A successful DRM_IOCTL_VIRTGPU_MAP on BLOB_MEM_HOST3D implies a
roundtrip.

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-by: Ryan Neph <ryanneph@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14658>

venus: init renderer_info at renderer creation (part 2)

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-by: Ryan Neph <ryanneph@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14658>

venus: init renderer_info at renderer creation (part 1)

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-by: Ryan Neph <ryanneph@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14658>

aco/insert_exec_mask: refactor and remove some unnecessary WQM handling code

Some cases cannot happen and don't need to be handled anymore.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14951>

aco/insert_exec_mask: refactor and simplify get_block_needs()

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14951>

aco/insert_exec_mask: remove ever_again_needs and Exact_Branch

This information is not required anymore.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14951>

aco/insert_exec_mask: remove some unnecessary WQM loop handling code

These workarounds are were necessary to prevent infinite loops
with helper lane registers containing wrong data.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14951>

aco/insert_exec_mask: remove Preserve_WQM flag

If WQM is needed anywhere after discard_if(), it will also
be flagged as WQM. We can rely on that to preserve the WQM mask.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14951>

aco: don't emit WQM for bool_to_scalar_condition

This was only necessary to ensure that the source is computed in WQM
if the current exec mask is in WQM.

Totals from 23170 (17.17% of 134913) affected shaders: (GFX10.3)
VGPRs: 1384464 -> 1383400 (-0.08%); split: -0.08%, +0.01%
SpillSGPRs: 7575 -> 7574 (-0.01%)
CodeSize: 146444752 -> 146317104 (-0.09%); split: -0.13%, +0.04%
MaxWaves: 429870 -> 429868 (-0.00%)
Instrs: 27202586 -> 27170316 (-0.12%); split: -0.17%, +0.05%
Latency: 379488313 -> 379335412 (-0.04%); split: -0.07%, +0.03%
InvThroughput: 69500561 -> 69487704 (-0.02%); split: -0.03%, +0.01%
VClause: 473080 -> 473038 (-0.01%); split: -0.02%, +0.01%
SClause: 1080576 -> 1080571 (-0.00%); split: -0.06%, +0.06%
Copies: 1492865 -> 1504604 (+0.79%); split: -0.40%, +1.19%
Branches: 711295 -> 716849 (+0.78%); split: -0.01%, +0.79%
PreSGPRs: 1331362 -> 1328402 (-0.22%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14951>

aco: don't propagate WQM for p_as_uniform

This was needed, so that in case of active helper lanes,
these contain the correct value. It is now handled implicitly.

Totals from 1004 (0.74% of 134913) affected shaders: (GFX10.3)
CodeSize: 7581020 -> 7580892 (-0.00%); split: -0.00%, +0.00%
Instrs: 1454940 -> 1454908 (-0.00%); split: -0.00%, +0.00%
Latency: 12984953 -> 12984894 (-0.00%); split: -0.00%, +0.00%
InvThroughput: 3173037 -> 3173049 (+0.00%); split: -0.00%, +0.00%
PreSGPRs: 47498 -> 47273 (-0.47%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14951>

aco/insert_exec_mask: stay in WQM while helper lanes are still needed

This patch flags all instructions WQM which don't require
Exact mode, but depend on the exec mask as long as WQM
is needed on any control flow path afterwards.
This will mostly prevent accidental copies of WQM values
within Exact mode, and also makes a lot of other workarounds
unnecessary.

Totals from 17374 (12.88% of 134913) affected shaders: (GFX10.3)
VGPRs: 526952 -> 527384 (+0.08%); split: -0.01%, +0.09%
CodeSize: 33740512 -> 33766636 (+0.08%); split: -0.06%, +0.14%
MaxWaves: 488166 -> 488108 (-0.01%); split: +0.00%, -0.02%
Instrs: 6254240 -> 6260557 (+0.10%); split: -0.08%, +0.18%
Latency: 66497580 -> 66463472 (-0.05%); split: -0.15%, +0.10%
InvThroughput: 13265741 -> 13264036 (-0.01%); split: -0.03%, +0.01%
VClause: 122962 -> 122975 (+0.01%); split: -0.01%, +0.02%
SClause: 334805 -> 334405 (-0.12%); split: -0.51%, +0.39%
Copies: 275728 -> 282341 (+2.40%); split: -0.91%, +3.31%
Branches: 92546 -> 90990 (-1.68%); split: -1.68%, +0.00%
PreSGPRs: 504119 -> 504352 (+0.05%); split: -0.00%, +0.05%

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14951>

Re-indentation after the previous commit

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Eliminate ir_assignment::condition

Reformatting is left for the next commit.

v2: Remove assignments from the contructors. :face_palm:

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Make ir_assignment::condition private

And add get_condition().

This proof that nothing remains that could possibly set ::condition to
anything other than NULL.

v2: Fix bad rebase.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Don't visit rvalues in the condition of an assignment

At this point, this should always be NULL.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Don't lower vector indexing in the condition of an assignment

At this point, this should always be NULL.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Don't split structures in the condition of an assignment

At this point, this should always be NULL.

v2: Fix bad rebase.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Don't split arrays in the condition of an assignment

At this point, this should always be NULL.

v2: Fix bad rebase.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Don't tree graft in the condition of an assignment

At this point, this should always be NULL.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Don't dead-built-in varying eliminate in the condition of an assignment

At this point, this should always be NULL.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Remove unused condition parameter from ir_assignment constructor

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Don't constant-fold the condition of an assignment

At this point, this should always be NULL.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Don't clone assignment conditions

At this point, this should always be NULL.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Eliminate unused conditional assignment constructor

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Remove the ability to read text IR with conditional assignments

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Add ir_assignment constructor that takes just a write mask

The other constructor that takes a write mask and a condition will be
removed shortly.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Lower if to conditional select instead of conditional assignment

Platforms that don't have flow control also don't have anything that
could be written that has a side effect.  It should be safe to implement
these condition writes as

    foo = csel(condition, bar, foo);

This should eliminate the last thing in the GLSL compiler that can
create new conditions on assignments.  Everything else that can store
something in ir_assignment::condition derives it from a pre-existing
condition.

v2: Fix bad rebase.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl/ir_builder: Eliminate unused conditional assignment builders

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Don't try to emit the "linear sequence" in lower_variable_index_to_cond_assign

When there are four or fewer elements left in the array partition, the
strategy changes from a binary search of nested flow control to sequence
of conditional assignments like

    (assign, dest, src[constant_i+0], index == constant_i+0)
    (assign, dest, src[constant_i+1], index == constant_i+1)
    (assign, dest, src[constant_i+2], index == constant_i+2)
    (assign, dest, src[constant_i+3], index == constant_i+3)

or

    (assign, dest[constant_i+0], src, index == constant_i+0)
    (assign, dest[constant_i+1], src, index == constant_i+1)
    (assign, dest[constant_i+2], src, index == constant_i+2)
    (assign, dest[constant_i+3], src, index == constant_i+3)

Realistically, the first case should use ir_triop_csel instead.

The second case will either get turned back into flow control like

    if (index == constant_i+0)
        (assign, dest[constant_i+0], src)

    if (index == constant_i+1)
        (assign, dest[constant_i+1], src)

    if (index == constant_i+2)
        (assign, dest[constant_i+2], src)

    if (index == constant_i+3)
        (assign, dest[constant_i+3], src)

or a sequence of conditional selects like

    (assign, dest[constant_i+0], (csel, index == constant_i+0, src, dest[constant_i+0]))
    (assign, dest[constant_i+1], (csel, index == constant_i+1, src, dest[constant_i+1]))
    (assign, dest[constant_i+2], (csel, index == constant_i+2, src, dest[constant_i+2]))
    (assign, dest[constant_i+3], (csel, index == constant_i+3, src, dest[constant_i+3]))

The former case should continue to use the binary search.  The later
case could be generated from the binary search by other lowering passes.

At the end of the day, conditional assignments don't really help
anything here, so stop using them.

Radeon R430
total instructions in shared programs: 2398683 -> 2398419 (-0.01%)
instructions in affected programs: 5143 -> 4879 (-5.13%)
helped: 9
HURT: 8

total vinst in shared programs: 616292 -> 616010 (-0.05%)
vinst in affected programs: 4467 -> 4185 (-6.31%)
helped: 9
HURT: 8

total sinst in shared programs: 315417 -> 315667 (0.08%)
sinst in affected programs: 2568 -> 2818 (9.74%)
helped: 2
HURT: 15

total flowcontrol in shared programs: 1049 -> 1048 (-0.10%)
flowcontrol in affected programs: 7 -> 6 (-14.29%)
helped: 1
HURT: 0

total presub in shared programs: 47027 -> 47027 (0.00%)
presub in affected programs: 127 -> 127 (0.00%)
helped: 1
HURT: 1

total omod in shared programs: 3618 -> 3615 (-0.08%)
omod in affected programs: 8 -> 5 (-37.50%)
helped: 3
HURT: 0

total temps in shared programs: 450757 -> 451312 (0.12%)
temps in affected programs: 837 -> 1392 (66.31%)
helped: 8
HURT: 6

total consts in shared programs: 1031928 -> 1031920 (<.01%)
consts in affected programs: 1211 -> 1203 (-0.66%)
helped: 6
HURT: 7

The shaders that were hurt for temps... are all lies.  None of those
shaders should have compiled as all 6 had more than 32 temps to begin
with.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Use csel in do_vec_index_to_cond_assign

This matches what NIR does.  See nir_vector_extract.

This improves code generation for several reasons.  First, it only
requires 3 comparisons instead of 4 (vec3(i > 0, i > 1, i > 2) vs
vec4(i == 0, i == 1, i == 2, i == 3)).

Secondly, it shortens the liverange of some values, possibly quite
dramatically.  Consider a loop in the old version (after lowering
if-statements to selects):

    loop {
        ...
x = csel(i == 0, a[0], x);
x = csel(i == 1, a[1], x);
x = csel(i == 2, a[2], x);
x = csel(i == 3, a[3], x);
        ...
    }

x is live for the whole loop across iterations.  In the new version, x
is only live while its value is needed:

    loop {
        ...
t0 = csel(i > 0 , a[1], a[0]);
t1 = csel(i > 2 , a[3], a[2]);
x = csel(i > 1, t1, t0);
        ...
    }

Outside a loop, this also means more values of the array may have their
liveness reduced sooner (by consuming two values at once).

All Intel platforms had similar results. (Tigerlake shown)
total instructions in shared programs: 21171336 -> 21163615 (-0.04%)
instructions in affected programs: 89680 -> 81959 (-8.61%)
helped: 40
HURT: 4
helped stats (abs) min: 1 max: 450 x̄: 193.68 x̃: 196
helped stats (rel) min: 0.41% max: 13.32% x̄: 6.01% x̃: 6.22%
HURT stats (abs)   min: 1 max: 12 x̄: 6.50 x̃: 6
HURT stats (rel)   min: 0.50% max: 0.66% x̄: 0.58% x̃: 0.58%
95% mean confidence interval for instructions value: -229.68 -121.28
95% mean confidence interval for instructions %-change: -6.93% -3.89%
Instructions are helped.

total cycles in shared programs: 832879641 -> 829513122 (-0.40%)
cycles in affected programs: 44738430 -> 41371911 (-7.52%)
helped: 35
HURT: 2
helped stats (abs) min: 2 max: 189948 x̄: 96186.49 x̃: 116154
helped stats (rel) min: 0.37% max: 11.08% x̄: 5.88% x̃: 6.47%
HURT stats (abs)   min: 4 max: 4 x̄: 4.00 x̃: 4
HURT stats (rel)   min: 0.69% max: 0.69% x̄: 0.69% x̃: 0.69%
95% mean confidence interval for cycles value: -112881.94 -69092.06
95% mean confidence interval for cycles %-change: -6.77% -4.27%
Cycles are helped.

total spills in shared programs: 8061 -> 7338 (-8.97%)
spills in affected programs: 873 -> 150 (-82.82%)
helped: 24
HURT: 0

total fills in shared programs: 7501 -> 6388 (-14.84%)
fills in affected programs: 1389 -> 276 (-80.13%)
helped: 24
HURT: 0

Radeon R430
total instructions in shared programs: 2449852 -> 2449136 (-0.03%)
instructions in affected programs: 6285 -> 5569 (-11.39%)
helped: 64
HURT: 0
helped stats (abs) min: 4 max: 12 x̄: 11.19 x̃: 12
helped stats (rel) min: 8.16% max: 21.62% x̄: 12.09% x̃: 10.91%

total consts in shared programs: 1032517 -> 1032482 (<.01%)
consts in affected programs: 966 -> 931 (-3.62%)
helped: 35
HURT: 0
helped stats (abs) min: 1 max: 1 x̄: 1.00 x̃: 1
helped stats (rel) min: 2.94% max: 10.00% x̄: 4.26% x̃: 3.57%

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl/lower_vector_derefs: Don't emit conditional assignments

Use if-statements instead.  Any hardware that supports this sort of
tessellation has flow control, so it will probably emit the conditional
assignment using an if-statement anyway.  This is definitely what
st_glsl_to_nir does.

v2: Fix copy-and-paste bug in the ir_type_swizzle handling.  This bug
caused segfaults in tests/spec/arb_tessellation_shader/execution/variable-indexing/tcs-patch-vec4-swiz-index-wr.shader_test.

Reviewed-by: Matt Turner <mattst88@gmail.com> [v1]
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

r300: transform fs sin and cos input to [0,1) range in NIR

shader-db stats with RV530 (together with the vs commit):
total instructions in shared programs: 65194 -> 64721 (-0.73%)
instructions in affected programs: 6718 -> 6245 (-7.04%)
total consts in shared programs: 45363 -> 45353 (-0.02%)
consts in affected programs: 466 -> 456 (-2.15%)

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5982
Reviewed-by: Emma Anholt <emma@anholt.net>
Signed-off-by: Pavel Ondračka <pavel.ondracka@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14957>

r300: transform vs sin and cos input to [-PI,PI] range in NIR

The python generator is mostly copy-pasted from lima.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
Signed-off-by: Pavel Ondračka <pavel.ondracka@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14957>

r300: use nir lowering for sin and cos on R300 and R400

The nir approximation is a bit more precise so there is one more
instruction for the scalar version but if the shader actually uses
vector one or some other stuff like sin(x) followed by cos(x) we
save more.

This nir approximation importantly seems to have better precision
so this should also fix some piglits/dEQPs.

With my shader-db and faked R300:
total instructions in shared programs: 67751 -> 65978 (-2.62%)
instructions in affected programs: 8637 -> 6864 (-20.53%)
total temps in shared programs: 9191 -> 9137 (-0.59%)
temps in affected programs: 486 -> 432 (-11.11%)
total consts in shared programs: 45427 -> 45412 (-0.03%)
consts in affected programs: 856 -> 841 (-1.75%)
total lits in shared programs: 2317 -> 2346 (1.25%)
lits in affected programs: 69 -> 98 (42.03%)

Reviewed-by: Emma Anholt <emma@anholt.net>
Signed-off-by: Pavel Ondračka <pavel.ondracka@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14957>

radv/winsys: fix initializing debug/perftest options if multiple instances

Since the winsys uses refcount, options like RADV_DEBUG_ZERO_VRAM might
have not been initialized if the first instance wasn't created with
application info.

This fixes missing zerovram for vkd3d-proton.

Cc: 21.3 22.0 mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14978>

anv: Handle VK_DESCRIPTOR_POOL_CREATE_HOST_ONLY_BIT_VALVE for descriptor sets

Ensure that surface state does not get created for
VK_DESCRIPTOR_POOL_CREATE_HOST_ONLY_BIT_VALVE and we don't bind to the
binding table for descriptor sets created with this pool.

Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14683>

radv: move nir_opt_shrink_stores from radv_optimize_nir()

No need to call this pass in a loop.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14480>

nir/opt_shrink_vectors: Remove shrinking of store intrinsics data source

This is done via nir_opt_shrink_stores.

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14480>

nir: split nir_opt_shrink_stores from nir_opt_shrink_vectors

This patch moves the shrinking of store sources into
a separate pass.
The reasoning behind this is that this pass usually only
needs to be called once while nir_shrink_vectors might
better be called several times. This allows to move
the pass(es) out of the optimization loops.

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14480>

Revert "nir: make tgsi_varying_semantic_to_slot(..) public"

This reverts commit edbdd977232f0cfe8f26a44ddbda792780168f4a.

As etnaviv's TGSI compiler is gone we make that function
private again.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Acked-by: Lucas Stach <l.stach@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12889>

etnaviv: drop TGSI based backend compiler

I think it is time to switch to NIR per default and forget
about the past. There might be stuff that will explode in
different ways now but thats all fixable.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Acked-by: Lucas Stach <l.stach@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12889>

v3dv: Use the common command pool implementation

The only interesting information stored in v3dv_cmd_pool is the list of
command buffers and that's already tracked by vk_command_pool.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14917>

v3dv: Don't use vk_alloc/free2 for command buffers

The pool will always have a valid allocator.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14917>

anv: Use the common vk_command_pool

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14917>

anv: Don't call DestroyCommandBuffers in AllocateCommandBuffers

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14917>

anv: Drop anv_cmd_buffer::pool

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14917>