anv/apply_pipeline_layout: Use the new helpers for images

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv/apply_pipeline_layout: Use the new helpers for early lowering

This also means that some of the newly added helpers need to grow a bit
to support VK_DESCRIPTOR_TYPE_INLINE_UNIFORM_DATA_EXT.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv/apply_pipeline_layout: Rework the desc_addr_format helper

We're about to add a new helper which is more detailed.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv/apply_pipeline_layout: Refactor all our descriptor address builders

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv/apply_pipeline_layout: Apply dynamic offsets in load_ssbo_descriptor

This function has exactly two call sites.  The first is where we had
these calculations before.  The second only cares about the size of the
SSBO so all the extra code we emit will be dead.  However, NIR should
easily clean that up and this lets us consolidate things a bit better.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv: Zero out the last dword of UBO/SSBO descriptors in the shader

This way, NIR can constant fold it.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv: Rework the 64bit_bounded_global resource index format

Instead of packing the descriptor offset into the packed portion, use
that unused channel we have lying around.  This potentially allows for
larger descriptor sets.  We also re-arrange the components a bit to make
it more like the 64bit_bounded_global memory address format.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv: Use 64bit_global_32bit_offset for SSBOs

This has the advantage of giving us cheaper address calculations because
we can calculate in 32 bits first and then do a single 64x32 add.  It
also lets us delete a bunch of code for dealing with descriptor
dereferences (vulkan_resource_reindex, and friends) because our bindless
SSBO pointers are now vec4s regardless of whether or not we're doing
bounds checking.  This also unifies UBOs and SSBOs.  The one down-side
is that, in certain variable pointers cases, it may end up burning more
memory and/or increasing register pressure.  This seems like a worth-
while trade-off.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

nir: Add a new 64+32-bit address format

This is a global address format where you have a 64-bit base pointer and
a 32-bit offset.  It's intentionally identical to 64bit_bounded_global
except nir_lower_explicit_io does no bounds checking with it.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv/apply_pipeline_layout: Add some switch statements

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv/apply_pipeline_layout: Plumb through a UBO address format

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv/apply_pipeline_layout: Move bounds checking later for index/offset

Instead of doing the array check at the load_vulkan_resource_index
intrinsic, stuff it in the vec2 and handle it at load_vulkan_descriptor
time.  This allows the bounds check to take any re-index intrinsics into
account.  This only affects variablePointers + SSBOs + Gen7.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv/apply_pipeline_layout: Run DCE between the early and late passes

This allows us to ignore UBOs in the late code going forward.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv/apply_pipeline_layout: Lower UBO loads in the early pass

We're about to enable bindless UBOs via A64 memory access like we do for
SSBOs.  In order to prevent 100% of UBOs from hitting that path, we
enable them in the early lowering.  This way we'll still get binding
table-based UBO access for any non-bindless ones.  In particular, we
need this for UBO pushing to work.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv/apply_pipeline_layout: Rework the early pass index/offset helpers

Rewrite them all to work on an index/offset vec2 instead of some only
returning the index.  This means SSBO size handling is a tiny bit more
complicated but it will also mean we can use them for descriptor buffers
properly.

This also fixes a bug where we weren't bounds-checking re-index
intrinsics because we applied the bounds check at the tail of the
recursion and not at the beginning.

Fixes: 3cf78ec2bd "anv: Lower some SSBO operations in..."
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv/apply_pipeline_layout: Refactor descriptor chasing code

This makes things a bit more generic for use in the next commit.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv: Use nir_shader_instructions_pass in apply_pipeline_layout

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv: Use load_global_constant for shader constants

NIR can do a bit better job optimizing this version.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

intel/fs,rt: Add a predicate to load_global_const_block

This allows us to do bounds checked A64 block load without the it being
counted as control-flow by NIR.  This means that NIR optimizations like
CSE will be able to work on these the same as a regular load.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

ci/bare-metal: Restart a run on intermittent kernel lockups.

Since enabling SMP on db820c and cranking up how many tests we run, we've
been seeing lockups like this a couple of times a week.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9655>

freedreno/drm: Avoid unitialized timestamp in submit fail

Saw a flood of "waiting on invalid fence" with a completely bogus
looking fence # in a log of a rather strange low-memory crash.  Not
sure if it is coming from memory corruption in userspace, but if a
submit ioctl is failing due to failed allocation (or other reason)
we would get left with random stack garbage as the fence #.  Let's
not have that as a potential problem.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9638>

aco: use a single instruction for uadd32_sat() on GFX8

fossil-db (GFX8):
Totals from 8 (0.01% of 147787) affected shaders:
SGPRs: 352 -> 368 (+4.55%)
CodeSize: 49576 -> 48788 (-1.59%)
Instrs: 9487 -> 9318 (-1.78%)
Latency: 49935 -> 49607 (-0.66%)
InvThroughput: 138493 -> 137443 (-0.76%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9598>

aco: use uadd32_sat() helper for nir_op_uadd_sat

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9598>

aco: implement 64-bit VGPR {u,i}find_msb

This can be created by subgroupBallotFindMSB().

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4458
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9598>

ac/gpu_info: fix more non-coherent RB and GL2 combinations

It ignored non-harvested chips with a non-power-of-two memory bus.

Fixes: abed921ce71 - amd: add support for Navy Flounder

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9568>

zink: fix free of ralloced pointer

When we alloc with ralloc, we also need to free with it.

But let's take a step back here; we don't just need to use ralloc, we
also need to destroy all other screen-resources. So let's call the
destructor here instead.

Fixes: 2643f9ed284 ("zink: ralloc screen objects")
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9647>

zink: fix emulation of no mipfilter

This approach is taken from the Vulkan spec[1], where a robust maxLod
of 0.25 is proposed instead of 0.0.

This has the effect of allowing room for both minification and
magnification filters, yet still rounding down to the right miplevel in
the end.

[1]: https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkSamplerCreateInfo.html#_description

Fixes: 8d46e35d16e ("zink: introduce opengl over vulkan")
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9649>

aco: Delete superfluous tess and ESGS I/O code.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

radv/llvm: Delete superfluous tess and ESGS I/O code.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

radv/llvm: Only store TCS outputs where they are really needed.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

radv: Use new, NIR-based I/O lowering.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

radv: Reorder some NIR optimizations in preparation for the I/O changes.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

radv: Fill some tess shader info earlier.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

radv: Determine tcs_in_out_eq in radv_pipeline instead of the compiler.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

radv: Calculate tess patches and LDS use outside the backend compilers.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

radv: Save I/O usage data to both shader infos for merged stages.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

radv: Lower IO and set driver locations earlier.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

ac: Add NIR passes to lower ES->GS I/O to memory accesses.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

ac: Add NIR passes to lower VS->TCS->TES I/O to memory accesses.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

ac/llvm: Emit more efficient code for load_shared.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

ac/llvm: Add constant offset to load/store_shared.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

ac/llvm: Make sure to always emit integer comparison for nir_op_ieq.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

ac/llvm: Make shared loads/stores work correctly for non-CS stages.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

ac/llvm: Implement new Geometry Shader intrinsics.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

ac/llvm: Implement the new tessellation intrinsics.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

ac/llvm: Implement AMD-specific buffer load/store intrinsics.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

aco: Implement new Geometry Shader intrinsics.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

aco: Implement the new tessellation I/O related NIR intrinsics.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

aco: Implement new buffer load/store intrinsics.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

nir: Add AMD-specific Geometry Shader related intrinsics.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

nir: Add tessellation related AMD-specific intrinsics.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

nir: Add nir_opt_offsets to fold const adds into load/store offsets.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

nir: Add AMD-specific buffer load/store intrinsics.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

nir: Add default unsigned upper bound configuration.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

nir: Add unsigned upper bound for TCS load_invocation_id.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

nir: Shrink vectors for load_shared.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

nir: Fix unsigned upper bound of local_invocation_index for non-CS stages.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

nir: Add a few more algebraic optimizations to help address calculation.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

nir: Add nir_builder helper for I/O address offset calculations.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

nir: Add new nir_builder helpers for iadd with no_unsigned_wrap.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

nir: Don't update base in vectorize_loads()

The offset is already updated with consideration to the base above under
"/* update the offset */".

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

aco/tests: add test for waNsaCannotFollowWritelane

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9187>

aco: fix NSA following writelane

No fossil-db changes on GFX10.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Fixes: c353895c922 ("aco: use non-sequential addressing")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9187>

aco/tests: add test for NSAToVMEMBug

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9187>

aco: fix NSA MIMG followed by MUBUF/MTBUF

No fossil-db changes on GFX10.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Fixes: c353895c922 ("aco: use non-sequential addressing")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9187>

freedreno/isa: assert if field's range is out of bitset's range

Also, update outdated comment along the way.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9628>

ir3: match mova1 mnemonic when writing to A1

For MOV to A1 blob uses "mova1" mnemonic, which is mov.u16u16;
change s16 to u16 when creating MOV to A1 in order to match the blob.

Before, couldn't be parsed back:
 mov.s16s16 ha0.y, 0

After, could be parsed back and matches blob behaviour:
 mova1 a1.x, 0

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9628>

ir3/isa,parser: fix encoding and parsing of bindless s2en SAM

Before, decoding showed that there is an error:
 sam.base0 (f32)(xyzw)r0.x, r0.z, a1.x   ; no field 'HAS_SAMP',
  WARNING: unexpected bits[0:7] in #cat5-samp-s2en-bindless-a1: 0x1 vs 0x0

After:
 sam.base0 (f32)(xyzw)r0.x, r0.z, s#1, a1.x

Fixes textures on the ground in TauCeti Vulkan Technology Benchmark

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9628>

ci/vc4/v3d: run piglit testsuite against Xorg

This increases the coverage adding tests that require an X server to
run.

Update also the expected results, and skip some flake tests.

Reviewed-by: Andres Gomez <agomez@igalia.com>
Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9562>

ci/v3dv: add flaky test in the skip list

Reviewed-by: Andres Gomez <agomez@igalia.com>
Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9562>

ci/armXX: add libgl1-mesa-dev dependency

Waffle building requires libgl1-mesa-dev to add support for GLX. This
package is pulled automatically in arm64 container as a suggested
package, but no in armhf. Which means we end up having support for GLX
in waffle in arm64 but not in armhf.

Thus let's install explicitly this package to have support in both
cases.

Reviewed-by: Andres Gomez <agomez@igalia.com>
Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9562>

vc4: destroy renderonly object if present

When destroying the VC4 screen, check if renderonly exists before
freeing it.

This fixes for instance a segmentation fault caused when invoking
wflinfo.

Fixes: 187218395d7 ("renderonly: remove layering violations")
Reviewed-by: Andres Gomez <agomez@igalia.com>
Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9562>

v3dv: avoid some maybe-uninitialized warnings

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9640>

v3dv/descriptor_set: don't free individual set if not allowed

If we have a host_memory_base pointer it means that we are handling
the pool memory as a whole, so each set is a sub-slice of the memory
pool. In this case we can't just free the individual set. In other
words, VK_DESCRIPTOR_POOL_CREATE_FREE_DESCRIPTOR_SET_BIT is not set.

Note tha at that point we were able to sub-allocate an set, and we are
failing to sub-allocate the pool bo for the descripto bo. So
technically we could try to return that set to the pool (so undo the
change on host_memory_ptr before), that I assume was my intention when
(wrongly) calling vk_free there. But in practice, at that point we are
already on a out of descriptor pool situation, so in the end it
doesn't even matter.

This fixes a double free crash with the UE4 VehicleGame demo.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9640>

broadcom/compiler: use nir_lower_wrmasks to simplify TMU general stores

This pass splits writemaks with non-consecutive bits into multiple
store operations ensuring that each store only has consecutive
writemask bits set.

We can use this to simplify writemask handling in our backend removing
a loop solely intended to handle this case.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9619>

broadcom/compiler: use nir_opt_load_store_vectorize

This will make it so we pack consecutive scalar operations into a vector
operation, reducing the amount of load/store operations in the NIR program.
Our backend can handle vector load/stores, and doing so may be more efficient
since we don't need to setup individual load/stores all the time.

A pathological case is:
dEQP-VK.spirv_assembly.instruction.compute.opcopymemory.array
which goes from 862 instructions to only 573 by converting
all scalar SSBO load/store operations to vec4 operations.

total instructions in shared programs: 13752607 -> 13733627 (-0.14%)
instructions in affected programs: 367117 -> 348137 (-5.17%)
helped: 1168
HURT: 371
Instructions are helped.

total threads in shared programs: 412230 -> 412272 (0.01%)
threads in affected programs: 54 -> 96 (77.78%)
helped: 23
HURT: 2
Threads are helped.

total uniforms in shared programs: 3790248 -> 3784601 (-0.15%)
uniforms in affected programs: 57417 -> 51770 (-9.84%)
helped: 1420
HURT: 19
Uniforms are helped.

total max-temps in shared programs: 2322170 -> 2322714 (0.02%)
max-temps in affected programs: 14353 -> 14897 (3.79%)
helped: 185
HURT: 306
Max-temps are HURT.

total spills in shared programs: 5940 -> 6010 (1.18%)
spills in affected programs: 65 -> 135 (107.69%)
helped: 0
HURT: 11

total fills in shared programs: 13372 -> 13494 (0.91%)
fills in affected programs: 75 -> 197 (162.67%)
helped: 0
HURT: 11

total sfu-stalls in shared programs: 31505 -> 31521 (0.05%)
sfu-stalls in affected programs: 751 -> 767 (2.13%)
helped: 210
HURT: 246
Inconclusive result (value mean confidence interval includes 0).

total inst-and-stalls in shared programs: 13784112 -> 13765148 (-0.14%)
inst-and-stalls in affected programs: 360283 -> 341319 (-5.26%)
helped: 1125
HURT: 366
Inst-and-stalls are helped.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9619>

broadcom/compiler: fix end of tmu sequence detection

TMUWT always terminates a TMU sequence.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9619>

radv: extend the dirty bits to 64-bit

For future work.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9603>

turnip: set depth plane control zmode to A6XX_LATE_Z when sample mask is written

Otherwise, gl_SampleMask[] writes are ignored and the stencil test works like
if all samples were enabled.

Fixes: dEQP-VK.renderpass.suballocation.multisample.*s8*

Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Reviewed-by: Hyunjun Ko <zzoon@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9478>

radv: add notccompatcmask debug option

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9415>

radv: enable TC-compat CMASK on GFX10+

Untested on older chips. Should help MSAA games by 1-2%.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9415>

radv: initialize TC-compat CMASK images with the DCC clear code

0xff is for fast-clears without MSAA.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9415>

radv: only configure the CMASK tiling for TC-compat on GFX8

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9415>

radv: make sure FMASK is enabled for TC-compat CMASK

Otherwise it makes no sense at all.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9415>

radv: remove redundant check when enabling TC-compat CMASK

We already check that the image has CMASK.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9415>

radv: do not enable TC-compat CMASK if the image isn't readable by a shader

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9415>

radv: use common entrypoints for VK_KHR_copy_commands2

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9600>

compiler/glsl: call util_cpu_detect from glsl_type_singleton_init_or_ref

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Closes: #4393
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9457>

vulkan/util: call glsl_type_singleton_init_or_ref from vk_instance_init

v2: link libvulkan_util with libglsl so it can find the glsl singleton symbols.
v3: link with libcompiler instead of libglsl (Jason)

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Dylan Baker <dylan.c.baker@intel.com>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com> for the v3dv bits.
Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> for the turnip bits.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> for the radv bits.
Acked-by: Dave Airlie <airlied@redhat.com> for the lvp bits.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9457>

v3dv: fix stride in buffer copy

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9402>

v3dv: fix assertion in job_compute_frame_tiling

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9402>

anv: Use ASSERTED for results that are only used in asserts.

This gets rid of unused variable warnings for these results.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9634>

anv: Use unreachable() in anv_genX.

This gets rid of unused variable warnings on genX_thing, because
now the compiler will think that the unknown hardware generation
case is unreachable.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9634>

intel/compiler: Make room for maximum dest size in nir_emit_texture.

The maximum dest_size is 5, not 4.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9634>

intel/compiler: Use assume() instead of assert() for array bounds.

This should make both GCC and clang happy and make them believe that
the array bounds are not exceeded.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9634>

CI: Change LAVA job visibility

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9635>

zink: enable PIPE_CAP_INVALIDATE_BUFFER

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9546>

zink: invalidate resources on map when discarding range

we can dump the whole vulkan object here for this case, which ends up being
much neater

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9546>

zink: handle streamout buffer rebinds

this really just means to nuke the counter buffer, and this can be done
by using a special bind_history bit that can be unset when the buffer has
been rebound

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9546>

zink: set valid region for streamout buffers on bind

this catches rebinds and is more accurate about the state of the region

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9546>