lavapipe: Use more Vulkan NIR heleprs

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17644>

vulkan: Call gather_xfb_info in vk_spirv_to_nir

In particular, we now call it before running dead variables so we get
the XFB info even for things which are never written.  This fixes a 102
Vulkan CTS tests on ANV and probably turnip as well.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17644>

vulkan/nir: Don't remove dead XFB outputs

Fixes: 21b405fbbc53 ("vulkan: Add a vk_shader_module_to_nir() helper")
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17644>

vulkan: Call lower_clip_cull_distance_arrays in vk_spirv_to_nir

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17644>

iris/bufmgr: Add assert and TODO comment for future small BAR uapi

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17349>

anv/allocator: Add assert and TODO comment for future small BAR uapi

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17349>

iris: Set clear_color_unknown if the bo is not mappable

Rework:
 * Ken: Check bo for IRIS_MMAP_NONE rather than the global
   intel_vram_all_mappable

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17349>

iris/bufmgr: Set mmap_mode to IRIS_MMAP_NONE for lmem in small-BAR mode

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Suggested-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17349>

iris/bufmgr: Add all_vram_mappable which is currently always true

This can be false on systems where the PCI Base Address Register (BAR)
is too small for the amount of VRAM. Eventually the kernel will be
able to tell us that a system can't map all of VRAM, and
`all_vram_mappable` will then be false.

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17349>

iris/resource: Avoid mapping when not needed in iris_resource_init_aux_buf()

We might not be able to map all vram buffers in the future, so only
map the buffer when actually required.

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17349>

iris/resource: Assert that DG2 CCS buffers don't also try to set BO_ALLOC_SMEM

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17349>

intel/dev: Add intel_vram_all_mappable()

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17349>

intel/tools: Print unmappable region info in intel_dev_info

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17349>

intel/dev: Add vram.unmappable.size region info

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17349>

intel/dev: deal with i915 unallocated_size on smem

We cannot rely on unallocated_size on system memory for
VK_EXT_memory_budget.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 4aecfbf0f4ab ("intel/dev: Add devinfo::mem to store i915 regions information")
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17349>

ttn: set dest_type for TXQ

It was failing an assertion in tgsi_to_nir and NIR validation.

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17691>

ir3: Stop using nir_legalize_16bit_sampler_srcs.

nir_fold_16bit_tex_image's only_fold_all option ensures that there is never
a mix of bit sizes.

Closes https://gitlab.freedesktop.org/mesa/mesa/-/issues/6899

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16978>

nir/lower_mediump: Add an option to only fold if all tex sources can be folded.

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16978>

nir: Rewrite and merge 16bit tex folding pass with 16bit image folding pass.

Allow folding constants/undef sources by sharing more code with the image_store
16bit folding pass.

Allow more than one set of sources because RADV wants two, one for
G16 (ddx/ddy) and one for A16 (all other sources).

Allow folding cube sampling destination conversions on radeonsi/radv because
I think the limitation only applies to sources.

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16978>

ir3: Lower alu to scalar if nir_legalize_16bit_sampler_srcs made progress.

Fixes: 003327dd95b ("freedreno/ir3: Pass 16-bit sampler coordinates when possible.")
Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16978>

ir3: Only run 16bit tex NIR passes on a5xx+.

16bit types aren't yet supported on older hardware.

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16978>

zink: add env var to abort on device-lost if no reset callback is set

the alternative here is to just spin aimlessly until the process ooms,
which causes problems when trying to detect failures in cts caselists

a separate env var is used so that it can be exported without affecting
ZINK_DEBUG

Acked-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17525>

gallium/tests: Remove format desc null checks

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17490>

util/format: Remove format desc null checks

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17490>

pvr: Remove format desc null check

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17490>

virgl: Remove format desc null checks

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17490>

softpipe: Remove format desc null check

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17490>

radeonsi: Remove format desc null checks

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17490>

r600: Remove format desc null checks

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17490>

r300: Remove format desc null check

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17490>

panfrost: Remove format desc null check

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17490>

llvmpipe: Remove format desc null checks

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17490>

etnaviv: Remove format desc null check

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17490>

agx: Remove format desc null check

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17490>

util: Remove format desc null check

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17490>

tgsi: Remove format desc null check

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17490>

gallivm: Remove format desc null checks

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17490>

turnip: Remove format desc null assert

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17490>

v3dv: Remove format desc null asserts

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17490>

radv: Use desc->format

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17490>

radv: Remove format desc null checks

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17490>

util/format: Assert that formats are valid

It should be the responsibility of the driver to make sure, that "format" is a valid pipe_format.

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17490>

util/format: Use an explicit length for the descs

The script that generates the format tables does not set every pipe_format.
In practice, the length of the format tables is equal to PIPE_FORMAT_COUNT.
I just added the explicit size to future-proof it.

(If the largest valid format is not part of the format tables,
 there will be a mismatch between the array length and PIPE_FORMAT_COUNT)

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17490>

radv: only force 1x sample for Bresenham lines when pipeline draws lines

Otherwise, this would affect non-line draws. While we are at it,
adjust a comment.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6303
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17657>

ci/lava: Increase boot timeout

Empirically, a successful LAVA boot time should take less than 3
minutes.

LAVA itself is configured to attempt thrice to boot the device,
summing up to 9 minutes.

It is better to retry the boot than cancel the job and re-submit to
avoid the enqueue delay.

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17646>

ci/turnip: Add a bit of spilling-vs-ballot testing on a618.

The shared reg usage involved in the subgroup-related macros can cause
trouble for the spiller, and spilling may be implicated in CTS failures
with old versions of the subgroup tests, so let's make sure we get some
coverage.  It does seem to catch a couple of failures.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17642>

freedreno: Enable A619

Enable A619 as found in various SKUs of the SM Lagoon SoC, such as
SM6350 and SM7225.

Signed-off-by: Konrad Dybcio <konrad.dybcio@somainline.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17495>

util: Gate simple_mtx_assert_locked on !NDEBUG

..Instead of DEBUG so these work in debugoptimized builds.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Acked-by: Dylan Baker <dylan@pnwbakers.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17408>

util: Enable list_assert in debugoptimized builds

In debugoptimized builds, DEBUG is not set (and neither is NDEBUG). The
intention of NDEBUG is to disable assertions. As such, list assertions should be
gated on !NDEBUG as opposed to on DEBUG.

But assert() is already disabled in that case, so we don't need our own special
assert (Eric).

This would have caught an assertion failure (due to the wrong iterator used)
sooner for the Valhall compiler.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17408>

vulkan/wsi: return VK_SUBOPTIMAL_KHR for sw/x11 on window resize

the other codepaths all end up checking geometry in one way or another
in order to validate the extents, so add a check here to do the same

fixes #6893

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17638>

glsl: Remove optimize_swizzles.

It will get turned into SSA and copy-propagated in NIR, no need to walk
the IR collapsing it here.

iris shader-db results appear to be noise:

total instructions in shared programs: 8932195 -> 8932147 (<.01%)
instructions in affected programs: 537 -> 489 (-8.94%)
LOST:   12
GAINED: 11

lost/gained are simd32 switches in unigine, l4d2, portal2, asphalt9.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17613>

zink: invoke descriptor_program_deinit for programs on context destroy

this should make multi-context shutdown more stable

affects:
glx@glx-visuals-depth -pixmap
glx@glx-visuals-stencil

cc: mesa-stable

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17658>

glsl: Use the same NIR path for shared mem lowering as SPIRV does.

Now that we have no non-NIR drivers, we can retire the old code.  We just
need to pass the variable accesses through to it.

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17610>

radv: Enable task shader feature for NV_mesh_shader.

Still hidden behind RADV_PERFTEST=nv_ms but now advertises
task shader support too.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>

radv: Copy BO list to ACE internal CS.

This is necessary to make sure the ACE internal cmdbuf
can access the same memory as the GFX cmdbuf.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>

radv: Submit internal compute cmdbuf.

Use scheduled dependencies to create two submissions:
first we submit ACE then GFX.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>

radv: Workaround MEC taskmesh dispatch hang when count buffer has zero.

The DISPATCH_TASKMESH_INDIRECT_MULTI_ACE packet has a firmware bug,
it hangs the GPU when the draw count is zero.

This commit adds a workaround sequence using COND_EXEC packets
which make sure that this indirect packet is never executed when
the draw count is zero.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>

radv: Support task shaders in secondary cmd buffers.

Special consideration is needed to keep ACE and GFX in sync.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>

radv: Synchronization for task shaders.

Add a separate flush_bits field for tracking cache
flushes in the ACE internal cmdbuf.
In barriers and image transitions we add these flush bits to ACE.

Create a semaphore in the upload BO which makes it possible
for ACE to wait for GFX for the purpose of synchronization.
This is necessary when a barrier needs to block task shaders.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>

radv: Implement mesh shading draw calls with task shaders.

This implements NV_mesh_shader draw calls with task shaders.

- On the GFX side:
  DISPATCH_TASKMESH_GFX for all draws
- On the ACE side:
  DISPATCH_TASKMESH_DIRECT_ACE for direct draws
  DISPATCH_TASKMESH_INDIRECT_MULTI_ACE for indirect draws

Additionally, the NV_mesh_shader indirect BO layout is
incompatible with AMD HW, so we add a function that copies
that into a suitable layout.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>

radv: Introduce radv_before_taskmesh_draw.

This includes additional code that takes care of
handling the internal ACE cmdbuf.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>

radv: Flush descriptors and push constants for task shaders.

Task shaders are executed on the internal compute cmdbuf, so they
need special consideration.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>

radv: Create internal cmdbuf when a graphics pipeline needs compute.

This is mainly going to be used by task shaders, because
the HW implementation mismatches the API:

- In the API, task shaders are considered graphics shaders which
  are part of a graphics pipeline and the draws are submitted to
  a graphics queue.
- The HW requires the driver to dispatch task shaders on
  an async compute queue.

When a pipeline is bound that has a task shader, create a
driver-internal ACE (async compute engine) cmdbuf which
we are going to submit to an ACE queue.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>

radv: Add dispatch_initiator_task field to radv_device.

This is going to be used with task shader dispatches.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>

radv: Allow reusing pipeline compute state emit functions.

We are going to reuse them outside of radv_pipeline.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>

radv: Minor cleanup of radv_queue_submit_normal.

- Move the uses_perf_counters ternary expression out of
  the loop into a variable called cs_offset.

- Constify cmd_buffer_count.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>

radv: Move inline push constants to a new function.

This cleans up radv_flush_constants and also
the new function will be reused later.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>

radv: Refactor view index emit to use a per-stage function.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>

radv: Refactor radv_emit_inline_push_consts to work with radeon_cmdbuf.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>

radv: Refactor radv_emit_descriptor_pointers to work with radeon_cmdbuf.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>

radv: Refactor radv_emit_userdata_address to work with radeon_cmdbuf.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>

radv: Refactor some CP DMA functions to work with radeon_cmdbuf.

Allow emitting these packets without a radv_cmd_buffer object.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>

radv: Refactor predication for compute queues.

Initialize the inverted predication VA only when it is used
for the first time.

This is needed to get conditional rendering work correctly with
task shaders because the internal compute cmdbuf may not exist
yet when conditional rendering starts.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>

v3d/ci: update expected list

Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17660>

zink: scale number of shader cache-get threads

this should improve startup times for short tests

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17659>

zink: handle null samplerview/image rebinds more gracefully

fixes a crash in nine

cc: mesa-stable

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17656>

zink: move layout-setting for dynamic render attachments after no-op case

this avoids unnecessarily terminating a renderpass to insert unnecessary barriers

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>

zink: allow no-op renderpass updates in zink_batch_rp()

in some cases it becomes desirable to "maybe" stop and start the current
renderpass, such as when updates MAY result in layout changes for attachments

for such cases, avoid splitting the renderpass unless it actually needs to
be split

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>

zink: improve granularity of renderpass switching

this should ensure that (future) renderpass recalcs will never split
a renderpass unnecessarily

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>

zink: add a bits union for zink_render_pass_state comparisons

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>

zink: reuse rt layout functions to refactor zink_render_pass_attachment_get_barrier_info()

maybe slightly easier to read?

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>

zink: break out color rt layout calc

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>

zink: break out zs renderpass layout calc

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>

zink: break out renderpass attachment load op calc

no functional changes

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>

zink: split out rp_changed to be more granular for dynamic render

sometimes a state change MAY require a renderpass change, but this change
will not require splitting the current renderpass

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>

zink: add an assert to verify that deferred clears aren't added in renderpass

this would be very, very broken

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>

zink: remove redundant renderpass update check for clears

this should be handled already in the clear code

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>

zink: unflag ctx->rp_changed after dynamic render update

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>

zink: do dynamic render implicit clear updates on rp_changed

this should be flagged correctly now

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>

zink: flag renderpass change if renderpass clears change

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>

zink: don't flag renderpass change on depth write change as often

this is only "sometimes" required and only if dynamic render isn't being
used

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>

zink: don't try to start a renderpass before CmdClearAttachments

it should be impossible to reach this point if a renderpass is not active

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>

llvmpipe: ignore multisample state for alpha_to_one

this is more accurate and fixes usage with lavapipe

cc: mesa-stable

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17627>

vulkan/wsi: fix multiple acquires for sw without mit-shm

in this case, lying about having multiple images and then returning the
same image every time doesn't work, so use the busy flag
and return an available image when possible

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17590>

radv, aco: Don't lower 16bit isign.

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17440>

radv: Remove ineg from lower_bit_size_callback callback.

It's always lowered to isub.

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17440>

radv,aco: Don't lower and vectorize 16bit iabs.

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17440>

aco: remove unnecessary exp_cnt increments

update_counters() already does this.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17257>

aco: Initialize spill_ctx members in constructor.

Fix defects reported by Coverity Scan.

Uninitialized scalar field (UNINIT_CTOR)
uninit_member: Non-static class member sgpr_spill_slots is not
initialized in this constructor nor in any functions that it calls.
uninit_member: Non-static class member vgpr_spill_slots is not
initialized in this constructor nor in any functions that it calls.

Fixes: 7d340449086 ("aco: refactor VGPR spill/reload lowering")
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17583>

zink: use gfx_barrier for synchronization when binding buffer descriptors

this will preserve existing shader stages in the access mask and avoid desync
later on

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17639>