asahi: Extend batch tracking for explicit sync

Now that we have stub sync support in the submission API, we can
implement the batch tracking changes required to support an explicit
sync world. This excludes the UAPI-specific bits (command decoding and
status parsing).

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21620>

asahi: Make agx_flush_resource reallocate non-shareable resources

It's not legal to share a resource that isn't PIPE_BIND_SHARED, but
flush_resource needs to prepare a resource for potential sharing.
Let's allocate a new resource and blit it over when this happens.

See also:
https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13154

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21620>

asahi: Support importing sync objects on BO export

When a BO is exported, implicit sync convention requires that writers
signal a fence on the object when complete. We already do this for BOs
that are *already* exported, but it is possible for a BO to be written
to, then exported for the first time.

Add a field to agx_bo to keep track of the current writer syncobj
handle. On first export, we use this to import it into the DMA-BUF.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21620>

asahi: Use a dynarray for writers

We don't want a writer hash table with persistent pointers to resources, because
the resources could be freed without the hash table being updated (even though
the underlying BO will not be freed until it's ready). To avoid the reference
count hell, do away with the pointer hash table and instead use a flat dynarray
for mapping BO (handles) to writer (batch indices).

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21620>

radv: Wait for shader uploads asynchronously.

This introduces tracking of the required semaphore values in pipelines,
which is then propagated to cmd_buffers on bind. Each queue also keeps
track the maximum count it has waited for, so that we can avoid the waiting
overhead once all the shaders are loaded and referenced.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16271>

radv: Upload shaders to invisible VRAM on small BAR systems.

Following PAL's implementation, this patch avoids allocating shader code
buffers in BAR and use SDMA to upload them to invisible VRAM
directly.

For some games like HZD, shaders can take as much as 400MB, which exceeds
the non-resizable BAR size (256MB) and cause inconsistent spilling
behavior. The kernel will normally move these to invisible VRAM on its own,
but there are a few cases that it does not reliably happen. This patch does
the moving explicitly in the driver to ensure predictable results.

In this patch, we upload the shaders synchronously; so the shader will be
ready as soon as vkCreate*Pipeline returns. A following patch will make
this asynchronous and don't block until we see a use of the pipeline.

As a side effect, when SQTT is used we now store the shaders on a cacheable
buffer which would speed up writing the trace to the disk.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16271>

radv: Introduce sdma_copy_buffer for GFX7+.

Helper salvaged from radeonsi (before SDMA removal).

This will be used for driver internal submissions to DMA shaders from GTT
to invisible VRAM.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16271>

radv: Use radeon_cmdbuf for sdma_copy_image.

For consistency with the sdma_copy_buffer helper that will be added next.

As a general justification, SDMA commands require little state tracking and
using radeon_cmdbuf makes it more suitable for driver internal use.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16271>

Revert "CI: Disable Windows runners"

This reverts commit 023d7e860eeb148d0d245bcc49f5b66b9aeb8363.

Reviewed-by: Giancarlo Devich <gdevich@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21932>

radv: Fix returning an expression from a void function

Fixes: d5de56bf ("radv: add RT shader args")
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21932>

vulkan/wsi: Fix Windows build

The headless path doesn't build for Windows (yet, anyway).

Fixes: d4a2c0fc ("vulkan/wsi: add a headless swapchain implementation/option")
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Giancarlo Devich <gdevich@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21932>

iris: implement TES distribution mode WA 22012785325

Set TEDMODE_RR_STRICT when TEEnable is set.

Signed-off-by: Anuj Phogat <anuj.phogat@gmail.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21899>

anv: implement TES distribution mode WA 22012785325

Set TEDMODE_RR_STRICT when TEEnable is set.

Signed-off-by: Anuj Phogat <anuj.phogat@gmail.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21899>

asahi: replace copies of .clang-format with symlinks

Avoid all the issues of having to keep them in sync, and few-enough
people (read: probably no-one ever) will be working on the asahi driver
from a Windows machine, so symlinks can be relied upon, especially for
something optional like automatic code formatting.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21951>

radv/rt: Don't upload the prolog twice.

radv_shader_create already calls radv_shader_binary_upload.

Fixes: 4b92a532859 ("radv: add radv_create_rt_prolog()")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21945>

v3dv/ci: fix test name (`,Fail` is not part of the test name)

Fixes: a0bf0adade0b23f72d69 ("ci/broadcom: move rare failure to the flakes")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21954>

zink/ci: update the radv expectations

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21873>

zink/ci: increase the parallelism of zink-radv-vangogh-valve

It current takes 17 minutes to run this job, which is above the 15
minutes target for Mesa CI. Let's increase the parallelism to reduce
exec time to closer to 13 minutes.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21873>

radv/ci: reduce the parallelism for vkcts-vangogh

We are about to enable pre-merge testing for radv-zink on vangogh,
which would mean the steam decks would be used for the following jobs:

 * Mesa pre-merge CI:
  * zink: 3 (~12 minutes)
 * Mesa Post-merge CI:
   * vkcts: 4 (~30 minutes)
   * vkd3d: 1 (~5 minutes)
 * DXVK CI: 1 (takes ~4 hours)

This means we could have 9 jobs running at the same time on steam
decks, despite only having 6 available. By reducing the number of decks
allocated for VKCTS runs from 4 to 2, we get closer to the actual
availability, and since vkd3d is so short + DXVK CI runs so
infrequently, we should never have to wait for a deck for too long!

Unfortunately, with the change of parallelism, a known flake started
failing more consistently, so I added it to the flakes list.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21873>

meson: correct typo in comment

Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21868>

virgl: Lower binding start into buffer indices

This requires that we limit the number of max combinded SSBOs to 31,
otherwisewe shaders that use SSBO binding points with higher values
will break on the host.

Fixes CTS:
  KHR-GL43.shader_storage_buffer_object.basic-atomic-case1
  KHR-GL43.shader_storage_buffer_object.basic-atomic-case2
  KHR-GL43.shader_storage_buffer_object.advanced-indirectAddressing-case2
  KHR-GL43.shader_storage_buffer_object.advanced-usage-case1
  KHR-GL43.shader_storage_buffer_object.advanced-usage-sync
  KHR-GL43.shader_storage_buffer_object.advanced-matrix

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21586>

ntt: add option to lower SSBO bindings to buffer index

When a shader uses SSBOs in various shader stages, then we have to track
the binding locations in order to be able to properly bind these SSBOs.

Therefore add a flag that enables adding the start index of the bindings to
the SSBO index.

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21586>

virgl: Don't try to do re-alloc or readback by transfer for blob resources

Blob resources are mapped directly, no need to copy data around, and
in any case, neither the resource nor the transfer info will have an
IOV attached to it, so the transfer would result error out on the host
anyway.

In addition, blob resources should not use re-allocation.

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21586>

virgl: Enable AMD_vertex_shader_(layer|viewport_index) when host supports it

This increase the number of cases when the texture upload from buffer
can use the PBO upload code path.

v2: Fix logic combination (Corentin)

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21586>

anv: use vk_get_physical_device_features

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21754>

panvk: Use vk_get_physical_device_features

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21754>

vulkan: use vk_features for vk_device::enabled_features

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21754>

vulkan: add hepler for vkGetPhysicalDeviceFeatures2

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21754>

vulkan: delete trailing namespace

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21754>

vulkan: tidy up vk_physical_device_features

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21754>

vulkan: relocate rmv to its correct home

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21754>

radeonsi: monolithic TCS emit tessfactor in nir directly

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21437>

radeonsi: lower nir_load_ring_tess_factors_offset_amd

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21437>

ac/llvm,radeonsi: lower nir_load_ring_tess_factors_amd

No one implement this intrinsic in llvm, so remove the
llvm entry too.

This will be used in TCS nir tess factor write.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21437>

ac/nir: handle tess factor output missing case

tcs_tess_lvl_(in|out)_loc may be not set if user miss tess
factor output.

Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21437>

ac/nir: init tess factor location with IO remap

Radeonsi is going to use nir tess factor write, so need to
remap tess factor location.

RADV set tess factor driver location to be 0 and 1 in
get_linked_variable_location(). While radeonsi also set them
to be 0 and 1 in st->map_io aka. si_shader_io_get_unique_index_patch().

We could just set them to be 0 and 1 at the beginning of
ac_nir_lower_hs_outputs_to_mem(), but in order to keep the
location map at the same place, we still do this in
lower_hs_output_store().

Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21437>

ac/nir: tcs write tess factor support pass by reg

For radeonsi usage.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21437>

ac/nir: move store_var_components to common place

It will be shared by other nir lowering too.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21437>

panfrost: Don't round up Midgard polygon list BOs

Rounding up the polygon list BO can waste large amounts of memory. In a common
case I observed, it rounded up 11MB to 16MB, wasting 5MB. That adds up quickly
across processes, especially on the 2GB machines.

This only applies to Midgard. On Bifrost and newer, the driver does not
explicitly allocate this data structure. Cc stable because this rounding is
incorrect and the increase in RAM usage can cause real problems (especially
given how slow the shrinker is).

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21831>

glsl: allow 64-bit integer on RHS of shift

Fixes: 9ba9a7f854dd ("glsl: Add 64-bit integer support to some operations.")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6862

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21919>

aco: remove aco::rt_stack variable

Since we initialize scratch in the RT proglog,
there is no need for this variable anymore.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>

radv/rt: use prolog for raytracing shaders

Co-authored-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>

radv/rt: Add shader config combination/postprocessing utils

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>

radv: Emit RT shader VA user SGPR

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>

radv: compile rt_prolog

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>

radv: add radv_create_rt_prolog()

Co-authored-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>

aco: implement select_rt_prolog()

Co-authored-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>

aco: create hw_init_scratch() function for p_init_scratch lowering

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>

aco: implement load_ray_launch_{id|size}

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>

aco: move rt_dynamic_callable_stack_base_amd to VGPR

In future, we will use a VGPR arg for that between RT stages.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>

aco: don't set private_segment_buffer/scratch_offset on GFX9+

It is unused. Also don't initialize scratch in raytracing stages as it gets
initialized in the prolog shader.

Co-authored-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>

aco: add RT stage enums

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>

radv: add RT shader handling to radv_postprocess_config

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>

radv: add RT stages to radv_get_shader_name()

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>

radv: handle RT stages in radv_nir_shader_info_pass()

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>

radv: add RT shader args

Co-authored-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>

mesa: add gl_shader_stage_is_rt()

to indicate raytracing shader stages.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>

driconf: Work around incorrect GI discard/invalidate

Genshin Impact does a depth+stencil invalidate (or discard, not sure
which entrypoint they are using) and then proceeds to do draws with
depth test enabled.  For IMRs (or freedreno in sysmem mode) this is no
problem.  But for tilers that use this as a hint that they can skip the
z/s tile load, it is.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Mike Blumenkrantz <michael.blumenkrantz@gmail.com>

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21916>

driconf: Add ignore_discard_framebuffer option

Some apps use glDiscardFramebuffer()/glInvalidateFramebuffer() when they
only kidding.  Add a knob to disable that.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Mike Blumenkrantz <michael.blumenkrantz@gmail.com>

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21916>

mesa: Rework discard_framebuffer()

Decouple the logic turning API attachment enums into gl_buffer_index.
This will make it easier to re-use discard_attachments() internally (See
!21910)

Signed-off-by: Rob Clark <robdclark@chromium.org>
Mike Blumenkrantz <michael.blumenkrantz@gmail.com>

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21916>

aco_validate: allow for wave32 in p_dual_src_export_gfx11

Fixes RADV_PERFTEST=pswave32

Fixes: bb90d29660b ("aco: add p_dual_src_export_gfx11 for dual source blending on GFX11")
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21934>

intel: use generated helpers for Wa_1409433168/Wa_16011107343

HSD 1306463417 is a hardware defect.  The originating software
workaround for the issue is Wa_1409433168.  Convert all references to
the software workaround number, and use generated helpers instead of
GFX comparisons.

Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21914>

ci/turnip: Move some more of the 1.3.5 new xfails under links.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21915>

ci/turnip: Disable dEQP-VK.image.queue_transfer.* for now.

We're waiting on the upstream CTS fix to land.  Until then, quiet the spam
in #freedreno-ci to see if we have any other flakes we need to be managing
right now.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21915>

ci/turnip: Clear out stale xfails.

These were all fixed in the CTS, but missed from removal due to fractional runs.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21915>

turnip: Don't push inline uniform buffer contents outside constlen.

This would apparently lead to trying to load from this inline uniform
buffer in later submits.

Fixes: 37cde2c63483 ("tu: Rewrite inline uniform implementation")
Fixes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8219
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21915>

aux/trace: delete GALLIUM_TRACE_NIR log message

this is not only annoying, it breaks trace xml output

cc: mesa-stable

Acked-by: Simon Zeni <simon@bl4ckb0ne.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21609>

anv: Handle external objects allocation in Xe

External(imported or exported) objects needs to have vm_id set to 0.

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21885>

anv: Properly alloc buffers that will be promoted to framebuffer in Xe KMD

Xe KMD does a special caching handling for buffers that will be
scanout to display, so that is why it needs a flag set during
allocation.

Checking if VK_STRUCTURE_TYPE_WSI_MEMORY_ALLOCATE_INFO_MESA
is available in AllocateMemory() and marking the buffer as scanout.

All WSI code paths but one sets
VK_STRUCTURE_TYPE_WSI_MEMORY_ALLOCATE_INFO_MESA.
The only one that doesn't requires that WSI is initialize with
wsi_device_options.sw_device = true to be executed, what is not the
case for ANV.

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21885>

anv: Implement Xe version of anv_physical_device_get_parameters()

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21885>

zink: fix `final_hash` update in `zink_gfx_program_update`

The logic that updates `ctx->gfx_pipeline_state.final_hash` assumed that
the program is replaced. It is supposed to xor `final_hash` with the
hash first and then with the new hash however when the program is
updated it end up xor-ing the new hash twice so it does nothing.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Fixes: 15450d2c2e2 ("zink: incrementally hash all pipeline component hashes")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21925>

zink: Add driver name and API version to renderer name

Having driver name in the renderer will be useful to differentiate
between open source and proprietary drivers as they can have different
feature sets/quirks.

Vulkan API version is also added to the name to match up with ANGLE.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21922>

zink: fix copy box merging adjacency

these cases need to increase the box size, not just adjust the offset

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21921>

zink: fix copy box iteration when adding

need to use the iterator, not the base pointer

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21921>

aco, radv: Remove VS IO information from ACO.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Acked-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16805>

ac/llvm: Remove unused function ac_build_struct_tbuffer_load.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Acked-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16805>

radv: Remove VS inputs code from LLVM backend.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Acked-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16805>

aco: Rename visit_load_input to visit_load_fs_input.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Acked-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16805>

aco: Remove VS inputs from visit_load_input.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Acked-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16805>

radv: Apply swizzle and alpha adjust in radv_nir_lower_vs_inputs.

Deal with VS input related things in a single pass instead of
having two different passes.

Fossil DB stats on Rembrandt (GFX10.3):

Totals from 174 (0.13% of 134913) affected shaders:
VGPRs: 7736 -> 7520 (-2.79%)
CodeSize: 354004 -> 353604 (-0.11%); split: -0.17%, +0.06%
MaxWaves: 4196 -> 4248 (+1.24%)
Instrs: 65228 -> 65139 (-0.14%); split: -0.19%, +0.06%
Latency: 265823 -> 265728 (-0.04%); split: -0.12%, +0.08%
InvThroughput: 84629 -> 84644 (+0.02%); split: -0.08%, +0.10%
VClause: 1618 -> 1606 (-0.74%); split: -0.93%, +0.19%
SClause: 1382 -> 1379 (-0.22%); split: -0.36%, +0.14%
Copies: 5586 -> 5566 (-0.36%); split: -0.55%, +0.20%
PreSGPRs: 4994 -> 5037 (+0.86%); split: -0.10%, +0.96%
PreVGPRs: 4948 -> 4955 (+0.14%); split: -0.04%, +0.18%

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16805>

radv: Lower non-dynamic VS inputs in NIR.

Add a new RADV specific NIR pass which lowers VS input
loads to AMD specific buffer load instructions.
We do this because we want to remove the RADV specific
VS input handling from the shader compiler back-ends.

Fossil DB stats on Rembrandt (GFX10.3):

Totals from 32507 (24.09% of 134913) affected shaders:
VGPRs: 1245512 -> 1245344 (-0.01%); split: -0.35%, +0.34%
SpillSGPRs: 1068 -> 1102 (+3.18%)
CodeSize: 90333192 -> 90327232 (-0.01%); split: -0.07%, +0.06%
MaxWaves: 881816 -> 881388 (-0.05%); split: +0.23%, -0.28%
Instrs: 17264710 -> 17264562 (-0.00%); split: -0.09%, +0.09%
Latency: 87300501 -> 86586480 (-0.82%); split: -1.07%, +0.25%
InvThroughput: 13700046 -> 13685931 (-0.10%); split: -0.20%, +0.10%
VClause: 361520 -> 361301 (-0.06%); split: -1.32%, +1.26%
SClause: 441018 -> 441505 (+0.11%); split: -0.54%, +0.65%
Copies: 1371477 -> 1373838 (+0.17%); split: -0.57%, +0.75%
Branches: 496639 -> 496611 (-0.01%); split: -0.01%, +0.00%
PreSGPRs: 1122956 -> 1122663 (-0.03%); split: -0.09%, +0.06%
PreVGPRs: 976051 -> 995717 (+2.01%); split: -0.12%, +2.14%

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16805>

ac/llvm: Implement typed buffer load intrinsic.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Acked-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16805>

aco: Implement load_typed_buffer_amd.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16805>

nir: Add load_typed_buffer_amd intrinsic.

This new intrinsic maps to the MTBUF instruction format on AMD GPUs
and represents a typed buffer load in NIR.

Also add an unsigned upper bound for the new intrinsic.
Code for that ported from aco_instruction_selection_setup.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16805>

aco: Remove vtx_binding from MUBUF/MTBUF instructions.

We haven't measured any noteworthy perf improvement
from these, and they are difficult to port to NIR,
so remove them before the NIR based VS input lowering
in order to make it easier to bisect and analyze stats.

Fossil DB stats on Rembrandt (GFX10.3):

Totals from 21750 (16.12% of 134913) affected shaders:
VGPRs: 868512 -> 868664 (+0.02%); split: -0.00%, +0.02%
CodeSize: 64406804 -> 64397572 (-0.01%); split: -0.08%, +0.07%
MaxWaves: 567904 -> 567888 (-0.00%); split: +0.00%, -0.00%
Instrs: 12327212 -> 12324851 (-0.02%); split: -0.10%, +0.08%
Latency: 61367324 -> 61371204 (+0.01%); split: -0.04%, +0.05%
InvThroughput: 9687734 -> 9686000 (-0.02%); split: -0.03%, +0.01%
VClause: 248207 -> 303449 (+22.26%); split: -0.02%, +22.28%
SClause: 314942 -> 315564 (+0.20%); split: -0.09%, +0.29%
Copies: 921581 -> 921820 (+0.03%); split: -0.16%, +0.19%
Branches: 341964 -> 341967 (+0.00%); split: -0.00%, +0.00%

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16805>

radeonsi: correct an assertion if we get a display list with no vertex buffers

It's possible to get a display list with no vertex buffers if the linker
eliminates all VS inputs or if the list was built with glArrayElement with
no enabled attribs.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21860>

radeonsi: don't use fp16_rtz for FP formats in the compute blit

It doesn't change the test results.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21860>

radeonsi: don't convert to fp16 in the compute blit if not testing

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21860>

radeonsi: fix AMD_TEST=computeblit being rejected on gfx < 11

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21860>

radeonsi: don't print the base non-view texture format for AMD_TEST=computeblit

it's confusing in the output and it affects nothing

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21860>

radeonsi: add AMD_DEBUG=nowcstream to enable caching for stream_uploader

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21860>

radeonsi: rework MSAA resolve averaging to exploit instruction-level parallelism

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21860>

radeonsi: remove duplicated gfx11 check in si_msaa_resolve_blit_via_CB

The function really does that twice. You'll find the same code a few lines
above this change.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21860>

radeonsi: replace nonir,noir,noasm,preoptir options with new reworked options

New options depending on what you want to print:
- initnir  = initial NIR of shader CSOs
- nir      = final NIR of variants after all lowering
- initllvm = LLVM IR before optimizations
- llvm     = final LLVM IR
- asm      = asm

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21860>

radeonsi: dump shader stats only if dumping asm shaders

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21860>

radeonsi: remove unused TCS/TES SGPR fields

We stopped using them when we switched to ac_nir_lower_hs_outputs_to_mem.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21860>

radv: set RADEON_SURF_NO_TEXTURE flag in radv_get_surface_flags()

In vkCreateImage() if block comrpessed format and VK_IMAGE_TILING_LINEAR is
used, then the app crashes in vega gpu.

This is because addrlib does not support BC + linear as from function
ValidateSwModeParams(). From Marek Olšák the addrlib behaviour is correct.

In pal driver, flags.texture is not set in DetermineSurfaceFlags() function
if BC + linear. pal driver does it because it is expected that the
BC + linear image is only used as transfer resource.

This patch sets RADEON_SURF_NO_TEXTURE flag if usage is not
VK_IMAGE_USAGE_SAMPLED_BIT and and VK_IMAGE_USAGE_STORAGE_BIT.
flags.texture flag is not set if RADEON_SURF_NO_TEXTURE and this fixes
the crash.

v1: set NO_TEXTURE if not SAMPLED or STORAGE (Marek Olšák)

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21422>

amd/surface: add RADEON_SURF_NO_TEXTURE flag

Block compressed + linear format is not supported in addrlib. But these
surface can be used as transfer resource. RADEON_SURF_NO_TEXTURE flag
indicates not to set flags.texture flag in gfx9_compute_surface().

This will help to fix the vkCreateImage() crash where block
compressed + linear format image is requested.

v2: combine RADEON_SURF_NO_TEXTURE to below line (Marek Olšák)
v1: add RADEON_SURF_NO_TEXTURE flag (Marek Olšák)

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21422>

anv: break out of the loop when the first color attachment is found

Fixes: 2bd304bc ("anv: Skip the RT flush when doing depth-only rendering")
Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21903>

vbo: fix current attribs not updating gallium vertex elements

An initial workaround that helped discover this was written by:
Illia Polishchuk illia.a.polishchuk@globallogic.com

Closes: #8440
Fixes: a18b9d07 ("st/mesa: optimize uploading zero-stride vertex attribs")

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21805>