radv: Set DB_Z_INFO.NUM_SAMPLES to MSAA_EXPOSED_SAMPLES without Z/S

This case is a new addition in GFX11, and according to PAL, when no
depth/stencil attachment is bound, it must be set to the number of coverage
samples (the number of SampleMask bits - which is MSAA_EXPOSED_SAMPLES):

https://github.com/GPUOpen-Drivers/pal/blob/4640888b579bc9b0951c586b08a4552f71780d0d/src/core/hw/gfxip/gfx9/gfx9UniversalCmdBuffer.cpp#L6978

Without this change, the maximum of depth/stencil and color sample counts
is used, and if there are no depth/stencil or color attachments (target-
independent rasterization), the Depth Block assumes 1 coverage sample, and
thus Primitive Ordered Pixel Shading doesn't work correctly (and fails 4xAA
fragment shader interlock CTS tests), and occlusion queries don't count the
correct number of samples (according to the "Sample Counting" section of
the Vulkan specification, "the occlusion query sample counter increments by
one for each sample with a coverage value of 1...")

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22375>

zink: try to prune resources from barrier jit on fb unbind

if a resource has no binds remaining then it should not remain in
the queue for barrier updates on the next draw

cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22398>

winsys/amdgpu: Fix amdgpu_cs_query_reset_state2 error log

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Signed-off-by: André Almeida <andrealmeid@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22253>

radv: Implement vk.check_status

Implement check_status function so the driver can check if the GPU has
been reset by the application, and thus if it's still available.

AMDGPU_CTX_QUERY ioctls work by asking amdgpu if this context was the
cause of a previous GPU reset.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Signed-off-by: André Almeida <andrealmeid@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22253>

radv/ci: update expected failures with BONAIRE

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22391>

radv/ci: remove no longer existing tests for PITCAIRN

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22366>

radv/ci: update expected failures for PITCAIRN

These have been recently fixed.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22366>

radv: enable VK_EXT_graphics_pipeline_library by default

You won't get your money back!

It's been a very long time but everything should be working great now.

This replaces RADV_PERFTEST=gpl by RADV_DEBUG=nogpl to disable the
extension for debugging purposes.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22362>

vulkan/pipeline_cache: move vk_log on failed deserialization to vk_pipeline_cache_load()

Partially evicted or corrupted disk cache entries should not create
application cache log entries.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21967>

vulkan/pipeline_cache: add cache parameter to deserialize() function

This allows for secondary cache lookups during deserialization.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21967>

vulkan/pipeline_cache: use vk_pipeline_cache_create_and_insert_object() during vk_pipeline_cache_load()

This avoids an extra serialize() step for disk-cache insertion.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21967>

vulkan/pipeline_cache: implement vk_pipeline_cache_create_and_insert_object()

This function directly inserts the serialized data into the disk cache
before calling deserialize() and inserting into the pipeline cache.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21967>

vulkan/pipeline_cache: Don't re-insert disk-cache hits into disk-cache

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21967>

vulkan/pipeline_cache: remove vk_device from vk_pipeline_cache_object

It is not necessary to store the extra pointer.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21967>

ci/v3d: add flaking spec@ext_framebuffer_blit@fbo-sys-blit

Flakes on both RPI 3 and RPI 4.

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22377>

ac/llvm: Cover runtime 0 in GFX10 gs_alloc_req workaround.

Previously, the workaround only covered compile-time zero, but
this is insufficient and can cause GPU hangs in RadeonSI when
NGG culling is enabled.

Fix this by handling runtime zero in the workaround.

Cc: mesa-stable
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22370>

mesa: Enable NV_texture_barrier in GLES2+

The spec text is written against 3.2 but only requires framebuffer
objects and ES2.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19234>

intel/compiler: Use nir_dest_bit_size() for ballot bit size check

There's no guarantee that this is a SSA value.  Use the helper to handle
both SSA values and register correctly.  Otherwise we read trash when we
encounter a register and make bad decisions on types, possibly leading
to our destination being UQ typed when the VGRF is only 32-bit.

Fixes compilation with -Dintel-clc=enabled since 7f6491b76d51f35e7671
(nir: Combine if_uses with instruction uses) but the bug is much older
than that, circa 2017.  We were just getting lucky before.

Fixes: 069bf7c9078a ("i965/fs: Match destination type to size for ballot")
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22374>

nir/validate: Only walk uses once

Ostensibly faster.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22343>

nir/repair_ssa: Refactor some use handling

We can mostly unify the instr-use and if-use handling, which is a lot more
concise.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22343>

nir: Remove nir_if_rewrite_condition_ssa

Now unused.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22343>

dxil: Use nir_src_rewrite_ssa

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22343>

nir: Use nir_src_rewrite_ssa

Where sensible.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22343>

nir: Factor out nir_src_rewrite_ssa helper

Like nir_instr_rewrite_ssa but without the asserted extra argument. Works on ifs
too, now that we have a unified use list.

We do need to assert that the source has actually been inserted and has valid
use/def chains. Previously, asserting on the parent instruction accomplished
that indirectly. For the more general helper, we instead directly assert that
there exists a non-null parent, whatever it is.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22343>

nir: Reduce indirection

A source used by an if is necessarily the condition of that if.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22343>

dxil: Avoid list_length

Do O(1) check instead.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22343>

nir/validate: Don't treat if-uses specially

We don't use the tag anywhere, so don't bother with it.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Suggested-by: Faith Ekstrand <faith@gfxstrand.net>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22343>

nir: Remove 2nd argument from nir_before_src

We can now determine whether a nir_src is for an if without a sideband, so
simplify the function signature.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Suggested-by: Faith Ekstrand <faith@gfxstrand.net>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22343>

nir/opt_loop_unroll: Avoid list_length

It is O(N) but can often be replaced with something O(1).

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22343>

nir/opt_ray_queries: Don't use list_length

Expensive.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22343>

nir: Combine if_uses with instruction uses

Every nir_ssa_def is part of a chain of uses, implemented with doubly linked
lists.  That means each requires 2 * 64-bit = 16 bytes per def, which is
memory intensive. Together they require 32 bytes per def. Not cool.

To cut that memory use in half, we can combine the two linked lists into a
single use list that contains both regular instruction uses and if-uses. To do
this, we augment the nir_src with a boolean "is_if", and reimplement the
abstract if-uses operations on top of that list. That boolean should fit into
the padding already in nir_src so should not actually affect memory use, and in
the future we sneak it into the bottom bit of a pointer.

However, this creates a new inefficiency: now iterating over regular uses
separate from if-uses is (nominally) more expensive. It turns out virtually
every caller of nir_foreach_if_use(_safe) also calls nir_foreach_use(_safe)
immediately before, so we rewrite most of the callers to instead call a new
single `nir_foreach_use_including_if(_safe)` which predicates the logic based on
`src->is_if`. This should mitigate the performance difference.

There's a bit of churn, but this is largely a mechanical set of changes.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22343>

tu: Use vk_features2_to_features

Sanitizes properties returned through GetPhysicalDeviceFormatProperties2.

Bit-31 is not valid to return in the original
vkGetPhysicalDeviceFormatProperties{2,}. Sanitize the bit returned from the
internal to ensure invalid bits aren't return to the application.

Falls in line with the other vulkan drivers.

Based on original commit by Ryan Houdek.

Closes: #8733
Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22217>

anv,hasvk: Use vk_features2_to_features

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22217>

pvr: Use vk_features2_to_features

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22217>

lavapipe: Use vk_features2_to_features

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22217>

v3dv: Use vk_features2_to_features

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22217>

radv: Use vk_features2_to_features

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22217>

vulkan: Add common features2_to_features

Needed to correctly implement GetPhysicalDeviceFormatProperties2.

Cc stable so the turnip patch can get backported cleanly.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22217>

Revert "zink: don't trigger shader variants on pcp change if driver supports dynamic pcp"

This reverts commit 472fcf74e250a72053d837f7730fb991c1c3cb7e.

this is sort of right but then also sort of broken, going to rework a little

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22368>

radv: copy stages instead of serializing NIR for GPL with the RETAIN flag

When the driver gets a cache hit for the binary, we still have to
retain shaders because we can't know if the LTO pipeline will be a
cache hit as well.

Though, serializing the NIR is too costly and most of the libraries
took more than 10ms to be created, which isn't acceptable. To fix this,
keep track of the shaders stage info for libs with the RETAIN flag.

This might be replaced by NIR caching later if it's worth a try.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22327>

radv: create a helper for copying VkPipelineShaderStageCreateInfo

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22327>

radv/rt: stop storing unused hashes/identifiers

This is never used.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22327>

zink: fix sampler array collision in `nir_to_spirv`

`nir_to_spirv` has flat arrays to map driver_location to sampler
variables.

Now when bindless textures are used together with non binless textures
the sampler vars are in different descriptor sets and the binding can be
the same between different descriptor sets, this causes a collision in
arrays.

This patches chamges `nir_to_spirv` to also index the array by whether
the texture is bindless.

Fixes: bc202553e9b ("zink: implement bindless textures")
Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22358>

zink: add `descriptor_bindless_id` to `zink_shader_info`

This field stores the id of the descriptor set used for bindless
textures.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22358>

dzn: Remove xfail for test that passes (if run)

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22346>

dzn: Support >2K samplers with bindless

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22346>

dzn: Remove skips now that WARP is faster

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22346>

dzn: Don't leave deleted physical devices in the instance pdev list

Fixes: cfa260cd ("dzn: Use common physical device list/enumeration helpers")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22346>

dzn: Use GetResourceAllocationInfo3 for castable formats

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22346>

dzn: Clean up ABI helpers now that we require DirectX-Headers 606

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22346>

dzn: Early-out on no-op barriers

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22346>

zink: don't trigger shader variants on pcp change if driver supports dynamic pcp

this otherwise pointlessly creates and binds shader variants that do nothing

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22365>

zink: reuse copy_vars for generated tcs

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22364>

zink: reuse d3d12 variable copying to make passthrough gs more robust

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22364>

anv: reset query pools using blorp

Previously we used PC to set query data to 0 during
CmdResetQueryPool. This was slow when clearing large query pools.
Switching to blorp to clear pools is faster for large query pools.

Red Dead Redemption 2: +1.5% speedup

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22178>

anv: replace query flush before gpu copy by semaphore wait

All the flushes should already have happened, we just need CS to wait
for the operations to complete. Just use a MI_SEMAPHORE_WAIT to check
the availability bit is set.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Felix DeGrood <felix.j.degrood@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22178>

anv: pass steam output as argument for anv_dump_pipe_bits

Just if you need to change it at some point ;)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Felix DeGrood <felix.j.degrood@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22178>

anv/blorp: add flush reasons to RT flushes

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22178>

anv/blorp: implement anv_cmd_buffer_fill_area

Implemented function to fill an area at an address.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22178>

anv/blorp: support surf generation for addresses

Already have support for anv_buff. Extended to support addresses.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22178>

dzn: Enable VK_KHR_get_memory_requirements2

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22349>

dzn: Enable VK_KHR_bind_memory2

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22349>

radv: import retained NIR shaders later in the compilation process

This allows us to remove the intermediate NIR shader pointer.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22356>

radv: do not retain noop FS for libs when a cache hit happened

Determine if the graphics pipeline needs a noop FS later instead of
retaining it. This was also suboptimal.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22356>

radv: simplify a check when retaining NIR shaders

The RETAIN flag is only allowed with graphics libs.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22356>

radv: move the serialized NIR to radv_graphics_lib_pipeline

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22356>

radv: add a helper for retaining NIR shaders

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22356>

zink: don't access non_fs part of zink_shader from fs

Fixes: a6de15eff56 ("zink: add flags to `zink_gfx_program` and `zink_context`")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22347>

zink: more explicitly track/check rp optimizing per-context

if tc creation fails for whatever reason, rp optimizing must be
marked as disabled for that context to avoid erroneous assumptions
about rp operation

fixes #8787

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22319>

ac/llvm: remove some unused code replaced by nir

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22304>

ac/nir/ngg: fix store shared alignment

For stream!=0, this align_mul=4 is not true. Not observe any
problem yet, just for correctness.

Fixes: 60ac5dda82e ("ac: Add NIR lowering for NGG GS.")
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22304>

ac/nir/ngg: fix gs culling vertex liveness check for odd vertices

If vertex does not complete a primitive, it should not set the odd
flag which miss lead liveness check when culling is enabled.

For example, if odd flag is set regardless of complete flag, when
culling is enabled, 3 vertices of a triangle's init prim flag:
  [0x00 0x04 0x01]

then after culling, this triangle has been culled, their prim flag:
  [0x00 0x04 0x00]

the second vertex is miss treat as live because its odd flag (code
check prim_flag!=0 for liveness).

Fixes: 1bdeb961bd8 ("ac/nir/ngg: add gs culling")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8725
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22304>

radeonsi: fix max scrach lds size calculation when ngg

Fixes: 028d0590f85 ("radeonsi: replace llvm ngg vs/tes with nir lowering")
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22304>

asahi: Allow explicit non-LINEAR modifiers for scanout

The compositor is responsible for picking the right supported modifiers
for scanout. If we get no modifiers, we have to assume linear, but if we
do, just roll with it and don't attempt to force things.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>

asahi: Flip kmsro around to allocate on the GPU

Our display controller can handle arbitrary GPU imports, so there is no
reason to use dumb KMS buffers. Allocate everything on the GPU instead.

This also allows us to be lazy about mapping things to the KMS side, so
only clients that really want a KMS handle actually do that, which stops
us from ending up with a bunch of junk mapped to DCP (e.g. X11 clients
always request SCANOUT even under XWayland).

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>

asahi: Fix compressed ZS support

Depth/stencil formats are "not renderable" but do support compression.

I swear I already fixed this at some point and the commit must've fallen
through the cracks...

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>

asahi: Print reasons why compression is disabled

For resource debug. Found a regression in compressed depth this way.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>

asahi: Add resource debugging

I keep re-implementing this every time I look at resource-related
issues. Let's just make it official so we can turn it on with a flag
instead of having to add printfs every time ^^

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>

asahi: Add a helper macro for debug/error messages

This includes the program short name in the message, which is useful
when running entire desktop sessions with a single log to figure out who
is doing what.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>

asahi: Make BO import path failures more robust

These operations can fail for complex reasons through no fault of mesa,
so we should have proper runtime checks for them even in release builds.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>

asahi: Implement valid buffer range tracking

A common pattern is to allocate a vertex/etc buffer and write to it in
subsets. Some games interleave this with draw calls using the buffer.
This causes very expensive flushing for every draw call.

Fix this by tracking which range of a buffer has been written to, and
elide syncs when the range was previously uninitialized.

Fixes Source engine game performance and probably helps a bunch of
others.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>

asahi: Fix style nits

Found with a grep abomination which is probably too broken/silly to
actually implement in CI... but hey, at least it found some.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>

asahi: Locate low VA BOs correctly

These need the shader_base added to them.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>

asahi: Enable glthread

This helps a lot with FEX, since the GPU driver runs emulated (and only
64bit supports thunking).

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>

asahi: Make agx_alloc_staging() take a screen instead of a context

This makes it clear that it is thread-safe.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>

Revert "asahi: Advertise dual-source blending"

This reverts commit f4e2b2264663482f76681dbc0a8e9fc1574a3ce7.

This is broken until GL3 is enabled, possibly due to a core Mesa bug,
but it's a corner case not worth fixing.

Fixes Chromium.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>

agx: Make signal_pix instructions explicit

Rather than implicitly packing them with the sample_mask. Again, this is just
changing where they're emitted, no functional changes yet. Bug for bug
compatibility with the old behaviour.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>

agx: Rename writeout to wait_pix

This is the name applegpu is currently using, to capture the semantics of a
pixel fence. I'm not sure what Apple calls this but wait_pix is closer than
writeout for sure.

This commit just does the rename. It doesn't fix the broken semantics we've had,
this is to ease review and bisection.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>

agx: Tease apart some sample_mask packing magic

There's a second instruction here, and a second source in the first instruction.
applegpu has known about the encodings for a while but I never updated the
packing code. We will need to stop hardcoding this for multisampling support, as
preparation tease apart the magic pieces.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>

asahi: Clamp texture buffer sizes

Per the spec / freedreno. Fixes
arb_texture_buffer_object-texture-buffer-size-clamp

Fixes: 6b22a02f908 ("asahi,agx: Implement buffer textures with gnarly NIR")
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>

asahi: Dirty track depth bias uploads

Reduces how much we upload in SuperTuxKart.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>

asahi: Lower 1D to 2D

Khronos APIs require that we support mipmapping even for 1D textures. However,
it isn't clear if this is supported in the hardware, and how it would work even
if it is. But 1D textures are pretty useless, so we just lower 1D textures to 2D
textures instead of worrying about that.

Fixes piles of Piglits relating to 1D textures.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>

asahi: Implement null textures

Use the same silly workaround that Metal does, to fill in texture descriptors
when there's nothing bound in the interest of robust behaviour.

Fixes null pointer dereference in
arb_shading_language_420pack-active-sampler-conflict.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>

asahi: Honour sampler count

It may not be equal to the texture count. Prevents a regression from the next
commit.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>

agx: Don't overallocate registers

We need to account for the full vector lengths. Especially important once we
start restricting the reg file.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>

agx: Coalesce more collects

Try harder to coalesce collects, by trying to allocate collects only to regions
of the register file where we actually have a full vector worth of registers
free. If we already know that the vector will be blocked later, it's not a good
base register to pick since we'd be force to shuffle later. So, this tweak to
the collect coalescing heuristic lets us eliminate a pile of pointless copying.

shader-db results are excellent. Note that, although we use more registers,
none of the shaders tested had their thread count affected, likely because the
max HURT isn't too high and most of the scary % here is from using a few more
registers when the register pressure is already low. In the near future, that
property will become guaranteed thanks to live range splitting, too.

total instructions in shared programs: 1507337 -> 1500562 (-0.45%)
instructions in affected programs: 428137 -> 421362 (-1.58%)
helped: 2658
HURT: 167
helped stats (abs) min: 1.0 max: 34.0 x̄: 2.63 x̃: 2
helped stats (rel) min: 0.10% max: 25.00% x̄: 3.04% x̃: 2.14%
HURT stats (abs)   min: 1.0 max: 10.0 x̄: 1.24 x̃: 1
HURT stats (rel)   min: 0.20% max: 23.81% x̄: 3.90% x̃: 3.57%
95% mean confidence interval for instructions value: -2.49 -2.31
95% mean confidence interval for instructions %-change: -2.76% -2.51%
Instructions are helped.

total bytes in shared programs: 10333670 -> 10293172 (-0.39%)
bytes in affected programs: 2996682 -> 2956184 (-1.35%)
helped: 2660
HURT: 175
helped stats (abs) min: 2.0 max: 204.0 x̄: 15.70 x̃: 12
helped stats (rel) min: 0.08% max: 23.08% x̄: 2.64% x̃: 1.83%
HURT stats (abs)   min: 2.0 max: 60.0 x̄: 7.26 x̃: 6
HURT stats (rel)   min: 0.12% max: 22.39% x̄: 3.19% x̃: 2.78%
95% mean confidence interval for bytes value: -14.81 -13.76
95% mean confidence interval for bytes %-change: -2.39% -2.18%
Bytes are helped.

total halfregs in shared programs: 417284 -> 427363 (2.42%)
halfregs in affected programs: 49814 -> 59893 (20.23%)
helped: 95
HURT: 3018
helped stats (abs) min: 1.0 max: 8.0 x̄: 2.29 x̃: 2
helped stats (rel) min: 2.44% max: 28.57% x̄: 9.20% x̃: 6.06%
HURT stats (abs)   min: 1.0 max: 14.0 x̄: 3.41 x̃: 4
HURT stats (rel)   min: 2.08% max: 150.00% x̄: 36.54% x̃: 27.27%
95% mean confidence interval for halfregs value: 3.17 3.31
95% mean confidence interval for halfregs %-change: 34.05% 36.23%
Halfregs are HURT.

total threads in shared programs: 16465280 -> 16465280 (0.00%)
threads in affected programs: 0 -> 0
helped: 0
HURT: 0

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>

asahi: Set PIPE_CAP_LOAD_CONSTBUF

The CAP is a bit of a misnomer, what it really does is relax the alignment
requirements for UBO packing. It should work fine and save us some memory.
Noticed while debugging piglit fails.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>

asahi/decode: Print VDM barriers

Instead of just decoding silently.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>

asahi/decode: Remove agxdecode_dump_bo

Now that we have proper parsing this is more of a nuissance than not.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>

agx: Add helper for calculating occupancy

Add information about the relationship between program register usage and
program occupancy (the maximum number of threads that may execute concurrently
on a single shader core). This table is derived from studying the
maxTotalThreadsPerThreadgroup property in Metal while varying the register
usage, something I blogged about a few years back. It's probably not 100%
accurate and it hasn't been tested against hardware, but it matters "only" for
performance (not correctness) so I'm not super stressed about the details.

In the (near) future, RA will be able to make use of this information to know
exactly when it can use more registers without hurting performance. In the
present, it's just used for better shader-db statistics.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>