zink: make zink_vk_query unref consistent

the no-check free seems suspicious

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21534>

zink: fix possible query destroy leak

if the starts array has been reset, then the counter will be inaccurate,
and some of the members will leak, so this needs to iterate over the capacity
of the array instead of the contents

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21534>

zink: fix zink_query_start initialization

not sure if this was a bug, but it would have fully zeroed the struct
and prevented previous zink_vk_query members from being unrefed

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21534>

zink: rewrite zink_query_start struct

making the non-pool members easier to zero

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21534>

zink: reset queries on the promoted cmdbuf when possible

this avoids splitting renderpasses just to reset queries

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21534>

zink: remove suspended queries from list before resuming

this avoids double-starting primgen queries from recursion when
needs_rast_discard_workaround is true

cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21534>

zink: move zink_batch_no_rp call for query reset

this could otherwise pointlessly terminate a renderpass

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21534>

mesa/shaderapi: fix path memory leak

For instance, with "piglit/arb_shading_language_include-api -auto -fbo" or
"piglit/shader_runner tests/spec/arb_shading_language_include/execution/replacement.shader_test -auto -fbo":
Direct leak of 66 byte(s) in 6 object(s) allocated from:
    #0 0x7fa4b59050 in __interceptor_strdup (/usr/lib64/libasan.so.6+0x59050)
    #1 0x7f9a098fe0 in validate_and_tokenise_sh_incl ../src/mesa/main/shaderapi.c:3383
    #2 0x7f9a0a43e8 in _mesa_NamedStringARB ../src/mesa/main/shaderapi.c:3547

Signed-off-by: Patrick Lerda <patrick9876@free.fr>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21553>

d3d12: Don't loop in update_draw_indirect_with_sysvals

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21567>

d3d12: Create varying structures as necessary, reference them

This changes instances of d3d12_varying_info to d3d12_varying_info*,
significantly reducing the size of the d3d12_shader_key,
d3d12_gs_variant_key, and d3d12_tcs_variant_key.

Associated changes to key fill, compare, hashing, and gs and tcs variant
maps significantly reduce the amount of time spent clearing and
comparing memory.

The biggest win here is not having to re-zero _or_ re-fill varyings in
d3d12_fill_shader_key, validate_geometry_shader_variant, and
validate_tess_ctrl_shader_variant.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21527>

d3d12: Don't memcmp gs/tcs keys

This is unnecessary and incomplete. Not only is it covered by the hash
map lookup, but also does not consider the other key members.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21527>

d3d12: Don't unnecessarily zero out gs/tcs keys

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21527>

d3d12: Track max varying slot, set and compare less bytes

Often, the full range of available slots don't need to participate in
the comparison or be zeroed out.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21527>

d3d12: Don't recompute has_flat_varyings or missing_dual_src_outputs

Per shader selection, these two flags can be expensive to compute.
Instead, recompute their values as neccessary, and save them in the
context.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21527>

anv: Skip BTI RT flush if we're doing an op that doesn't use render targets.

rt_flushes emitted on zink sauer.trace --loop=500 -2.02118% +/- 1.15992% (n=8).

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21508>

anv: Skip the RT flush when doing depth-only rendering.

The spec citation says it's just for when the RT write message BTI might
point to a different RT, and if we don't have any color attachments then
we won't have one of those at all.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21508>

aux/tc: only call tc_parse_draw() when parsing renderpass info

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21533>

aux/tc: track whether queries have been terminated in a renderpass

on tilers it's important to know whether a query is ended mid-renderpass
so that the query begin can occur inside/outside of the renderpass

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21533>

panfrost: Use NIR scoped barriers instead of memory barriers

Now both GLSL and SPIR-V will produce the scoped barriers, so no
need to handle the old ones.

Control barriers are still present in some cases, so keep that
for now.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3339>

pan/midgard: Handle nir_intrinsic_scoped_barrier in Midgard compiler

Behave the same as the existing more specific barrier intrinsics.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3339>

pan/compiler: Fix handling of nir_intrinsic_scoped_barrier

Only emit anything if this is a control barrier, i.e. contains an
execution scope.  Also change the assertion to look at that scope
instead of the memory scope.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3339>

intel/compiler: Mark various memory barriers intrinsics unreachable

Now that both SPIR-V and GLSL are using scoped barriers, we can stop
handling the specialized ones.

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3339>

glsl: Implement use_scoped_barrier option for lowering memory barriers

When the option is enabled, lower memory barriers to the
unified nir_intrinsic_scoped_barrier.

The translation of the following is based on
https://www.khronos.org/registry/OpenGL/extensions/ARB/ARB_gl_spirv.txt

- memoryBarrier()
- memoryBarrierBuffer()
- memoryBarrierImage()
- memoryBarrierShared()
- groupMemoryBarrier()

Also use scoped barrier for the memory counterparts of the GLSL
(control) barrier() when the option is enabled.  The execution
part of a (control) barrier() remains using the old intrinsic.

For memoryBarrierAtomicCounter() there's no corresponding
nir_var_atomic_counter mode. Since atomic counters are lowered
to SSBOs, use the nir_var_mem_ssbo mode in the scoped barrier
instead.

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Acked-by: Rob Clark <robclark@freedesktop.org>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3339>

gallivm: Fix handling of nir_intrinsic_scoped_barrier

A scoped barrier only has a control barrier behavior if there's
an execution scope.

Fixes: 13d900de0dcf ("llvmpipe: set nir_shader_compiler_options::use_scoped_barrier")
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3339>

microsoft/compiler: Handle scoped barrier in Tess splitting

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3339>

nir: Support use_scoped_barrier in nir_lower_atomics_to_ssbo

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Acked-by: Rob Clark <robclark@freedesktop.org>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3339>

Revert "freedreno: Account for multi-draw in num_draws"

autotune already divides draw-cost by # of draws, but only increments
the draw-cost once per multi-draw.  We could either _also_ account for
draw-cost by multiply by # of draws for treat multi-draw as a single
draw.  The latter saves an integer multiply per draw.

Fixes a performance regression triggered by transition from GMEM to
sysmem rendering.

This reverts commit 6bfee9e6697e286c82f36a15d0e8a7c1ec7d2a02.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21543>

ci/crocus: Mark unvanquished as flaky.

It's started rendering something different again, with a similar sort of
bad rendering to what's linked in the bug report (though this time it's a
'P' that became a white square).  Commit range 65b62db0..964323fe has
nothing particularly likely in it, so I expect this is some sort of cache
flushing fail or something.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21565>

ci/crocus: Add new tess xfails and a link to the regression bug report.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21565>

zink/ci: skip one more test that timeout with RADV

It usually takes more than 60s to run.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21564>

ci/zink: move threading tests to flakes

We have some failed CI jobs where some of those tests randomly pass.

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21561>

docs: add pipeline library support for tu

Acked-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21560>

Revert "venus: temporarily redirect VkDrmFormatModifierPropertiesListEXT to "2" variant"

No longer need to redirect, since the renderer should use the fixed
decoder for VkDrmFormatModifierPropertiesListEXT::drmFormatModifierCount
by now.

This reverts commit 525b8c582f2e8e3e8f12589c1d384d683fee5dc4.

Signed-off-by: Ryan Neph <ryanneph@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21395>

venus: update venus-protocol headers to fix WA1

Follow-up to previous commit, this time to fix encoding/decoding for
VkDrmFormatModifierPropertiesListEXT::drmFormatModifierCount. Fixes a
workaround (WA1) in the venus-protocol.

Signed-off-by: Ryan Neph <ryanneph@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21395>

nir/lower_blend: Don't dereference null

If a dual source blend colour is never written, src1 will be null and it will be
invalid to dereference it. src1 is dereferenced both for the f2fN instruction
but also if a dual blend factor is used... even if the latter isn't strictly
valid, segfaulting in the NIR pass seems a lot meaner than blending with zero.

The referenced commit hosed Asahi, causing anything that used blending to crash.
Panfrost is unaffected since it always supplies a dual colour due to our crude
construction of blend shaders.

Fixes: 83130165437 ("nir/lower_blend: Consume dual stores")
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21544>

aco: always update orig_names in get_reg_phi()

No idea why this wasn't done if pc.first was a renamed temporary.

Fixes navi10 RA validation error with
dEQP-VK.binding_model.descriptor_buffer.multiple.graphics_geom_buffers1_sets3_imm_samplers

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8349
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21501>

radv: split linker script for android since it requires different symbols

Fixes: 4956f6d0bf477568731f ("radv: Add Android module info to linker script.")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8338
Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21518>

meson: Use sse2_arg and sse2_args to replace usage of c and c_sse2_args

And now c_sse2_arg and c_sse2_args are remvoed

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21375>

meson: Split sse2_arg and sse2_args out of c_cpp_args

This is used to replace c_sse2_arg and c_sse2_args in latter commits

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21375>

meson: When sse2 enabled, both c and cpp using sse2 options

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21375>

vulkan/wsi/x11: make 4 image minimum for xwayland driver-specific

this avoids adding extra frames of latency to drivers that don't need
it

Acked-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21447>

vulkan/wsi: switch to using an options struct for last param

this makes adding values easier since the drivers won't need to be updated

Acked-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21447>

aco: use and swizzle mask in dpp quad perm

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21412>

aco/gfx11: use dpp_row_xmask and dpp_row_share

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21412>

aco: use v_permlane(x)16_b32 for masked swizzle

Should be cheaper than ds_swizzle.

Totals from 8 (0.01% of 134913) affected shaders:
CodeSize: 16316 -> 16388 (+0.44%)
Instrs: 3088 -> 3086 (-0.06%)
Latency: 49558 -> 49508 (-0.10%)
InvThroughput: 9180 -> 9198 (+0.20%)
Copies: 376 -> 384 (+2.13%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21412>

amd: d16 uses rtz conversion for 32bit float

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21404>

amd: don't use d16 for integer loads

D16 saturates to min/max instead of just truncating.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21404>

nir: change 16bit image dest folding option to per type

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21404>

radv: enable primitiveUnderestimation on GFX9+

It's passing dEQP-VK.rasterization.conservative.underestimate.* on
NAVI21.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21459>

radv: set MSAA_NUM_SAMPLES to 0 for underestimate rasterization

Based on PAL.

Fixes
dEQP-VK.rasterization.conservative.underestimate.samples_*.triangles.normal.test.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21459>

radv: stop setting ENABLE_POSTZ_OVERRASTERIZATION to 1

According to PAL this isn't set.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21459>

radv: cleanup radv_emit_{conservative,msaa}_state() functions

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21459>

lavapipe: EXT_image_sliced_view_of_3d

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21514>

vulkan/runtime: store parameters of VK_EXT_sliced_view_of_3d

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21514>

radv/sqtt: Use code buffer from radv_shader directly instead of copying.

The reference-counted radv_shader always outlives the pipeline, so we can
use this buffer directly when dumping code objects to the trace.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21513>

radv: Keep shader code ptr in a separately allocated buffer.

RGP traces need a dump of shader code in order to display ISA and
instruction trace. Previously, this was read back from GPU at trace
creation time. However, for future changes that implements upload shader
to invisible VRAM, the upload destination will be a temporary staging
buffer and will be only accessible during shader creation.

To allow dumping in such cases, copy the shader code to a separate buffer
at creation time, if thread tracing is enabled.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21513>

zink: allow direct memory mapping for any COHERENT+CACHED buffer

some drivers may provide this in heaps that get used by non-staging resources,
so avoid extra copies in that case

unlike the previous attempt at this optimization, this utilizes the screen-based
context for thread-safe transfers, which should avoid races/crashes

fix #8171

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21452>

zink: add locking for zink_screen::copy_context and defer creation

the copy context isn't always used, so this allows its creation to
be deferred and potentially save a bunch of memory

also add locking for multi-context thread safety

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21452>

zink: avoid adding ubo/ssbo bindings multiple times for different bitsizes

these are valid variables, but the descriptor binding needs to be unique

cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21522>

zink: simplify/rework image typing in ntv

the array approach was broken if a shader contained both bindless
and non-bindless resources, whereas a hash table is simpler and can
handle both images and samplers

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21522>

zink: delete unused emit_image param in ntv

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21522>

zink: fix bindless texture barrier generation

whenever I redid barriers I forgot to handle bindless textures,
which meant they weren't getting barriers added

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21522>

zink: rework descriptor unbind params to use is_compute directly

much simpler

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21522>

zink: fix shader read access removal for barrier generation

barrier access is based on total binds per gfx/compute, not per stage

cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21522>

zink: delete dead uniform variables

this just obfuscate nir, so delete them now

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21522>

agx: Lower sampler LOD bias

G13 does not support sampler descriptor LOD biasing, so this needs to be lowered
to shader code for APIs that require this functionality. Add an option to do
this lowering while doing our other backend texture lowerings. This generates
lod_bias_agx texture instructions which the driver is expected to lower
according to its binding model.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21276>

asahi: Lower lod_bias_agx to uniform registers

Track the LOD bias of samplers and upload them at draw time to uniform
registers. This could be optimized in the future.

Vulkan will probably want to pull from a descriptor set instead.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21276>

nir: Add nir_texop_lod_bias_agx

Add a new texture opcode that returns the LOD bias of the sampler. This will be
used on AGX to lower sampler LOD bias to txb and friends. This needs to be a
texture op (and not a new intrinsic) to handle both bindless and bindful
samplers across GL and Vulkan in a uniform way.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21276>

ac/llvm,radeonsi: lower fbfetch in abi

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21436>

radeonsi: add num_component param to load_internal_binding

Prepare for different component number, ie. 8 when image desc.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21436>

ac/llvm,radeonsi: lower nir_load_barycentric_at_sample in abi

RADV already did this in radv_lower_fs_intrinsics().

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21436>

nir: lower to fragment_mask_fetch/load_amd with EQAA correctly

Fixes: 194add2c232 ("nir: lower image add lower_to_fragment_mask_load_amd option")
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Signed-off-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21436>

nir/lower_blend: Consume dual stores

Now that we're working on lowered I/O, passing in the dual source blend colour
via a sideband doesn't make any sense. The primary source blend colours are
implicitly passed in as the sources of store_output intrinsics; likewise, we
should get dual source blend colours from their respective stores. And since
dual colours are only needed by blending, we can delete the stores as we go.
That means nir_lower_blend now provides an all-in-one software lowering of dual
source blending with no driver support needed! It even works for 8 dual-src
render targets, but I don't have a use case for that.

The only tricky bit here is making sure we are robust against different orders
of store_output within the exit block. In particular, if we naively lower

   x = ...
   primary color = x
   y = ...
   dual color = y

we end up emitting uses of y before it has been defined, something like

   x = ...
   primary color = blend(x, y)
   y = ...

Instead, we remove dual stores and sink blend stores to the bottom of the block,
so we end up with the correct

   x = ...
   y = ...
   primary color = blend(x, y)

lower_io_to_temporaries ensures that the stores will be in the same (exit)
block, so we don't need to sink further than that ourselves.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21426>

panfrost: Use proper locations in blend shaders

Rather than always blending to FRAG_RESULT_DATA0. This removes silly special
cases in the compiler.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21426>

intel/rt: Fix L3 bank performance bottlenecks due to SW stack stride alignment.

Power-of-two SW stack sizes are prone to causing collisions in the
hashing function used by the L3 to map memory addresses to banks,
which can cause stack accesses from most DSSes to bottleneck on a
single L3 bank.  Fix it by padding the SW stack stride by a single
cacheline if it was a power of two.  This has been reported by Felix
DeGrood to improve Quake2 RTX performance by ~30% on DG2-512 in
combination with other RT patches Lionel Landwerlin has been working
on.

Many thanks to Felix DeGrood for doing much of the legwork and
providing several iterations of Q2RTX performance counter dumps which
eventually prompted me to consider the hash collision theory and
motivated this patch, and for providing additional performance counter
dumps confirming that there is no longer an appreciable imbalance in
traffic across L3 banks after this change.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21461>

docs: Fix formatting for RMV tracing docs

Fixes: e1cbff22 ("docs: Add short documentation about RMV tracing variables")
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21502>

meson: print c_cpp_args

Acked-by: Dylan Baker <dylan@pnwbakers.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21539>

radv/rt: Pre shift cull_mask

This removes the need for masking the instance mask.

Totals from 14 (14.43% of 97) affected shaders:
CodeSize: 378696 -> 378308 (-0.10%); split: -0.12%, +0.02%
Instrs: 70854 -> 70855 (+0.00%); split: -0.02%, +0.02%
Latency: 1651235 -> 1651215 (-0.00%); split: -0.00%, +0.00%
InvThroughput: 336290 -> 336285 (-0.00%); split: -0.00%, +0.00%
Copies: 9915 -> 9923 (+0.08%); split: -0.03%, +0.11%
PreSGPRs: 890 -> 896 (+0.67%)

 PERCENTAGE DELTAS Shaders  CodeSize   Instrs   Latency  InvThroughput   Copies   PreSGPRs
 q2rtx-pipe        48        -0.02%    -0.02%    -0.00%      -0.00%      -0.03%      .
 q2rtx_1           49        -0.10%    +0.02%    +0.00%      +0.00%      +0.14%    +0.31%
 -------------------------------------------------------------------------------------------
 All affected      14        -0.10%    +0.00%    -0.00%      -0.00%      +0.08%    +0.67%
 -------------------------------------------------------------------------------------------
 Total             97        -0.06%    +0.00%    -0.00%      -0.00%      +0.06%    +0.16%

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21530>

pvr: remove duplicate define

The same define appears a few lines above.

Signed-off-by: Frank Binns <frank.binns@imgtec.com>
Reviewed-by: Karmjit Mahil <Karmjit.Mahil@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21495>

pvr: stop restricting the compiler to the Sascha Willems triangle demo

Do this by removing the compatibility table and only using hard coded shaders
when present. The hard coded shaders, along with the hard coding framework
itself, can be dropped once the compiler is capable of compiling the hard coded
shaders. In the meantime we don't want to risk regressing things that we know
work because we temporarily can't test them.

This restriction is being dropped now as the new compiler framework has been
merged and we want to make use of it so it can be developed further.

Signed-off-by: Frank Binns <frank.binns@imgtec.com>
Reviewed-by: Karmjit Mahil <Karmjit.Mahil@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21495>

d3d12: Track up to 16 active context resource states locally in d3d12_bo

After 16 entries, we fall back to the previous logic that used a hash
map to link the resource's state per context.

Preventing hash map churn by cheaply tracking up to 16 context's worth
of states per resource significantly reduces CPU cost in
find_or_create_state_entry

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21528>

d3d12: Assign up to 16 simultaneously active contexts unique IDs

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21528>

d3d12: Move d3d12_context_state_table_entry to d3d12_resource_state.h

Also renamed desired_resource_state to d3d12_desired_resource_state,
since it's also in the header now.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21528>

tu/kgsl: Propagate tu_physical_device_init() errors

Should fix dEQP-VK.api.device_init.create_instance_device_intentional_alloc_fail.basic

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21394>

tu+meson: Re-work KMD selection

Now that turnip can support multiple kernel-mode drivers in a single
build, re-work the meson option to have a single list of KMDs, rather
than special options to enable kgsl for turnip or virtio for gallium.

It is temporarily a bit awkward as gallium does not yet support kgsl
and turnip does not yet support virtio.  But both of those are planned
or in-progress, so long term a single list is the most sensible option.

TODO freedreno/drm support to build with only virtio support.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21394>

turnip: Refactor device loading

1) Allow the two different entrypoints for drm vs non-drm (kgsl) to
   coexist.
2) Split the generic drm related device initialization from the msm
   specifics.  This will simplify adding support for additional drm
   based kernel mode drivers (ie. virtgpu)

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21394>

turnip: Handle kgsl vs drm specifics at runtime

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21394>

turnip: Move QueueWaitIdle entrypoint to kgsl

Handle in the same way as other kgsl specific entrypoints.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21394>

turnip: Allow knl backend specific entrypoints

Because kgsl sync primitives are not drm_syncobj, the kgsl kernel
support needs the ability to patch in it's own entrypoints related
to fences, etc.  The current entrypoint table magic using weak syms
won't work if we are building both kgsl and drm support into one
binary, so switch to runtime patching in the kgsl specific entry-
points.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21394>

turnip: Split out vfuncs for kernel interface

To allow support for multiple KMD's to coexist, split out vfuncs.  This
doesn't quite *yet* let us have kgsl and msm coexist, because of the
different vk entrypoints needed by the kgsl backend.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21394>

turnip: drm code-motion

Move a few functions around in prep for next commit.  Only code-motion.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21394>

turnip: Move things to prep for multi-kernel support

The end goal is a single build that supports kgsl (non-drm) as well
as msm and virtgpu (both drm).  First lets move around some files to
accomodate that.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21394>

vk/runtime: Allow enumerate and try_create_for_drm to coexist

For drivers that can support both drm and non-drm kernel mode drivers it
is useful to be able to provide both entrypoints.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21394>

ci/freedreno: add flaking KHR-GL45.buffer_storage.map_persistent_dispatch

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21531>

docs: update link to intel optimization reference manual

Seems this document has moved since last we updated this link. But
instead of chasing the exact CDN link, let's link to the document on
Intel's website. There's both a download-link there, as well as the
ability to read the document online.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21448>

ci: correct typo in name of linkcheck job

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21448>

intel: use c_see2_arg instead of explicit -msse2

This allows us to also inherit `-mfpmath=sse` added in previous commit.

Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21371>

intel: enable -mfpmath=sse on x86

It's not enabled by default until `-msse2` and -ffast-math is passed.
We pass only the `-msse2`. Let's align it with main `meson.build`.

See: https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html (-mfpmath).

Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21371>

r600/sfn: Fix minimum required registers

Don't count the local registers, just the arrays, because local
registers might be renamed during register allocation.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8100

Fixes: 9fd9f3cd10cf58549f850aeca9103d8cca5a5f81
   r600/sfn: Set minimum required registers based on array allocation

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21523>

radv/rt: Merge cull_mask and flags

Since cull_mask is only one byte, we can trivially store it in the same
register as the flags. This leaves us with a 2% performance gain in
Quake II RTX:

Totals from 7 (14.00% of 50) affected shaders:
VGPRs: 720 -> 688 (-4.44%)
CodeSize: 213052 -> 212980 (-0.03%); split: -0.05%, +0.02%
MaxWaves: 67 -> 70 (+4.48%)
Instrs: 39429 -> 39394 (-0.09%); split: -0.15%, +0.06%
Latency: 1096258 -> 1096943 (+0.06%); split: -0.05%, +0.11%
InvThroughput: 230661 -> 222963 (-3.34%); split: -3.42%, +0.08%
VClause: 1208 -> 1206 (-0.17%); split: -0.25%, +0.08%
Copies: 5321 -> 5269 (-0.98%); split: -1.22%, +0.24%
Branches: 1903 -> 1902 (-0.05%)
PreVGPRs: 650 -> 645 (-0.77%)

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21470>