nir/opt_intrinsic: optimize quad vote

Optimizes a quadAll()/quadAny() pattern created by dxil-spirv:
https://github.com/HansKristian-Work/dxil-spirv/commit/7adc87d4deaba8078bcdef8dfbebdda0165cd7bc

dxil-spirv can't use clustered reductions because they are not guaranteed
to include helper invocations.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23621>

nir,aco: add INCLUDE_HELPERS index to reduce intrinsic

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23621>

aco: include helpers in emit_uniform_{reduce,scan}

Found by inspection.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23621>

nir/peephole_select: allow some invocation broadcast intrinsics

fossil-db (navi21):
Totals from 3 (0.00% of 133428) affected shaders:
Instrs: 2074 -> 2083 (+0.43%)
CodeSize: 10596 -> 10692 (+0.91%)
Latency: 75754 -> 75946 (+0.25%)
InvThroughput: 16900 -> 16975 (+0.44%)
Copies: 312 -> 309 (-0.96%)
Branches: 150 -> 132 (-12.00%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23621>

treewide: Remove unused builders

-Wunused-variables kicks in now that it can see through the init.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23860>

treewide: Use nir_builder_create more

perl -p0e 's/nir_builder_init\(&([^,]*), /\1 = nir_builder_create(/g' -i $(git grep -l nir_builder_init)

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23860>

nir: Use nir_builder_create

perl -p0e 's/nir_builder ([^;]*);\s*nir_builder_init\(&\1, /nir_builder \1 = nir_builder_create(/g' -i $(git grep -l nir_builder_init)

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23860>

nir: Add nir_builder_create returning nir_builder

More ergonomic.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23860>

nir/builder_opcodes: Remove nir_build_ prefixed helpers

This patch decreases the size of nir_builder_opcodes.h from 14292 loc to
13763 loc.

nir_build_ versions are still needed if the nir_ is a custom helper.
Intrinsics which need such a helper have to be added to
build_prefixed_intrinsics.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23858>

nir: Use nir_ instead of nir_build_ helpers

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23858>

vtn: Use nir_ instead of nir_build_ helpers

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23858>

freedreno: Use nir_ instead of nir_build_ helpers

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23858>

intel: Use nir_ instead of nir_build_ helpers

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23858>

microsoft: Use nir_ instead of nir_build_ helpers

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23858>

amd: Use nir_ instead of nir_build_ helpers

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23858>

nir/lower_blend: Optimize masked out RTs

While debugging KHR-GLES31.core.draw_buffers_indexed.color_masks, the noise from
piles of store_output(load_output) instructions got in the way. Optimize it out.

This does not fix the test, but if this case ever happened in a real app it
would improve performance. This is only load bearing on Asahi (and PanVK?),
since Panfrost wouldn't call nir_lower_blend at all in this case.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23836>

asahi: Use txf for background program

More straightforward (txf instead of tex, with integer coords). No discrernible
performance difference.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23836>

agx: Use nir_lower_frag_coord_to_pixel_coord

Instead of open-coding the logic.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23836>

pan/bi: Use lower_frag_coord_to_pixel_coord

Instead of vendoring the logic. This has a side benefit of letting NIR
optimize the generated code a bit.

total instructions in shared programs: 2687284 -> 2687281 (<.01%)
instructions in affected programs: 532 -> 529 (-0.56%)
helped: 3
HURT: 1
Inconclusive result (value mean confidence interval includes 0).

total cycles in shared programs: 140711.33 -> 140711.31 (<.01%)
cycles in affected programs: 2.53 -> 2.52 (-0.62%)
helped: 1
HURT: 0

total fma in shared programs: 22059.44 -> 22059.39 (<.01%)
fma in affected programs: 2.69 -> 2.64 (-1.74%)
helped: 3
HURT: 0

total cvt in shared programs: 14659.09 -> 14659.09 (0.00%)
cvt in affected programs: 1.56 -> 1.56 (0.00%)
helped: 1
HURT: 1

total quadwords in shared programs: 1455408 -> 1455416 (<.01%)
quadwords in affected programs: 128 -> 136 (6.25%)
helped: 0
HURT: 1

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23836>

nir: Add lower_frag_coord_to_pixel_coord pass

We've open coded this in a few backends.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23836>

nir: Add pixel_coord, frag_coord_zw intrinsics

On some architectures, gl_FragCoord.xy is available as an integer but
gl_FragCoord.zw requires interpolation. Add dedicated intrinsics so we can
lower it all in NIR.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23836>

ci: add a test-dozen-deqp flake

this times out regularly

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23876>

radv/amdgpu: workaround a kernel bug when replacing sparse mappings

AMDGPU has a bug when clearing mappings for BOs that are always valid
in VM with OP_REPLACE.

See https://lists.freedesktop.org/archives/amd-gfx/2023-June/094648.html

The current workaround is to re-use OP_MAP/OP_UNMAP until all stable
kernels have the fix.

This partially reverts "radv/winsys: update sparse mappings with
OP_REPLACE instead of OP_MAP/OP_UNMAP".

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23741>

radv/amdgpu: skip adding per VM BOs for sparse during CS BO list build

This should be similar but it will help for re-introducing
OP_MAP/ON_UNMAP instead of OP_REPLACE.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23741>

mapi: Now _glapi_get_dispatch_table_size always equal to sizeof(struct _glapi_table) / sizeof(void *)

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Corentin Noël <corentin.noel@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23822>

mapi: Fixes compile error with build option "-D shared-glapi=disabled"

Fixes: 398a8d43dc9 ("mapi: Delete dynamic stub generation.")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9245

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Corentin Noël <corentin.noel@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23822>

pvr: Change winsys flag defines to bitfields

Makes it easier to see which flags are set while debugging, and
prevent errors where the `BITFIELD_BIT()` is missing or using the
wrong flag for something.

Some fields have also been renamed to better fit with the naming
scheme around the code base.

Signed-off-by: Karmjit Mahil <Karmjit.Mahil@imgtec.com>
Reviewed-by: Frank Binns <frank.binns@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23765>

pvr: Fix missing BITFIELD_BIT for winsys frag job flag

On submission SPMSCRATCHBUFFER was acting like GET_VIS_RESULT +
DEPTH_BUFFER_PRESENT. This was causing hardware resets on barrier
stores as the depth buffer isn't actually present so the
store would be carried out to a NULL address.

Signed-off-by: Karmjit Mahil <Karmjit.Mahil@imgtec.com>
Reviewed-by: Frank Binns <frank.binns@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23765>

util: include "util/compiler.h" instead of "pipe/p_compiler.h"

And pipe/p_compiler.h are removed as it not used any more

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: David Heidelberg <david.heidelberg@collabora.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Acked-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23577>

util: Merge p_compiler.h into src/util/compiler.h

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: David Heidelberg <david.heidelberg@collabora.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23577>

treewide: style fixes after replace the usage of ubyte/ushort with uint8_t/uint16_t

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: David Heidelberg <david.heidelberg@collabora.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23577>

treewide: Replace the usage of ubyte/ushort with uint8_t/uint16_t

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: David Heidelberg <david.heidelberg@collabora.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Acked-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23577>

treewide: Replace the usage of TRUE/FALSE with true/false

this is a separate patch as it's won't affect the code style

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: David Heidelberg <david.heidelberg@collabora.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Acked-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23577>

treewide: style fixes after replace usage of boolean to bool

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: David Heidelberg <david.heidelberg@collabora.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23577>

treewide: replace usage of boolean to bool

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: David Heidelberg <david.heidelberg@collabora.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Acked-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23577>

llvmpipe: altivec.h inclusion in -std=c++98..11 causes bool to be redefined

So we carefully include altivec.h in lp_setup_tri.c

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: David Heidelberg <david.heidelberg@collabora.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23577>

gallium/draw: Replace the usage of ushort to uint16_t in files that can not found by tools

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: David Heidelberg <david.heidelberg@collabora.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23577>

hud: Use bool/true/false to replace boolean/TRUE/FALSE in hud/hud_context.c

auto tools can not find and replace
```
#define HUD_DEFAULT_VISIBILITY TRUE
```
So did the of this line manually,

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: David Heidelberg <david.heidelberg@collabora.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23577>

aux/indices: use stdint.h types

These generated sources uses older, less portable types such as ubyte,
ushort and uint. But we have stdint.h everywhere now, so let's use those
types instead.

To stay consistent, let's talk about UINT8 etc instead of UBYTE for the
entirety of the u_indices infrastructure.

Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23853>

etnaviv: linker: add fallback lookup to VARYING_SLOT_BFC[n]

There are valid NIR shaders pairs where the vertex shader has
a VARYING_SLOT_BFC0 shader_out and the corresponding framgent
shader has a VARYING_SLOT_COL0 shader_in.
So at link time if there is no matching VARYING_SLOT_BFC[n],
we must map VARYING_SLOT_BFC0[n] to VARYING_SLOT_COL[n].

Example shader pair from 'spec@!opengl 2.0@vertex-program-two-side back':

shader: MESA_SHADER_VERTEX
source_sha1: {0xf916f77d, 0xffa6ab5e, 0x160976a7, 0xb59fe59c, 0x92e8f3f6}
name: GLSL3
internal: false
stage: 0
next_stage: 4
inputs_read: 0
outputs_written: 0,13
subgroup_size: 1
bit_sizes_float: 0x20
bit_sizes_int: 0x20
first_ubo_is_default_ubo: true
flrp_lowered: true
inputs: 1
outputs: 2
uniforms: 0
decl_var shader_in INTERP_MODE_NONE vec4 gl_Vertex (VERT_ATTRIB_POS.xyzw, 0, 0)
decl_var shader_out INTERP_MODE_NONE vec4 gl_Position (VARYING_SLOT_POS.xyzw, 0, 0)
decl_var shader_out INTERP_MODE_NONE vec4 gl_BackColor (VARYING_SLOT_BFC0.xyzw, 1, 0)
decl_function main (0 params)

impl main {
        block block_0:
        /* preds: */
        vec1 32 ssa_0 = deref_var &gl_Vertex (shader_in vec4)
        vec4 32 ssa_1 = intrinsic load_deref (ssa_0) (access=0)
        vec4 32 ssa_2 = load_const (0x00000000, 0x00000000, 0x3f000000, 0x00000000) = (0.000000, 0.000000, 0.500000, 0.000000)
        vec1 32 ssa_5 = deref_var &gl_BackColor (shader_out vec4)
        vec4 32 ssa_11 = mov ssa_2
        vec4 32 ssa_13 = fsat ssa_11
        intrinsic store_deref (ssa_5, ssa_13) (wrmask=xyzw, access=0)
        vec1 32 ssa_7 = deref_var &gl_Position (shader_out vec4)
        vec4 32 ssa_12 = mov ssa_1
        intrinsic store_deref (ssa_7, ssa_12) (wrmask=xyzw, access=0)
        /* succs: block_1 */
        block block_1:
}

shader: MESA_SHADER_FRAGMENT
source_sha1: {0x5059da66, 0x00c609e5, 0x5329c39a, 0x13e2fc88, 0x8e68cb71}
name: GLSL3
internal: false
stage: 4
next_stage: 4
inputs_read: 1
outputs_written: 2
subgroup_size: 1
first_ubo_is_default_ubo: true
flrp_lowered: true
inputs: 1
outputs: 1
uniforms: 0
decl_var shader_in INTERP_MODE_NONE vec4 gl_Color (VARYING_SLOT_COL0.xyzw, 0, 0)
decl_var shader_out INTERP_MODE_NONE vec4 gl_FragColor (FRAG_RESULT_COLOR.xyzw, 0, 0)
decl_function main (0 params)

impl main {
        block block_0:
        /* preds: */
        vec1 32 ssa_0 = deref_var &gl_Color (shader_in vec4)
        vec4 32 ssa_1 = intrinsic load_deref (ssa_0) (access=0)
        vec1 32 ssa_2 = deref_var &gl_FragColor (shader_out vec4)
        intrinsic store_deref (ssa_2, ssa_1) (wrmask=xyzw, access=0)
        /* succs: block_1 */
        block block_1:
}

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23850>

etnaviv: nir: call nir_remove_dead_variables(..) before linking setup

There are cases where there is a chain to an unused nir variable that get removed
by nir_opt_dce. This breaks our current linker as the variable can still be accessed
via nir_foreach_shader_in_variable(..) macro.

So lets call nir_remove_dead_variables(..) just before we setup our linking.

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Acked-by: Lucas Stach <l.stach@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23673>

ir3/analyze_ubo_ranges: Move IR3_DBG_NOUBOOPT check

There is no need to walk the hole nir shader if IR3_DBG_NOUBOOPT
is used.

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23854>

v3dv: don't use the TLB path if we might be copying partial tiles

With TLB paths we are always storing full tiles, so we can't use it
if the regions we store are not a multiple of the tile size (or the
full image).

Unfortunately, at the point we call this we don't usually have the
tile size yet so for now we skip the path if we are not copying
full mip levels.

Fixes various CTS fails in:
dEQP-VK.ycbcr.copy.*.optimal*buffer_optimal*

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23739>

v3dv: fix slice size for miplevels >= 2

We want to store the slice size in pixels not the level size
after padding to a power of 2 we use miplevels >= 2.

Fixes: 1cb2d2a5ee ('v3dv: store slice dimensions in pixels')
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23739>

docs: Update envvars used for tracing

Updates the environment variables reference to document the new common
tracing infrastructure.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20573>

vulkan/rmv,radv: Use common trace trigger

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20573>

radv/rgp: Use common trace trigger

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20573>

radv/rra: Use common trace trigger

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20573>

vulkan/wsi/x11: Capture traces using a hotkey

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20573>

radv: Add radv_trace_mode

...and add the RADV specific trace modes to the VK instance.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20573>

vulkan: Common trace capturing infrastructure

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20573>

meson: Add a xcb-keysyms dependency

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20573>

radv: Call radv_pipeline_init_scratch per shader

Compute pipelines only have one shader, which was not handled correctly
in the case of ray tracing pipelines. Adding radv_shader as an argument
allows us to handle the ray tracing prolog. The original loop is inlined
into its only user (radv_pipeline_graphics.c).

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23812>

intel/ds: Track CCS cache flush bit

Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23786>

iris: implement recommended flush/wait of AUX-TT invalidation

This patch implements the recommended flush/wait of AUX-TT invalidation
according to per command streamer (engine).

Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23786>

iris: Fix AUX-TT invalidation

In order to make sure RCS engine is idle, we need to add
DC flush + CS stall + Render target Cache flush + Depth Cache
on Gfx 12 and additional CCS cache flush on Gfx12.5.

Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23786>

iris: Add CCS cache flush bits

Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23786>

anv: implement recommended flush/wait of AUX-TT invalidation on compute

This patch implements the recommended flush/wait of AUX-TT invalidation
for compute/render command streamer.

Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23786>

anv: Fix AUX-TT invalidation

In order to make sure RCS engine is idle, we need to add
DC flush + CS stall + Render target Cache flush + Depth Cache
on Gfx 12 and additional CCS cache flush on Gfx12.5.

Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23786>

anv: Add CCS cache flush bits to anv_pipe_bits

This will help us to flush the entries out of the CCS cache.

v2:
- Move enum value close to HW bits section (Lionel)

Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23786>

intel/genxml: Add Compute/Blitter CCS aux invalidation register

Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23786>

intel/genxml: Add CCS cache flush field to PIPE_CONTROL

Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23786>

vc4/v3d/ci: update expected results

Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23862>

radv: Enable VK_EXT_fragment_shader_interlock

ACO only currently - not available in LLVM.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250>

zink/ci: Add broken fragment shader interlock test to RADV flakes

Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250>

radv: Disable VRS forcing with Primitive Ordered Pixel Shading

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250>

radv: Apply the POPS missed overlap hardware bug workaround

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250>

ac/gpu_info: Check whether the device has the POPS missed overlap bug

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250>

radv: Handle Primitive Ordered Pixel Shading in DB_SHADER_CONTROL

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250>

radv: Enable the null export workaround with POPS

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250>

radv: Enable POPS collision wave ID shader argument

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250>

radv: Declare POPS collision wave ID shader argument

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250>

radv: Ensure 1x1 shading rate on GFX10.3 with interlock execution mode

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250>

radv: Detect the use of Primitive Ordered Pixel Shading

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250>

radv: Remove unconditional POPS_DRAIN_PS_ON_OVERLAP setting

This hardware hang workaround (PAL waMiscPopsMissedOverlap) is needed only
on some Vega chips, and only for 8 or more samples per pixel. It has a
significant performance cost (around 1.5x-2x in
nvpro-samples/vk_order_independent_transparency), so it should be precisely
configured when setting up Primitive Ordered Pixel Shading.

It was added in 47b780be21d917eaa6a6a6c9e30ba9fba52d9acd, when POPS was not
used in Mesa, with the change being described as "this may not be needed
yet, but let's set it now".

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250>

radeonsi: Remove unconditional POPS_DRAIN_PS_ON_OVERLAP setting

This hardware hang workaround (PAL waMiscPopsMissedOverlap) is needed only
on some Vega chips, and only for 8 or more samples per pixel. It has a
significant performance cost (around 1.5x-2x in
nvpro-samples/vk_order_independent_transparency), so it should be precisely
configured when setting up Primitive Ordered Pixel Shading.

It was added in 47b780be21d917eaa6a6a6c9e30ba9fba52d9acd, when POPS was not
used in Mesa, with the change being described as "this may not be needed
yet, but let's set it now".

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250>

aco: Implement fragment shader interlock intrinsics

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250>

aco: Add Primitive Ordered Pixel Shading waitcnt rules

When letting the overlapping waves enter their ordered sections, there must
be no memory accesses to resources which need primitive-ordered access that
are still pending, or there would be a race between the current wave and
the overlapping waves.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250>

aco: Send MSG_ORDERED_PS_DONE where necessary

If the wave has set the Primitive Ordered Pixel Shading packer ID hardware
register, it must send MSG_ORDERED_PS_DONE once before the program ends.
It's also safe to send the message if the packer ID register hasn't been
set yet, therefore the message may be sent conservatively. For simplicity,
to ensure that it's sent on all execution paths after setting the packer ID
register, always sending it from a top-level block. This is required for
GFX9-10.3 POPS.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250>

aco: Add Primitive Ordered Pixel Shading scheduling rules

Implementing the acquire/release semantics of fragment shader interlock
ordered section in Vulkan, and preventing reordering of memory accesses
requiring primitive ordering out of the ordered section.

Also, the ordered section should be as short as possible, so not reordering
the instructions awaiting overlapped waves upwards, and the exit from the
ordered section downwards.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250>

aco: Skip waitcnt insertion in the discard early exit block

Waits are needed for early exits from inside a Primitive Ordered Pixel
Shading ordered section, but that code doesn't insert them reliably anyway
because it doesn't obtain the counters for the exact locations of the
jumps, which may be anywhere inside the predecessor blocks.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250>

aco: Add Primitive Ordered Pixel Shading pseudo-instructions

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250>

aco: Add s_wait_event argument bit definitions

A wait for export_ready (if the corresponding bit is not set in the
instruction) is done to enter the Primitive Ordered Pixel Shading ordered
section on GFX11.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250>

ac: Define POPS collision wave ID argument SGPR

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250>

aco: Support pops_exiting_wave_id PhysReg usage

pops_exiting_wave_id is a volatile ALU source operand containing the ID of
the latest wave that hasn't exited yet, for comparing with the newest
overlapped wave ID in overlapping waves.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250>

ac/nir: Support Primitive Ordered Pixel Shading in lower_ps

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250>

docs/amd: Document Primitive Ordered Pixel Shading

Acked-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250>

gallivm: Use NIR_PASS macros

These run nir_validate in debug builds, which will avoid bugs slipping in. It's
not enough that llvmpipe doesn't mind illegal NIR, these passes are well within
their rights to fail spectacularly if the NIR wouldn't validate. So validate so
we catch issues early.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23804>

nir/lower_locals_to_regs: Add bool bitsize knob

GLSL booleans (and hence bool derefs) may be translated either as 1-bit or
32-bit NIR registers, depending whether the backend uses nir_lower_bool_to_int32
or not. Add a knob for this and choose the right type for different backends.

Fixes nir_validate failure on
dEQP-VK.subgroups.ballot_broadcast.graphics.subgroupbroadcast_bvec3 run under
lavapipe. That test indexes into a bvec3 array, and gallivm first lowers bools
and then lowers derefs to registers, resulting in random 1-bit booleans mixed in
with 32-bit bools.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23804>

nir/lower_bool_to_int32: Fix progress reporting

If we only lower parameters, that's still progress. Technically.

Fixes: 6a29cb2654f ("nir/lower_bool_to_int32: add support for lowering functions.")
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23804>

rusticl/api: Wire up CL_DEVICE_PROFILING_TIMER_RESOLUTION

Wire up the CL_DEVICE_PROFILING_TIMER_RESOLUTION from the PIPE_CAP.
While here, also set CL_PLATFORM_HOST_TIMER_RESOLUTION to 1;
that's bogus since we're using the same value as for device, but
at this point we don't have a device to ask.

Signed-off-by: Dr. David Alan Gilbert <dave@treblig.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23639>

rusticl/api: Implement get_{device_and_}host_timer

Use the get_timestamp as both the device_timestamp in
get_device_and_host_timer and host_timestamp in that
and get_host_timer.

Having eliminited most other clock sources, discussions
on previous versions have concluded it's best to use the
same timer as the 'host_timestamp' since the main requirements
are that it must be one that's a time seen by the device and
that it's very closely coupled.

Signed-off-by: Dr. David Alan Gilbert <dave@treblig.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23639>

rusticl/device: Stash timestamp availability

Check if the device claims to have timestamps and a valid resolution
and stash it in the device.

Signed-off-by: Dr. David Alan Gilbert <dave@treblig.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23639>

rusticl/screen: Wrap get_timestamp

Add a wrapper on our screen type to call get_timestamp.

Signed-off-by: Dr. David Alan Gilbert <dave@treblig.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23639>

draw: use unsigned instead of uint

uint isn't a standard type, just something we accidentally get from some
other headers.

Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23833>

draw: match type of pipe_draw_start_count_bias::count

Reviewed-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23833>

cso: use unsigned instead of uint

uint isn't a standard type, just something we accidentally get from some
other headers.

Reviewed-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23833>

draw: use stdint.h types

Here, we want explicitly sized types, not just types that happen to be
of the right size.

Reviewed-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23833>

draw: track vertices and vertex_ptr as byte-pointers

Reviewed-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23833>

draw: use enum for primitive-type

Reviewed-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23833>