intel/compiler: Update opt_split_sends() for Xe2 reg size

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/compiler/fs: Support Xe2 reg size in assign_curb_setup

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/xe2+: Round up size to reg_unit() in fs_reg_alloc::alloc_spill_reg().

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/fs/xe2+: Fix calculation of spill message width for Xe2 regs.

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/fs/xe2+: Fix execution width of SHADER_OPCODE_GET_BUFFER_SIZE for SIMD16 EU.

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/fs/xe2+: Update regioning lowering offset alignment checks for Xe2 regs.

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/fs: Lower unsupported regioning with non-trivial 2D regions on FIXED_GRFs.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/fs/xe2+: Update TASK/MESH payload setup for Xe2 reg size.

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/fs/xe2+: Update BS payload setup for Xe2 reg size.

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/fs/xe2+: Update TES payload setup for Xe2 reg size.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/fs/xe2+: Update TCS payload setup for Xe2 reg size.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/fs/xe2+: Update GS payload setup for Xe2 reg size.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/compiler/xe2: Account for reg_unit() in TES intrinsics

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/compiler/xe2: Account for reg_unit() in TCS intrinsics

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/fs/xe2+: Fix payload layout of sampler messages for Xe2 reg size

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/compiler/xe2: TXD is lowered to SIMD16 in SIMD32 mode

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/compiler/xe2: Use SIMD16 for nir_intrinsic_image_size

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/compiler/xe2: Update fs_visitor::setup_vs_payload to account for Xe2 reg size

[ Francisco Jerez: Simplify. ]

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/compiler: Adjust barrier emission for Xe2+

Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/fs/xe2+: Scale BRW_MAX_MSG_LENGTH by native register size.

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/compiler: Adjust fence message lengths for new register width on Xe2+

Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/compiler: Adjust CS payload registers for new register width on Xe2+

Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/fs/xe2+: Round up fs_builder::vgrf() size calculation to HW register unit.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/fs/xe2+: Update encoding of FB write message payload.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/compiler/xe2+: Represent dispatch_grf_start_reg in native GRF units.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/fs/xe2+: Allow increased SIMD width for various get_fpu_lowered_simd_width() restrictions.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/eu/xe2+: Update validation of GRF region size to account for Xe2 reg size

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/fs/xe2+: Scale MAX_SAMPLER_MESSAGE_SIZE by native register size.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/fs/xe2+: Fixes for increased accumulator register width.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/fs/xe2+: Fix grf_count in post-RA scheduling for updated register file size.

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/fs/xe2+: Fix payload node live range calculations for change in register size.

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/fs: Fix signedness of payload_node_count argument of calculate_payload_ranges().

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/eu/xe2+: Fix encoding of various message descriptors for change in register size.

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/fs/ra/xe2: Scale up register allocation granularity by 2x on Xe2+ platforms.

v2: Fix spill register allocation.  Switch to brw_reg::nr
    representation in fake 256b units.

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/compiler: Make MAX_VGRF_SIZE macro depend on devinfo and update it for Xe2.

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/vec4/ra: Define REG_CLASS_COUNT constant specifying the number of register classes.

Rework:
 * Jordan: 16=>20 following d33aff783d9 ("intel/fs: add support for
   sparse accesses")

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

intel/fs/ra: Define REG_CLASS_COUNT constant specifying the number of register classes.

Rework:
 * Jordan: 16=>20 following d33aff783d9 ("intel/fs: add support for
   sparse accesses")

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25020>

docs: add another 23.1.x

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25310>

docs: update calendar for 23.1.8

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25310>

docs: add sha256sum for 23.1.8

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25310>

docs: add release notes for 23.1.8

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25310>

amd: Use inverse ballot intrinsic if available

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25123>

nir/spirv: Add inverse_ballot intrinsic

This is actually a no-op on AMD, so we really don't want to lower it to
something more complicated.  There may be a more efficient way to do
this on Intel too. In addition, in the future we'll want to use this for
lowering boolean reduce operations, where the inverse ballot will
operate on the backend's "natural" ballot type as indicated by
options->ballot_bit_size, instead of uvec4 as produced by SPIR-V. In
total, there are now three possible lowerings we may have to perform:

- inverse_ballot with source type of uvec4 from SPIR-V to inverse_ballot
with natural source type, when the backend supports inverse_ballot
natively.
- inverse_ballot with source type of uvec4 from SPIR-V to arithmetic,
when the backend doesn't support inverse_ballot.
- inverse_ballot with natural source type from reduce operation, when
the backend doesn't support inverse_ballot.

Previously we just did the second lowering unconditionally in vtn, but
it's just a combination of the first and third. We add support here for
the first and third lowerings in nir_lower_subgroups, instead of simply
moving the second lowering, to avoid unnecessary churn.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25123>

nir/lower_subgroups: Don't do multiple lowerings at once

Since using nir_shader_lower_instructions(), instructions get revisited
before proceeding with the next one. This already guarantees that any
subsequent lowerings of those instructions happen during the same pass
of nir_lower_subgroups().

v2: use nir_shader_lower_instructions() instead of setting the cursor.

Co-authored-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25123>

zink: Store zink_vertex_elements_hw_state::b.strides by binding id

Currently, we store strides by vertex buffer id, which means that we have
to map the binding index to the vertex buffer index every time we want to
get a stride for a given binding. This also creates an order mismatch when
we pass strides directly to CmdBindVertexBuffers2EXT. Instead of converting
strides for CmdBindVertexBuffers2EXT too, we can just store strides by
binding id, and drop the mapping in other places.

Fixes: 76725452 ("gallium: move vertex stride to CSO")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9817
Signed-off-by: Sviatoslav Peleshko <sviatoslav.peleshko@globallogic.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25305>

radv/rt: Skip cull_mask handling if it is FF

Totals from 9 (1.32% of 680) affected shaders:
Instrs: 609329 -> 609057 (-0.04%)
CodeSize: 3267328 -> 3265664 (-0.05%)
Latency: 8289582 -> 8275874 (-0.17%)
InvThroughput: 2166498 -> 2163147 (-0.15%)
VClause: 23581 -> 23583 (+0.01%)
Copies: 51076 -> 51028 (-0.09%)
Branches: 24637 -> 24603 (-0.14%)
PreVGPRs: 996 -> 986 (-1.00%)

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25268>

radv/ray_queries: Skip cull_mask handling if it is FF

Stats for Metro Exodus:

Totals from 26 (0.99% of 2627) affected shaders:
Instrs: 14586 -> 14232 (-2.43%)
CodeSize: 77024 -> 75192 (-2.38%)
VGPRs: 1408 -> 1208 (-14.20%)
Latency: 315076 -> 309898 (-1.64%)
InvThroughput: 42345 -> 41677 (-1.58%)
VClause: 366 -> 374 (+2.19%)
Copies: 2840 -> 2800 (-1.41%); split: -1.48%, +0.07%
Branches: 587 -> 561 (-4.43%)
PreSGPRs: 897 -> 853 (-4.91%)
PreVGPRs: 1290 -> 1122 (-13.02%)

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25268>

radv/bvh: Treat instances with mask == 0 as inactive

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25268>

anv: refactor to fix pipe control debugging

While earlier changes to pipe control emission allowed debug dump of
each pipe control, they also changed debug output to almost always print
same reason/function for each pc. These changes fix the output so that
we print the original function name where pc is emitted.

As example:

pc: emit PC=( +depth_flush +rt_flush +pb_stall +depth_stall ) reason: gfx11_batch_emit_pipe_control_write
pc: emit PC=( ) reason: gfx11_batch_emit_pipe_control_write

changes back to:

pc: emit PC=( +depth_flush +rt_flush +pb_stall +depth_stall ) reason: gfx11_emit_apply_pipe_flushes
pc: emit PC=( ) reason: cmd_buffer_emit_depth_stencil

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25282>

v3dv: we can sample from 1D array too

Fixes: 95f881ad ('v3dv: add support for sampling simple 2D linear textures')
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25286>

freedreno/a6xx: Add L8_SRGB

Avoids a tragic slow-path with CS:GO

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25298>

ci/zink: Add a few updates for anv/tgl from the nightly runs.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25301>

ci/virgl: Disable virgl-iris-traces.

It's been failing with "No virgl contexts available on hostlibEGL warning:
egl: failed to create dri2 screen" for ages, and nobody seems to care.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25301>

ci/intel: Add various updates from our nightly runs.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25301>

vc4: mark buffers as initialized at vc4_texture_subdata

This fixes several tests when the initially uploaded buffer
from CPU was being ignored because vc4_texture_subdata was not
marking the resource as written/initialized.

The usage flags management available at vc4_resource_transfer_map
is generalized into vc4_map_usage_prep and reused at
vc4_resource_transfer_map. This makes vc4 implementation more similar
to v3d.

This fixes 7 text in the following subgroups:
  -dEQP-GLES2.functional.fbo.render.texsubimage.*
  -dEQP-GLES2.functional.texture.specification.basic_copytexsubimage2d.*
  -spec@arb_clear_texture@arb_clear_texture-*

Cc: mesa-stable
Reviewed-by: Juan A. Suarez <jasuarez@igalia.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25297>

iris: assert(bo->deps) after realloc()

Iris in general doesn't really like checking the return value of its
allocations, but in some places it does assert that those pointers are
non-NULL. We've recently investigated a bug that could have been
coming from a failed bo->deps realloc(), so add the assert() here to
help give us more confidence over things the next time we're debugging
issues.

Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25236>

iris: avoid stack overflow in iris_bo_wait_syncobj()

Keep most cases using the stack as it's cheaper, but fall back to the
heap when the size gets too big.

This should fix a stack overflow reported by @rhezashan for a case
where we had lots of iris_screens.

Credits to Matt Turner and José Roberto de Souza for their work on
this issue, which led us to find its root cause.

Cc: mesa-stable
Reported-by: rheza shandikri (@rhezashan in gitlab)
Credits-to: José Roberto de Souza <jose.souza@intel.com>
Credits-to: Matt Turner <mattst88@gmail.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25236>

iris: assert bufmgr->bo_deps_lock is held

This is the only place that touches bo->deps but does not explicitly
lock it and is not a setup/teardown function where locking won't help
anything.

I'm confident we won't hit this assertion, but I've recently had this
lock as the suspect of a bug and had to check the callers to see if we
could be calling from any unlocked place. Having the assert helps
increasing our confidence.

Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25236>

nir/move_vec_src_uses_to_dest: allow to skip reuse of constant sources

And enable this for r300 and intel-vec4

crocus HSW (mostly helps few doplhin ubershaders):
total instructions in shared programs: 1576736 -> 1576589 (<.01%)
instructions in affected programs: 38235 -> 38088 (-0.38%)
helped: 12
HURT: 0
total cycles in shared programs: 111025838 -> 110944796 (-0.07%)
cycles in affected programs: 5646582 -> 5565540 (-1.44%)
helped: 15
HURT: 6
total spills in shared programs: 447 -> 432 (-3.36%)
spills in affected programs: 186 -> 171 (-8.06%)
helped: 12
HURT: 0
total fills in shared programs: 792 -> 774 (-2.27%)
fills in affected programs: 291 -> 273 (-6.19%)
helped: 12
HURT: 0

r300 RV530:
total instructions in shared programs: 96655 -> 96304 (-0.36%)
instructions in affected programs: 15020 -> 14669 (-2.34%)
helped: 79
HURT: 18
total temps in shared programs: 13027 -> 12952 (-0.58%)
temps in affected programs: 677 -> 602 (-11.08%)
helped: 41
HURT: 9
total cycles in shared programs: 147745 -> 147314 (-0.29%)
cycles in affected programs: 21831 -> 21400 (-1.97%)
helped: 84
HURT: 19

r300 RV370:
total instructions in shared programs: 63678 -> 63669 (-0.01%)
instructions in affected programs: 931 -> 922 (-0.97%)
helped: 12
HURT: 6
total temps in shared programs: 10028 -> 10013 (-0.15%)
temps in affected programs: 339 -> 324 (-4.42%)
helped: 33
HURT: 10
total cycles in shared programs: 101118 -> 101087 (-0.03%)
cycles in affected programs: 2659 -> 2628 (-1.17%)
helped: 22
HURT: 6

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24932>

nir/move_vec_src_uses_to_dest: skip reuse if vec is used only once in store_output

lima and etnaviv show no change in shader-db.

crocus HSW:
total instructions in shared programs: 1576762 -> 1576736 (<.01%)
instructions in affected programs: 485 -> 459 (-5.36%)
helped: 28
HURT: 1
total cycles in shared programs: 111025898 -> 111025838 (<.01%)
cycles in affected programs: 1248 -> 1188 (-4.81%)
helped: 29
HURT: 0

RV370:
total instructions in shared programs: 63889 -> 63558 (-0.52%)
instructions in affected programs: 9116 -> 8785 (-3.63%)
helped: 129
HURT: 0
total temps in shared programs: 10071 -> 10016 (-0.55%)
temps in affected programs: 285 -> 230 (-19.30%)
helped: 51
HURT: 0
total cycles in shared programs: 101344 -> 100997 (-0.34%)
cycles in affected programs: 9326 -> 8979 (-3.72%)
helped: 129
HURT: 0

RV530:
total instructions in shared programs: 93597 -> 93267 (-0.35%)
instructions in affected programs: 10309 -> 9979 (-3.20%)
helped: 166
HURT: 0
total temps in shared programs: 13019 -> 12955 (-0.49%)
temps in affected programs: 337 -> 273 (-18.99%)
helped: 61
HURT: 1
total cycles in shared programs: 144506 -> 144159 (-0.24%)
cycles in affected programs: 10662 -> 10315 (-3.25%)
helped: 165
HURT: 0

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24932>

r300: enable nir_move_vec_src_uses_to_dest

We want to do this in general, right now the stats are not that good but
that will be taken care of in the next commits.

RV530:
total instructions in shared programs: 93561 -> 93597 (0.04%)
instructions in affected programs: 39015 -> 39051 (0.09%)
helped: 207
HURT: 212
total temps in shared programs: 12864 -> 13019 (1.20%)
temps in affected programs: 2010 -> 2165 (7.71%)
helped: 57
HURT: 181
total cycles in shared programs: 144639 -> 144506 (-0.09%)
cycles in affected programs: 54524 -> 54391 (-0.24%)
helped: 191
HURT: 234

RV370:
total instructions in shared programs: 63692 -> 63811 (0.19%)
instructions in affected programs: 16851 -> 16970 (0.71%)
helped: 121
HURT: 141
total temps in shared programs: 9966 -> 10050 (0.84%)
temps in affected programs: 969 -> 1053 (8.67%)
helped: 33
HURT: 126
total cycles in shared programs: 101042 -> 101205 (0.16%)
cycles in affected programs: 20606 -> 20769 (0.79%)
helped: 121
HURT: 155

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24932>

docs: Update Clover's env variable documentation

Fixes: 981bc603b46c ("clover: implement CLOVER_DEVICE_TYPE like RUSTICL_DEVICE_TYPE")

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21657>

anv: define clear color localy within can_fast_clear_color_att

We can drop a extra function argument this way.

Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24972>

v3dv: only handle Android Hardware Buffer on Android

Fixes: 733909a6 ('v3dv/android: Add AHardwareBuffer support')

Fixes the following CTS regression on Linux:
dEQP-VK.api.external.memory.android_hardware_buffer.dedicated.image.info
dEQP-VK.api.external.memory.android_hardware_buffer.suballocated.image.info

Reviewed-by: Roman Stratiienko <r.stratiienko@gmail.com>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25283>

radv: remove absolute_depth_bias workaround

This was only used with Path of Exile and the game bug seems fixed.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25198>

radv: remove drirc workarounds for Path Of Exile

According to https://gitlab.freedesktop.org/mesa/mesa/-/issues/9798,
all game bugs should have been fixed.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9798
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25198>

drirc: remove Path of Exile workarounds

According to https://gitlab.freedesktop.org/mesa/mesa/-/issues/9798,
all game bugs should have been fixed.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25198>

ac/perfcounter: add GFX11 groups

Source from PAL.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25245>

ac/perfcounter: add SG_WQP group for GFX11

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25245>

radv: fix missing ISA with RGP and GPL

The pipeline hash is required for RGP to correctly report the ISA, so
it should be computed for fast-linked pipelines with GPL (libraries
aren't captured).

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9169
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25275>

radv: fix checking if RGP is enabled with others tracing tools

This is a bitmask.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25275>

crocus: avoid issues with undefined clip distance

Cc: mesa-stable
Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25177>

iris: avoid issues with undefined clip distance

Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9797
Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25177>

egl/wayland: enable WL_bind_wayland_display for zink

Acked-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24975>

egl/wayland: use more registry listeners to better handle device init

this handles globals like dmabuf and wl_drm and also enables creating
egl devices with valid fds

Acked-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24975>

egl/wayland: split out wl drm extension init

no functional changes

Acked-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24975>

egl/swrast: expose EXT_swap_buffers_with_damage and EXT_present_opaque

Acked-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24975>

radeonsi/vcn: Don't hang GPU when using DCC surface as encoder input

Using DCC surface as encoder input will result in corrupted image in the
video, but early returning here will instead hang GPU.
Replace return with assert.

Reviewed-by: Leo Liu <leo.liu@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25259>

venus: drop device, family, index, flags tracking from vn_queue

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25262>

venus: use more common vk_queue related implementations

This change uses common impl for below:
- GetDeviceQueue2
- DeviceWaitIdle

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25262>

venus: use common ANB implementation

This change has a dependency over
https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25185

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25262>

venus: use common vk_queue object

This change only updates the object base to be vk_queue.

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25262>

vulkan/android: drop vk_buffer dependency from common AHB impl

Unlike AHB image, the spec has ensured no special treatment for
allocationSize for AHB buffer export operation.

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25263>

vulkan/android: add missing AHARDWAREBUFFER_USAGE_GPU_DATA_BUFFER usage

An AHB backing a Vkbuffer requires AHARDWAREBUFFER_USAGE_GPU_DATA_BUFFER
usage bit, which is missed from the original ANV and RADV Android
frontends as well as the common VK Android refactor.

Cc: mesa-stable
Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25263>

d3d12: Video - Relax ID3D12VideoDevice QI version for decode, process

Currently asking for ID3D12VideoDevice2 for process and
ID3D12VideoDevice3 for decode, which in reality they only
need ID3D12VideoDevice.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9824
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25272>

vulkan/android: add missed STACK_ARRAY_FINISH()

Fixes: 3c4c263dc73 ("vulkan/android: improve vkQueueSignalReleaseImageANDROID")
Signed-off-by: Ryan Neph <ryanneph@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25277>

nir: add a deref slot counter that handles compact

Conor suggested this, so we can mark slots properly
in the io marking.

This fixes a problem seen when rewriting llvmpipe to use
nir info instead of tgsi info.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24803>

nir: Remove nir_ssa_for_src

It is now unused and has no real use cases now that nir_register is gone.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Christian Gmeiner <cgmeiner@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25247>

treewide: Remove remaining nir_ssa_for_src

Coccinelle missed these, a few manual changes here.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Christian Gmeiner <cgmeiner@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25247>

treewide: Drop nir_ssa_for_src users

Via Coccinelle patch:

    @@
    expression b, s, n;
    @@

    -nir_ssa_for_src(b, *s, n)
    +s->ssa

    @@
    expression b, s, n;
    @@

    -nir_ssa_for_src(b, s, n)
    +s.ssa

Reviewed-by: Christian Gmeiner <cgmeiner@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25247>

agx: Enable sinking ALU

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24833>

nir/opt_sink: Also consider load_preamble as const

Acts like constants, schedule them like constants. This lets us move lowered
frag coord code down. Results on dolphin ubers:

   total instructions in shared programs: 195144 -> 196633 (0.76%)
   instructions in affected programs: 175737 -> 177226 (0.85%)
   helped: 28
   HURT: 27
   Instructions are HURT.

   total bytes in shared programs: 1379980 -> 1388308 (0.60%)
   bytes in affected programs: 1244250 -> 1252578 (0.67%)
   helped: 28
   HURT: 27
   Bytes are HURT.

   total halfregs in shared programs: 13591 -> 13557 (-0.25%)
   halfregs in affected programs: 2176 -> 2142 (-1.56%)
   helped: 12
   HURT: 2
   Inconclusive result (%-change mean confidence interval includes 0).

   total threads in shared programs: 233728 -> 234112 (0.16%)
   threads in affected programs: 3264 -> 3648 (11.76%)
   helped: 6
   HURT: 0
   Threads are helped.

Results on Android shader-db:

   total instructions in shared programs: 1775324 -> 1775912 (0.03%)
   instructions in affected programs: 155305 -> 155893 (0.38%)
   helped: 353
   HURT: 548
   Instructions are HURT.

   total bytes in shared programs: 11676650 -> 11678454 (0.02%)
   bytes in affected programs: 1058924 -> 1060728 (0.17%)
   helped: 370
   HURT: 547
   Inconclusive result (value mean confidence interval includes 0).

   total halfregs in shared programs: 484143 -> 471212 (-2.67%)
   halfregs in affected programs: 98833 -> 85902 (-13.08%)
   helped: 2478
   HURT: 674
   Halfregs are helped.

Instr count changes due to losing the RA lottery.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24833>

nir/opt_sink: Move ALU with constant sources

In general, sinking ALU instructions can negatively impact register pressure,
since it extends the live ranges of the sources, although it does shrink the live range
of the destination.

However, constants do not usually contribute to register pressure. This is not a
totally true assumption, but it's pretty good in practice, since...

* constants can be rematerialized (backend-dependent)
* constants can often be inlined (ISA-dependent)
* constants can sometimes be promoted to free uniform registers (ISA-dependent)
* constants can live in scalar registers although the ALU destination might need
  a vector register (and vector registers are assumed to be much more expensive
  than scalar registers, again ISA-dependent)

So, assume that constants have zero effect on register pressure. Now consider an
ALU instruction where all but one source is a constant. Then there are two
cases:

1. The ALU instruction is moved past when its source was otherwise killed. Then
   there is no effect on register pressure, since the source live range is
   extended exactly as much as the destination live range shrinks.

2. The ALU instruction is moved down but its source is still alive where it's
   moved to. Then register pressure is improved, since the source live range is
   unchanged while the destination live range shrinks.

So, as a heuristic, we always move ALU instructions where n-1 sources are
constant. As an inevitable special case, this also (necessarily) moves unary ALU
ops, which should be beneficial by the same justification. This is not 100%
perfect but it is well-motivated. Results on AGX are decent:

   total instructions in shared programs: 1796101 -> 1795652 (-0.02%)
   instructions in affected programs: 326822 -> 326373 (-0.14%)
   helped: 800
   HURT: 371
   Inconclusive result (%-change mean confidence interval includes 0).

   total bytes in shared programs: 11805004 -> 11801424 (-0.03%)
   bytes in affected programs: 2610630 -> 2607050 (-0.14%)
   helped: 912
   HURT: 462
   Inconclusive result (%-change mean confidence interval includes 0).

   total halfregs in shared programs: 525818 -> 515399 (-1.98%)
   halfregs in affected programs: 118197 -> 107778 (-8.81%)
   helped: 2095
   HURT: 804
   Halfregs are helped.

   total threads in shared programs: 18916608 -> 18917056 (<.01%)
   threads in affected programs: 4800 -> 5248 (9.33%)
   helped: 7
   HURT: 0
   Threads are helped.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24833>

nir/opt_sink: Do not move derivatives

At the moment, this does nothing. It will prevent problems from the next patch,
however.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24833>

nir/opt_sink: Sink frag coord instructions

load_input-like. ubershaders:

   instructions in affected programs: 72392 -> 72522 (0.18%)
   helped: 8
   HURT: 18
   Inconclusive result (value mean confidence interval includes 0).

   total bytes in shared programs: 1468550 -> 1469170 (0.04%)
   bytes in affected programs: 560486 -> 561106 (0.11%)
   helped: 10
   HURT: 17
   Inconclusive result (value mean confidence interval includes 0).

   total halfregs in shared programs: 13946 -> 13898 (-0.34%)
   halfregs in affected programs: 3642 -> 3594 (-1.32%)
   helped: 21
   HURT: 0
   Halfregs are helped.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24833>

nir/opt_sink: Sink load_local_pixel_agx

This is the AGX version of load_output, which shaders can use for framebuffer
fetch. It is beneficial to sink framebuffer fetch as late as possible, both to
reduce register pressure but also to reduce serialization of overlapping
fragments.

Results on a collection of ubershaders:

   total bytes in shared programs: 1468928 -> 1468550 (-0.03%)
   bytes in affected programs: 495300 -> 494922 (-0.08%)
   helped: 24
   HURT: 0
   Bytes are helped.

   total halfregs in shared programs: 14162 -> 13946 (-1.53%)
   halfregs in affected programs: 5148 -> 4932 (-4.20%)
   helped: 27
   HURT: 0
   Halfregs are helped.

   total threads in shared programs: 216896 -> 217664 (0.35%)
   threads in affected programs: 6912 -> 7680 (11.11%)
   helped: 12
   HURT: 0
   Threads are helped.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24833>

nir/opt_sink: Sink load_constant_agx

By the time this runs, we will have already lowered load_ubo and load_vbo to
load_constant_agx so we need to handle the backend version.

This is very important for reducing register pressure in monolithic VS+GS
shaders on AGX. Since no other backend has _agx intrinsics, there's no need for
an option to gate this.

The additional instruction count is from more frequent wait instructions due to
fewer instructions grouped together. This should be mitigated in the future with
an ACO-style latency-reducing scheduler in the backend, after register pressure
is reduced by opt_sink.

total instructions in shared programs: 1793385 -> 1796101 (0.15%)
instructions in affected programs: 199816 -> 202532 (1.36%)
helped: 3
HURT: 941
Instructions are HURT.

total bytes in shared programs: 11799628 -> 11805004 (0.05%)
bytes in affected programs: 1345656 -> 1351032 (0.40%)
helped: 34
HURT: 919
Bytes are HURT.

total halfregs in shared programs: 533151 -> 525818 (-1.38%)
halfregs in affected programs: 40335 -> 33002 (-18.18%)
helped: 613
HURT: 42
Halfregs are helped.

total threads in shared programs: 18910464 -> 18916608 (0.03%)
threads in affected programs: 6144 -> 12288 (100.00%)
helped: 12
HURT: 0
Threads are helped.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24833>

nir/gather_info: Use nir_op_is_derivative

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24833>

nir/opt_gcm: Use nir_op_is_derivative more

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24833>

nir/opt_preamble: Use nir_op_is_derivative

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24833>