nir/lower_shader_calls: skip zero-sized qsort

Fixes UBSan:
src/compiler/nir/nir_lower_shader_calls.c:1681:7: runtime error: null pointer passed as argument 1, which is declared to never be null

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25853>

nir/serialize: fix signed integer overflow

Fixes UBSan error:
src/compiler/nir/nir_serialize.c:1277:70: runtime error: left shift of 524287 by 13 places cannot be represented in type 'int'

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25853>

nir/trivialize_registers: Handle obscure load hazard

Somebody less tired than me would add a unit test for this. Offending snippet:

        32    %58 = @load_reg (%55) (base=0, legacy_fabs=0, legacy_fneg=0)
        32    %57 = @load_reg (%55) (base=0, legacy_fabs=0, legacy_fneg=0)
        32    %21 = iadd %57, %15 (0x1)
                    @store_reg (%21, %55) (base=0, wrmask=x, legacy_fsat=0)
        32    %56 = @load_reg (%55) (base=0, legacy_fabs=0, legacy_fneg=0)
        32    %22 = i2f32 %56
        32    %23 = load_const (0x41000000 = 8.000000)
        32    %24 = fdiv %22, %23 (8.000000)
        32    %90 = mov %24
                    @store_reg_indirect (%90, %78, %58) (base=0, wrmask=x, legacy_fsat=0)

Closes: #10031
Fixes: d313eba94ef ("nir: Add pass for trivializing register access")
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reported-by: Timothy Arceri <tarceri@itsqueeze.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25865>

docs/features: remove empty lines confusing mesamatrix

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25886>

radv: report the last GPUVM fault when a device lost is detected

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23238>

radv: query and report the last GPUVM fault with RADV_DEBUG=hang

Tested with a small use-after-free Vulkan test. It's pretty basic
for now but I think I will add status decoding support to report
more useful information.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23238>

radv/amdgpu: add support quering the last GPUVM fault

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23238>

amd: add has_gpuvm_fault_query

Whether the kernel allows to query the last GPUVM fault.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23238>

amd: update amdgpu_drm.h

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23238>

radv: remove outdated RADV_DEBUG=vmfaults support

It's been a very long time since this was useless because dmesg
required sudo access for a while.

This will be replaced by the new GPUVM fault interface which allows
to query from the kernel directly instead of trying to parse dmesg.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23238>

zink: Fix SyntaxWarning in zink_extensions script

Fix regex pattern in zink_extensions to use proper escaping. While the original
code works, it relies on Python ignoring incorrect syntax and fallbacking to
passing through the escape. Current behaviour results in SyntaxWarning
whenever the code is used.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25881>

panfrost: Add env variable for max AFBC packing ratio

Add `PAN_MAX_AFBC_PACKING_RATIO` variable. If the ratio (size of the
packed resource) / (size of the sparse resource) * 100 is above that
value, we don't care about packing it.

Signed-off-by: Louis-Francis Ratté-Boulianne <lfrb@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25012>

panfrost: Add some debug utility methods for resources

Add a method to dump a specific superblock given its index (header
and few first bytes from the body) and a method to dump a resource
to a ppm file.

Signed-off-by: Louis-Francis Ratté-Boulianne <lfrb@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25012>

panfrost: Add debug flag to force packing of AFBC textures on upload

Add `forcepack` flag that will force conversion to AFBC-packed right
after a texture is uploaded when possible. We only pack 2D resources
larger than 32x32 as of now.

Signed-off-by: Louis-Francis Ratté-Boulianne <lfrb@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25012>

panfrost: Don't force constant modifier after converting

After converting a texture from one modifier to another, there is
no reason to force the modifier to stay constant afterwards. Set
back `modifier_constant` to false because it is changed by
`resource_setup` as it is causing issues when implementing AFBC
packing.

Signed-off-by: Louis-Francis Ratté-Boulianne <lfrb@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25012>

panfrost: Legalize resource when attaching to a batch

Make sure we don't convert the texture for nothing by only
legalizing when creating a batch instead of on surface creation.
Also, to avoid recursive blit, we need to legalize the destination
resource before blitting.

Finally, make sure the resource has a sparse memory layout if
AFBC compressed. The GPU doesn't support rendering to a AFBC-packed
texture.

Signed-off-by: Louis-Francis Ratté-Boulianne <lfrb@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25012>

panfrost: Add support for AFBC packing

When the GPU is converting a texture from linear/tiled to compressed
AFBC, it uses a sparse memory layout. That means that the
superblocks are stored starting at intervals equal to the size of an
uncompressed superblock. When memory usage needs to be optimized, it
is possible to pack the resource by trimming each superblock as much
as possible. The GPU will still be able to read from these packed
textures, but won't be able to write directly to them. If the
layout is AFBC-tiled, the packing process will also de-tile as
tiled+packed is not supported by Mali GPUs.

No new modifier flag has been added as the absence of the
`AFBC_FORMAT_MOD_SPARSE` flag means the resource will be packed.

Signed-off-by: Louis-Francis Ratté-Boulianne <lfrb@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25012>

panfrost: Add method to get size of AFBC superblocks valid data

The compute shader is going through all the AFBC header blocks and
calculating the size of valid data by summing up the subblock
sizes (and taking into account solid color, uncompressed mode and
copy blocks). The result is written into the given buffer (array
of `pan_afbc_block_info`).

The size is rounded up to the alignment parameter directly in the
shader.

Signed-off-by: Louis-Francis Ratté-Boulianne <lfrb@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25012>

panfrost: Add infrastructure for internal AFBC compute shaders

A few compute shaders are needed to support AFBC-packing. Here is
just the boilerplate to create, compile and retrieve the shaders.

Signed-off-by: Louis-Francis Ratté-Boulianne <lfrb@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25012>

panfrost: Split out internal of `panfrost_launch_grid`

For an upcoming feature (AFBC packing), we're gonna run internal
compute shaders on resources. This commit just split the internal
of `panfrost_launch_grid` so we can run the shader on a given
batch.

Signed-off-by: Louis-Francis Ratté-Boulianne <lfrb@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25012>

panfrost: Make panfrost_resource_create_with_modifier public

It allows creating a resource when we already know which modifier
we're gonna use.

Signed-off-by: Louis-Francis Ratté-Boulianne <lfrb@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25012>

panfrost: Add panfrost_batch_write_bo

Needed when writing to a BO from a compute shader when the BO isn't
assigned to a resource yet.

Signed-off-by: Louis-Francis Ratté-Boulianne <lfrb@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25012>

panfrost: Precalculate stride and nr of blocks for AFBC layouts

For AFBC packing, we will need to have the stride of the resource
in terms of superblocks and the total number of blocks. It is
easier to pre-calculate once when initializing the resource layout.

Signed-off-by: Louis-Francis Ratté-Boulianne <lfrb@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25012>

panfrost: Add method to get size of AFBC subblocks

Each AFBC superblock is subdivised into subblocks that are
compressed individually. For all formats that we currently support,
the subblocks are 4x4 pixels.

Signed-off-by: Louis-Francis Ratté-Boulianne <lfrb@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25012>

panfrost: Add methods to determine slice and body alignment

Those methods are needed to have the same alignment everywhere
without hard-coding the values

Signed-off-by: Louis-Francis Ratté-Boulianne <lfrb@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25012>

panfrost: Fix error in comment

"linear" -> "tiled"

Signed-off-by: Louis-Francis Ratté-Boulianne <lfrb@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25012>

egl: fix leaking drmDevicePtr in _eglFindDevice

Fixes: 3a9e1014e13 ("egl: Rename _eglAddDevice() to _eglFindDevice()")
Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25524>

anv: fixup 32bit build of internal shaders

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 11b4c23d19 ("anv: add ring buffer mode to generated draw optimization")
Fixes: db335d9b73 ("anv: factor out host/gpu internal shaders interfaces")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10037
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25870>

etnaviv: Don't leak disk_cache

Fixes: 77af1ca690f ("etnaviv: add disk cache")
Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Reviewed-by: Tomeu Vizoso <tomeu@tomeuvizoso.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25877>

ci: Bump PyYAML to 6.0.1

Fixes an issue I am seeing on Fedora 39. For more details
about the upstream fix see https://github.com/yaml/pyyaml/pull/702

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25878>

intel/dev: expand existing fix for all gfx12 with small EU count

Commit 7db1b94e07f added a fix for ADL-N but this issue has been
reproduced also on RPL-S and is likely common with all gfx12 variants
with a small EU count.

cc: mesa-stable

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25861>

compiler/types: Move comments and reorganize declarations

Move comments from C++ member functions to the C functions.  In
some cases just delete comments or consolidate them together.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Remove warnings about potential fallthrough

None of those cases are expected to fallthrough, but should be unreachable.
Just break them so they get to the unreachable entry at the end.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Rename glsl_types.cpp to glsl_types.c

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Annotate extern "C" only once in glsl_types.cpp

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Remove usages of C++ members in glsl_types.cpp

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Use C instead of C++ constants for builtin types

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Add remaining type extraction functions and use them in C++

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Add glsl_simple_explicit_type() and simplify glsl_simple_type()

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Implement glsl_type::field_type() in terms of existing functions

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Add glsl_get_explicit_*() functions and use them in C++

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Add glsl_get_std430_array_stride() and use it in C++

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Add glsl_type_uniform_locations() and use it in C++

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Add glsl_type_compare_no_precision() and use it in C++

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Add glsl_get_mul_type() and use it in C++

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Add more glsl_contains_*() functions and use them in C++

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

rusticl: Add bindings for glsl_vector_type()

This is going to become inline functions, so add the corresponding
Rust bindings for it first.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Move remaining code from nir_types to glsl_types

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Flip wrapping of numeric type conversion functions

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Flip wrapping of remaining small data getters

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Flip wrapping of remaining non-trivial type getters

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Flip wrapping of get row/column type helpers

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Flip wrapping of various get instance functions

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Flip wrapping of texture/sampler/image get instance functions

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Flip wrapping of get_instance()

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Flip wrapping of record_compare

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Flip wrapping of layout related functions

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Flip wrapping of interface related functions

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Flip wrapping of struct related functions

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Flip wrapping of size related functions

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Flip wrapping of CL related functions

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Flip wrapping of cmat related functions

Also add a missing `struct` related to cmat.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Flip wrapping of array related functions

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

compiler/types: Flip wrapping of "type contains?" predicate functions

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25129>

ci_run_n_monitor: Always resolve --rev arguments for looking up pipelines.

Otherwise you'd loop forever never finding it with --rev <paste from
gitlab UI> or --rev <branch name>

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25824>

radeonsi/vcn: Add encode support for H264 B-frames

Tested with ffmpeg and gstreamer vah264enc.

References to B-frames (b_depth/b-pyramid) not supported.

Reviewed-by: Ruijing Dong <ruijing.dong@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25565>

frontends/va: Fix parsing packed headers without emulation bytes

Don't try to handle emulation bytes and only parse first NAL unit
if the packed header has no emulation bytes.

This fixes parsing packed headers from gstreamer.

Reviewed-by: Ruijing Dong <ruijing.dong@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25565>

util/vl: Fix vl_rbsp parser with bitstreams without emulation bytes

This is used for parsing VA packed headers and those can be without
emulation prevention bytes.
Add emulation_bytes argument to vl_rbsp_init and skip all emulation
bytes handling when set.

Reviewed-by: Ruijing Dong <ruijing.dong@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25565>

frontends/va: Parse H264 SPS for max_num_reorder_frames

Reviewed-by: Ruijing Dong <ruijing.dong@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25565>

anv: add gen9 astc workaround

gen9 does not handle denorms in void extent blocks correctly.  We need
to flush them to zero.

Signed-off-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25800>

anv: prep for gen9 astc workaround

We will reuse astc emu for gen9 astc workaround.  This commit contains
minor cleanups and has no functional change.

Signed-off-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25800>

radv: Support SDMA in si_cs_emit_write_event_eop.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25833>

radv: Support SDMA in radv_cp_wait_mem.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25833>

radv: Support SDMA in radv_cs_write_data_head.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25833>

nir: Add NVIDIA-specific geometry shader opcodes

Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25000>

nir: Add NVIDIA-specific I/O intrinsics

NVIDIA hardware doesn't take a vertex index for per-vertex I/O.
Instead, it takes an offset into the primitive.  This has to be fetched
using a combination of SR_INVOCATION_INFO and the ISBERD instruction.
To keep things simple and allow for maximum CSE, we do the lowering in
NIR and patch the load/store_per_vertex_input/output intrinsic.

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25000>

nir: Add a range to most I/O intrinsics

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25000>

nir: Add an load_barycentric_at_offset_nv intrinsic

NVIDIA hardware takes the offset as two 4.12 fixed-point values packed
into a single 32-bit value.

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25000>

nir: Add NV-specific texture opcodes

These are for implementing various texture queries.

Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25000>

nir: Add a load_sysval_nv intrinsic

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25000>

nir/lower_tex: Add a lower_txd_clamp option

Some of us want to lower all TXD with min_lod regardless of whether or
not it's shadow or cube or whatever.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25000>

nir: Add convert_alu_types to divergence analysis

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25000>

nir: Add a nir_ssa_def_all_uses_are_fsat() helper

Extracted from nir_lower_to_source_mods, this is useful for back-ends
which don't want to rely on NIR source and destination modifiers.

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25000>

radv: enable mesh shader gs_fast_launch=2 and multi-row export

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>

radv: implement mesh shader multi-row export

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>

ac/nir: implement mesh shader multi-row export

Unlike AMDVLK, this has separate loops for attribute stores and exports,
so that the stores from different rows can overlap.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>

ac/nir,radv: pass workgroup size to ac_nir_lower_ngg_ms

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>

ac/nir: add emit_ms_outputs helper

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>

radv: implement mesh shader gs_fast_launch=2

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>

ac/nir: optimize mesh shader local_invocation_index

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>

ac/nir: implement mesh shader gs_fast_launch=2

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>

ac/nir: refactor mesh vertex/primitive export

To prepare for row export, move them into helpers with index and row
parameters.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>

ac/nir: remove dead code

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>

ac/nir: add row parameter to helpers

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>

aco,nir: add export_row_amd intrinsic

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>

nir: improve ms_cross_invocation_output_access with local_invocation_id

Since GFX11, RADV doesn't need to lower local_invocation_id.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>

intel/compiler: Don't emit calls to validate() in release build

While the fs_visitor::validate() implementation is empty in release
build, we still emit calls to it since it is defined in a separate
compilation unit than its callers.  To fix this, just expose an inline
empty function in the header for the release mode.

Fossil run time differences in TGL laptop (difference at 95.0% confidence):

```
Rise of The Tomb Rider (Native) [n=7]
        -0.482857 +/- 0.010932
        -1.60608% +/- 0.0363621%

Cyberpunk 2077 (DXVK) [n=7]
        -0.987143 +/- 0.0904516
        -0.82996% +/- 0.076049%

Batman Arkham City (DXVK) [n=7]
        -7.74857 +/- 0.329561
        -1.46298% +/- 0.0622231%
```

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25847>

ac/gpu_info: fix querying the maximum number of IBs per ring

The kernel uses 32-bit and Mesa was using 8-bit...

Fixes: 96345ae2ed3 ("ac/gpu_info: query the maximum number of IBs per submit from the kernel")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25867>

anv: selectively enable FCV optimization for DG2

Enabling FCV on MTL breaks a number of games and benchmarks. Let's
disable it for now till we can root cause the issue.

Closes: #9987
Fixes: 26c2c9 ('anv: enable FCV for Gen12.5')
Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25863>

anv: turn off non zero fast clears for CCS_E

This helps fix a performance regression on games such as F1 22 and RDR2.
Turning on non zero fast clears causes additional partial resolves for
these games that degrades performance. Let's turn off non zero fast
clears till we can eliminate the partial resolves.

Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25863>