tgsi: use enum for tgsi-file type

Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24002>

tgsi: use stdint.h types

Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24002>

tgsi: use enum instead of defines

Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24002>

aux/util: uint -> unsigned

Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24002>

aux/util: use stdint.h types

Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24002>

aux/util: use unsigned instead of uint

Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24002>

aux/util: use enum for primitive-type

Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24002>

aux/util: match type of pipe_draw_start_count_bias::start/count

Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24002>

aux/util: use enum for render-condition

Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24002>

aux/pp: use unsigned instead of uint

Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24002>

gallivm: use unsigned instead of uint

Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24002>

pipebuffer: use unsigned instead of uint

Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24002>

aux/trace: use stdint.h types

...as well as stdbool.h ;)

Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24002>

math: drop MAT_[ST][XYZ] defines

These were used a single time each, and that argually made the code a
bit more confusing to read. Let's just drop these, and use the correct
indexes directly.

Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24003>

math: remove unused defines

Since libmesa_classic got removed, we no longer have the assembly
sources that used these defines, so let's get rid of them.

Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24003>

math: fix indentation in m_matrix.[ch]

Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24003>

lima: use u_pipe_screen_lookup_or_create in the renderonly path too

close #8739

Signed-off-by: Zhang Ning <zhangn1985@outlook.com>
Reviewed-by: Erico Nunes <nunes.erico@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23538>

docs/features.txt(fix): mark VK_EXT_pipeline_robustness as supported on radv

This was implemented in https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23912

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24013>

vbo: correctly restore _VaryingInputs for display list fast path

Fixes: 3a294ff0 - mesa: move the _mesa_set_varying_vp_inputs call to where the state changes
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9007

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23796>

st/glsl: merge st_glsl_to_ir.cpp with st_glsl_to_nir.cpp

There is no longer any other IR so lets finally merge these together
to make the code easier to follow.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22846>

st/glsl: merge st_link_glsl_to_nir() into st_link_nir()

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22846>

st/glsl: merge link_shader() into st_link_nir()

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22846>

glsl: remove glsl ir optimisation loop from linker

We no longer need to call this here and can depend on the glsl_to_nir
pass to call it and clean up anything it cannot handle. Everything
else can now depend on the nir optimisation passes.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22846>

glsl: move store_fragdepth_layout() to nir linker

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22846>

glsl: drop link_invalidate_variable_locations()

All this code does is reinitialise the values to what the original
ir_variable() call already set them too. This code is very old dating
to the initial glsl compiler support, it has probably been unrequired
for a long time now.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22846>

glsl: set last_vert_prog in the nir linker

Another step to removing the GLSL IR linker

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22846>

glsl: inline link_varyings()

Most code has already been moved to the NIR linker.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22846>

glsl: remove the always_active_io flag from GLSL IR

No longer used.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22846>

glsl: move disable_varying_optimizations_for_sso() to NIR linker

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22846>

nir/glsl: add nir_var_declared_implicitly enum

This will be used in the following patch.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22846>

glsl: drop the dce of global vars from GLSL IR linker

All this does is compilcate things such as forcing us to set
var->data.always_active_io in the glsl linker. Just let NIR clean
these up for us instead.

A Zink test hits a new assert but this is not a regression it just
uncovers an existing mesa bug.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22846>

glsl: remove unused system vars

System vars are added to the resource list as inputs so remove any
dead ones before building the list.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22846>

glsl: remove unused buffer objects with packed layout

These are currently removed by the GLSL IR DCE pass but we will
drop that in a following patch. Also there are scenarios where these
might not be detected as unused until the NIR optimisations have
been run so we really need to do it here too anyway.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22846>

glsl: call assign_attribute_or_color_locations() in NIR linker

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22846>

glsl: port assign location code for VS inputs or FS outputs

Here we port the code to the NIR linker.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22846>

glsl: move lowering linker code out of st

Move all these lowering calls into the linker where they belong. This
makes future changes to the linker more flexible and is needed to
allow some following patches as we need to call things in a specific
order.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22846>

glsl: move some compiler code out of st

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22846>

mesa: add some new constants

These will be used by the compiler in a following patch.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22846>

glsl: add some more c wrappers for string_to_uint_map

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22846>

broadcom/compiler: don't leak v3d_compile when finding a new best strategy

If we had selected a best strategy and find an even better one we need to
make sure we free the previous one.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24001>

broadcom/compiler: free defin and defout arrays if they already exist

Just like we do for everything else here, since we are going to realloc
them again right below. Notice this is not exactly a memory leak, since
all these arrays are allocated with ralloc using v3d_compile as context,
so all allocations will be eventually freed when the context is destroyed.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24001>

tu: Add missing dbg reg stomping to tu_CmdBeginRendering

Also we shouldn't stomp PC_DRAW_INDX_BASE - we never use it and
stomping it together with some other reg leads to a fault.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23993>

llvmpipe,lavapipe: Relayout lp_descriptor

This is more robust in case of a descriptor type mismatch.

Fixes test_null_descriptor_mismatch_type (vkd3d-proton).

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23996>

llvmpipe: Allocate more dummy sample functions for FORMAT_NONE

When a null texture is sampled with a non-null sampler, having only one
set of sample functions would lead to an OOB read.

Fixes test_null_srv (vkd3d-proton).

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23996>

llvmpipe: Allow comparison sampling for float formats

Fixes test_gather_c (vkd3d-proton).

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23996>

lavapipe: Set the descriptor count to what vkd3d-proton requires

vkd3d-proton tries to allocate a maximum size descriptor set. The
previous limits were to high for that to work reliably.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23996>

gallivm: Fix atomic_global types

Passes LLVM validation on the test_mesh_shader_rendering vkd3d-proton
test.

cc: mesa-stable

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23996>

glsl: Fix yylloc.source propagation in YYLLOC_DEFAULT

Currently, it's always initialized to 0, but we should take the value from
the grouping passed to the macro. This way parser will have the full
location info, and errors originating from it will show the correct
source file number.

Fixes: a0cfe8c4 ("glsl: Fix missing initialization of yylloc.source")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9229
Signed-off-by: Sviatoslav Peleshko <sviatoslav.peleshko@globallogic.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23966>

radeonsi: fix refcnt imbalance related to util_blitter_save_fragment_constant_buffer_slot()

Indeed, util_blitter_clear() requires a call to
util_blitter_save_fragment_constant_buffer_slot(),
but most other blitter functions do not.

For instance, this issue is triggered with:
"piglit/bin/object-namespace-pollution glDrawPixels buffer -auto -fbo"
while setting GALLIUM_REFCNT_LOG=refcnt.log.

Fixes: 03bc7503d471 ("radeonsi: save the fs constant buffer to the util blitter context")
Signed-off-by: Patrick Lerda <patrick9876@free.fr>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23856>

clc: llvm 17 requires opaque pointers.

You can't turn this off for llvm17+, this at least makes things
run against llvm git now instead of blowing up in clang.

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24000>

r300: assert that every writer has a reader

Dead writes can lead to problems with regalloc, so add a safety assert
to catch such cases in the vertex shaders at least in the meantime.

Additionally we could think there are no readers due to some shortcoming
of out dataflow analysis or some other bug which we would also like to
know about.

Reviewed-by: Filip Gawin <filip.gawin@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23927>

r300: disable ntt regalloc for vertex shaders

We have much better regalloc in the backend, and additionally having
a close to ssa form means some optimizations can be more effective.

RV370:
total instructions in shared programs: 82500 -> 81645 (-1.04%)
instructions in affected programs: 32147 -> 31292 (-2.66%)
helped: 396
HURT: 1
total temps in shared programs: 12355 -> 12465 (0.89%)
temps in affected programs: 368 -> 478 (29.89%)
helped: 5
HURT: 96

GAINED: shaders/trine/vp-237.shader_test VS
GAINED: shaders/trine/vp-79.shader_test VS

RV530:
total instructions in shared programs: 130706 -> 129684 (-0.78%)
instructions in affected programs: 40902 -> 39880 (-2.50%)
helped: 428
HURT: 1
total temps in shared programs: 16811 -> 16920 (0.65%)
temps in affected programs: 421 -> 530 (25.89%)
helped: 7
HURT: 89

The instruction decrease is from the channel merging pass which can be
much more agressive when we have ssa-like form.

The temp regressions are cases where we merge something like
  3: MAD output[1].xy, const[8].xy__, input[1].ww__, temp[0].xy__;
 ....
 12: MOV output[1].zw, none.__00;

We always merge the first instruction into the second one, which means
the liverange for temp[0] will be unnecessarily extended here.
This can be fixed with the following draft MR
https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19790
however if we ever get a VS pair scheduling support this will be solved
as well as a consequence, so let it be for now.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7693
Reviewed-by: Filip Gawin <filip.gawin@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23927>

r300: remove most of backend contant folding

This is now done in NIR. The remaining one for ADD + 0 to MOV is kept
until we move some remaining part of FS lowering to NIR.

There single regressions is in one d3d->glsl shader from Wine.
Wine sets invariant for glPosition which translates to exact bit for all
calculations leading to it (or the TGSI PRECISE flag). r300 backend
ignores is completelly, so removing the backend optimizations should
even make us more correct in this regards.

RV530:
total instructions in shared programs: 130705 -> 130706 (<.01%)
instructions in affected programs: 16 -> 17 (6.25%)
helped: 0
HURT: 1

RV370: no change

Reviewed-by: Filip Gawin <filip.gawin@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23927>

r300: add some early safe bool lowering

This lowers some of the bool-producing comparisons and following bcsels
if the bool comparison results is only used in the bcsel.
This is a temporary solution before we can fork ntt and optimize
the pass sequence there. Right now if we have something like
bcsel(a,b,0.0) we lower it to flrp in nir_lower_bool_to_float. The
flrp goes to backend where it will be lowered to 2 MADs. However in this
case with one of the arguments being a constant one MAD is enough. The
backend can figure this out in the constant folding pass, however this
is actually one of the last things we need it for. So if we do early
translation of the bcsels, than the algebraic pass can clean it up and
we can remove more backend code in the next patch.

no significant change with RV370 shader-db:
total instructions in shared programs: 82497 -> 82496 (<.01%)
instructions in affected programs: 1029 -> 1028 (-0.10%)
helped: 4
HURT: 3
total temps in shared programs: 12351 -> 12355 (0.03%)
temps in affected programs: 10 -> 14 (40.00%)
helped: 0
HURT: 4

Reviewed-by: Filip Gawin <filip.gawin@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23927>

r300: lower undefs to zero

They will get translated to read from random register otherwise, which is
not problematic per se, but they will not be regalloced and if the
initial register index was too high, we can fail the shader compilation
because we think we run out of registers.

Almost no effect with shader-db on RV530:
total instructions in shared programs: 130707 -> 130705 (<.01%)
instructions in affected programs: 1012 -> 1010 (-0.20%)
helped: 2
HURT: 1

Reviewed-by: Filip Gawin <filip.gawin@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23927>

radeonsi: Use ALIGN_POT instead ALIGN_TO

ALIGN_POT would be a bit faster as it's have no divide arithmetic

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23987>

radv: Re-enable RT pipeline capture/replay handles

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>

radv/rt: Rework radv_GetRayTracingCaptureReplayShaderGroupHandlesKHR

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>

radv/rt: Replay shader allocations according to capture/replay handle

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>

radv/rt: Associate capture/replay handles with stages

For stages where the capture/replay handle is only known after compiling
and uploading the shader, the shader needs to be relocated to the VA
corresponding to the capture/replay address.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>

radv/rt: Only compare the non-recursive capture/replay handle

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>

radv: Add radv_rt_capture_replay_handle

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>

radv: Add support for creating capture/replay shaders

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>

radv: Split up implementation of radv_shader_create

This will make it easy to re-use the split-up parts for creating
replayed shaders.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>

radv: Break up radv_shader_nir_to_asm

radv_shader_nir_to_asm actually had 3 functions: compiling the NIR to
asm, uploading the shaders and generating debug info for them.
This reduces the functionality of radv_shader_nir_to_asm to only compile
NIR to asm. Uploading the shader and generating debug info is split into
separate functions.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>

radv: Add radv_shader_reupload

Used for relocating RT shaders with capture/replay.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>

radv: Add utilities to serialize and deserialize shader allocation info

Can be used to capture/replay an arbitrary sequence of shader
allocations while preserving VAs.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>

radv: Add option to allocate shaders in replayable VA range

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>

radv: Move shader arena allocation to a separate function

The arena size is also needed for capture/replay.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>

radv: Add radv_shader_free_list

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>

iris: Convert slab address to canonical

This was the only missing case of bo->address that could possibly not
formated as canonical.

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23994>

nir: Fix breaking in nir_foreach_phi(_safe)

When I reading through some of my older commits I noticed that `break` in
`nir_foreach_phi` is broken because I used the two-loop trick wrong. Rewrite the
macros to fix this, and also to generally be a lot cleaner.

Fixes: 7dc297cc141 ("nir: Add nir_foreach_phi(_safe) macro")
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23957>

kmsro: assert that scanout refcount is larger than 0

The dumb buffer backing the renderonly_scanout is only destroyed if the
refcount reaches zero. If a driver does not correctly initialize the
refcount, the refcount may be negative and the buffer will never be
freed.

Add an assert to ensure that drivers correctly initialize the refcount.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23743>

panfrost: remove BO from cache before closing GEM

If the GEM is closed before setting the BO in the sparse array to zero,
a newly allocated GEM may be associated with a stale BO that is left in
the cache reusing an old BO.

Zero the BO before closing the GEM to make sure that the BO is removed
from the cache and won't be associated with a different GEM.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23744>

winsys/panfrost: Make sure we reset scanout on error in create_kms_dumb_buffer_for_resource()

If an error occured, make sure we reset the scanout object before
leaving, otherwise the next user of this handle will hit the
refcnt == 0 assert.

Fixes: ad4d7ca83324 ("kmsro: Fix renderonly_scanout BO aliasing")
Cc: mesa-stable
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Italo Nicola <italonicola@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23746>

renderonly: Make sure we reset scanout on error in create_kms_dumb_buffer_for_resource()

If an error occured, make sure we reset the scanout object before
leaving, otherwise the next user of this handle will hit the
refcnt == 0 assert.

Fixes: ad4d7ca83324 ("kmsro: Fix renderonly_scanout BO aliasing")
Cc: mesa-stable
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23746>

renderonly: Fix potential NULL deref in the error path

scanout can be NULL.

Fixes: ad4d7ca83324 ("kmsro: Fix renderonly_scanout BO aliasing")
Cc: mesa-stable
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23746>

winsys/panfrost: Fix a scanout resource leak

Use ro->bo_map to alloc scanout and make sure we initialize the refcnt
to one.

This fixes leaking the scanout object and the underlying dumb-buffer.

Fixes: ad4d7ca83324 ("kmsro: Fix renderonly_scanout BO aliasing")
Cc: mesa-stable
Signed-off-by: Robert Beckett <bob.beckett@collabora.com>
Reviewed-by: Italo Nicola <italonicola@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23746>

aux/trace: fix bindless texture dumping

cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23971>

asahi: Use txf_ms for MSAA background programs

Fixes regression in assorted dEQP tests including:

dEQP-EGL.functional.color_clears.multi_context.gles3.rgba8888_window

Fixes: d4424950ace ("asahi: Use txf for background program")
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23998>

agx: Don't leak ssa_to_reg_out

calloc'd in the RA, should be freed in the RA. Identified with valgrind.

Fixes: 6b13616cba2 ("agx: Implement vector live range splitting")
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23998>

asahi: Take ownership of compute shader NIR

Fixes massive leak of compute shader NIR. Identified with valgrind.

Yes, this requires casting away const *.

Yes, Gallium is dumb.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23998>

asahi: Use ralloc harder

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23998>

asahi: Fix scissor_culls_everything check

Account for the possibility that the scissor is outside the render area. Fixes
the usual assertion fail:

   glcts: ../src/gallium/drivers/asahi/agx_state.c:1015:
   agx_upload_viewport_scissor: Assertion `maxx > minx && maxy > miny' failed.

on the following dEQP tests with my conformance build:

  dEQP-GLES3.functional.fragment_ops.scissor.outside_render_line
  dEQP-GLES3.functional.fragment_ops.scissor.outside_render_point
  dEQP-GLES3.functional.fragment_ops.scissor.outside_render_tri

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23998>

asahi: Assert we don't transition shared resources

This is an invariant maintained by all current callers and subtly required for
the BO swapping to work. Assert it to make it obvious.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23998>

asahi: Arrange VS varyings in the correct order

The GPU ABI requires varyings to be grouped as follows:

- Position
- Smooth shaded fp32
- Flat shaded fp32
- Linear shaded fp32
- Smooth shaded fp16
- Flat shaded fp16
- Linear shaded fp16
- Point size

Use the flat shaded mask info we now have in the vertex shader key to
sort things properly, and pass the counts to the hardware.
FP16 is still TODO.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23998>

asahi: Add flat/linear shaded varyings mask to the VS shader key

We need this information in order to arrange varyings properly, which
means we need shader variants. Add this to the shader key, taking the
value from the FS input info.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23998>

asahi: Fix type confusion for fragment shader keys

We can't attempt to access the fs union member if this is not a FS.
That worked so far since there wasn't a VS shader key at all, but we're
about to introduce one.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23998>

asahi: Gather flat/linear shaded input info from uncompiled FS

We need to propagate shading model metadata from the FS to the VS in
order to correctly lay out the uniforms in the right order. This means
we need VS variants depending on this data.

We could use the existing shader info structure, but that applies to
compiled shaders which would introduce a dependency from the VS compile
to the FS compile. This information does not change with FS variants, so
we can introduce an agx_uncompiled_shader_info structure and gather it
early at precompilation time.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23998>

asahi: Identify the separate varying count fields

Flat/goraud/linear and 32/16 need to be specified separately. This
change identifies the new fields but should be a functional no-op.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23998>

agx: Assert that barriers are not used in the preamble

It is nonsensical and confuses the hardware.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23998>

agx: Use nir_opt_shrink_vectors

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23998>

agx: Use nir_opt_shrink_stores

This especially helps with image stores, where we otherwise insert a bunch of
pointless moves to collect a vector even when we know the format only has a
single channel.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23998>

asahi: Remove ; in perf_debug_ctx

Otherwise `if(x) perf_debug_ctx(); else if (y) ...` doesn't work.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23998>

asahi: Identify PBE::sRGB flag

Needed to write out sRGB images correctly.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23998>

asahi: Rename 'Render Target' to 'PBE'

It's used for all PBE operations, including regular image writes, so use the
more general name. Compare the powervr driver.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23998>

asahi: Strip ? in GenXML

Sometimes it's nice to have boolean flags with ? in the name, allow this by
stripping ? when generating the sanitized C name.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23998>

asahi: match_soa: Treat offsets as signed

An offset may be negative, indexing backwards from the array base.
When we right shift an offset by the format shift, we need to use a
signed shift to ensure that the resulting offset is still negative.

Fixes Nautilus faults/pink crashes.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23998>

agx: Smarten discard_agx -> sample_mask lowering

In 97a1bbeaf26 ("agx: Fix discards"), we made our discard lowering very simple,
since we had just discovered the underlying instruction behaviour and needed a
hotfix for misrendering in the wild. Now that we understand the behaviour, we
can do better. There are two potential performance issues with the lowering in
that commit:

1. It generates extra sample_mask instructions. For a shader that has a single
   discard_if at root level, it would generate two instructions

      sample_mask foo, 0
      sample_mask ~0, ~0

   rather than a single

      sample_mask ~0, ~foo

2. It runs depth/stencil testing/updates at the end of the shader, even when it
   could be run immediately after the discard. This might cause pipeline stalls.

The solution is to insert the "trigger testing" sample_mask instruction as soon
after the "discard" instruction as possible, fusing them if they would be next
to each other. There are two cases:

1. The last discard is executed unconditionally. In this case, we can test
   immediately after, unconditionally, and fuse together.

2. The last discard is executed conditionally. In this case, we test in the
   first unconditional block after the discard. Example shader:

   ...
   loop {
      if .. {
         loop {
            discard_if <-- discard here
            ...
         }
      ..
      }
   ...
   }
   <---- we test here
   ...
   store_output

Together this covers all the usual patterns for single-sampled discard. We could
still do better with multisampling, but whatever.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23998>

agx: Add algebraic opt to help with discard lowering

When lowering discards, it will be convenient to generate the pattern:

   (cond ? 255 : 0) ^ 255

Add rules to optimize that to

   (cond ? 0 : 255)

This is not part of the main algebraic optimizer since this lowering happens
late.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23998>