venus: add no_sparse debug option to disable sparse resource support

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24877>

util/driconf: add workarounds for the Chronicles of Riddick

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24567>

zink: fix rewrite_read_as_0 filtering

Fixes: 9e42553ca8d ("zink: use lowered io (kinda) for i/o vars")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24874>

nouveau/mme: Fix a compile warning

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24840>

nvk: Plumb no_prefetch through to the DRM back-end

Instead of using bit 23 of nvk_cmd_push::range for this, pass it as a
separate bool.  This lets us use the actual kernel flag with the new
UAPI.

Reviewed-by: Danilo Krummrich <dakr@redhat.com>
Tested-by: Danilo Krummrich <dakr@redhat.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24840>

drm-uapi: Sync nouveau_drm.h

From https://cgit.freedesktop.org/drm-misc/

    commit 443f9e0b1ab5e3b95abf8606097d13e30e2f2413
    Author: Danilo Krummrich <dakr@redhat.com>
    Date:   Wed Aug 23 20:15:34 2023 +0200

        drm/nouveau: uapi: don't pass NO_PREFETCH flag implicitl

Reviewed-by: Danilo Krummrich <dakr@redhat.com>
Tested-by: Danilo Krummrich <dakr@redhat.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24840>

util/rb-tree: Fix typo in comment

Trivial.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24856>

util/rb-tree: Return the actual first node from rb_tree_search

Previously rb_tree_search would return the first node encountered, but
that may not be the first node that would be encoutnered by, say,
rb_tree_foreach.

Two test cases are added.

v2: Add more curly braces. Suggested by Faith.

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24856>

ci/iris: add GL46.arrays_of_arrays_gl.SizedDeclarationsPrimitive timeout

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24870>

tu: Pass real size of prime buffers to allocator

The msm driver reserves the actual DMABUF size in the memory map, while
TU can request smaller memory chunk to be allocated. This potentially
can lead to a situation when next allocation IOVA will be in the middle
of the address space which is reserved for the DMABUF. Pass the
`real_size' to TU allocator instead, so that kernel and userspace have
the same picture of memory allocations.

Cc: mesa-stable
Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24861>

treewide: Also handle struct nir_builder form

Via Coccinelle patch:

    @def@
    typedef bool;
    typedef nir_builder;
    typedef nir_instr;
    typedef nir_def;
    identifier fn, instr, intr, x, builder, data;
    @@

    static fn(struct nir_builder* builder,
    -nir_instr *instr,
    +nir_intrinsic_instr *intr,
    ...)
    {
    (
    -   if (instr->type != nir_instr_type_intrinsic)
    -      return false;
    -   nir_intrinsic_instr *intr = nir_instr_as_intrinsic(instr);
    |
    -   nir_intrinsic_instr *intr = nir_instr_as_intrinsic(instr);
    -   if (instr->type != nir_instr_type_intrinsic)
    -      return false;
    )

    <...
    (
    -instr->x
    +intr->instr.x
    |
    -instr
    +&intr->instr
    )
    ...>

    }

    @pass depends on def@
    identifier def.fn;
    expression shader, progress;
    @@

    (
    -nir_shader_instructions_pass(shader, fn,
    +nir_shader_intrinsics_pass(shader, fn,
    ...)
    |
    -NIR_PASS_V(shader, nir_shader_instructions_pass, fn,
    +NIR_PASS_V(shader, nir_shader_intrinsics_pass, fn,
    ...)
    |
    -NIR_PASS(progress, shader, nir_shader_instructions_pass, fn,
    +NIR_PASS(progress, shader, nir_shader_intrinsics_pass, fn,
    ...)
    )

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24852>

treewide: Use nir_shader_intrinsic_pass sometimes

This converts a lot of trivial passes. Nice boilerplate deletion. Via Coccinelle
patch (with a small manual fix-up for panfrost where coccinelle got confused by
genxml + ninja clang-format squashed in, and for Zink because my semantic patch
was slightly buggy).

    @def@
    typedef bool;
    typedef nir_builder;
    typedef nir_instr;
    typedef nir_def;
    identifier fn, instr, intr, x, builder, data;
    @@

    static fn(nir_builder* builder,
    -nir_instr *instr,
    +nir_intrinsic_instr *intr,
    ...)
    {
    (
    -   if (instr->type != nir_instr_type_intrinsic)
    -      return false;
    -   nir_intrinsic_instr *intr = nir_instr_as_intrinsic(instr);
    |
    -   nir_intrinsic_instr *intr = nir_instr_as_intrinsic(instr);
    -   if (instr->type != nir_instr_type_intrinsic)
    -      return false;
    )

    <...
    (
    -instr->x
    +intr->instr.x
    |
    -instr
    +&intr->instr
    )
    ...>

    }

    @pass depends on def@
    identifier def.fn;
    expression shader, progress;
    @@

    (
    -nir_shader_instructions_pass(shader, fn,
    +nir_shader_intrinsics_pass(shader, fn,
    ...)
    |
    -NIR_PASS_V(shader, nir_shader_instructions_pass, fn,
    +NIR_PASS_V(shader, nir_shader_intrinsics_pass, fn,
    ...)
    |
    -NIR_PASS(progress, shader, nir_shader_instructions_pass, fn,
    +NIR_PASS(progress, shader, nir_shader_intrinsics_pass, fn,
    ...)
    )

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24852>

ci: switch to 6.4 kernel, improving Adreno 660 reliability

Second argument is to leave Linux 6.3 which is EOL.

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24815>

ci/freedreno: There is only one King of Town.

Drop -r1, which got removed in the Linux 6.4.

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24815>

ci/piglit: add extra space on top to prevent single quote getting into URL

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24815>

zink: update some compute caps

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24859>

zink: fix global stores

We have to cast the value if the type doesn't match.

Fixes: ddc5c304899 ("zink: handle global and scratch vars")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24859>

zink: fix source type in load/store scratch

Fixes: ddc5c304899 ("zink: handle global and scratch vars")
Signed-off-by: Karol Herbst <git@karolherbst.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24859>

zink: use Aligned with global load/store ops

this is required by spec

Fixes: ddc5c304899 ("zink: handle global and scratch vars")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24859>

zink: handle global atomic intrinsics

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24859>

hasvk/android: remove numFds check

Change required for compatibility with minigbm gralloc4
due to gralloc handle having DRV_MAX_FDS = (DRV_MAX_PLANES + 1)

https://android.googlesource.com/platform/external/minigbm/+/refs/tags/android-13.0.0_r18/cros_gralloc/cros_gralloc_handle.h#14

Cc: "22.3" mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7807
Reviewed-by: Roman Stratiienko <r.stratiienko@gmail.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20231>

anv/android: remove numFds check

Change required for compatibility with minigbm gralloc4
due to gralloc handle having DRV_MAX_FDS = (DRV_MAX_PLANES + 1)

https://android.googlesource.com/platform/external/minigbm/+/refs/tags/android-13.0.0_r18/cros_gralloc/cros_gralloc_handle.h#14

Cc: "22.2" "22.3" mesa-stable
Reviewed-by: Roman Stratiienko <r.stratiienko@gmail.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20231>

anv: Don't reject Android image format if external props not supplied

anv_GetPhysicalDeviceImageFormatProperties2 returns 'not supported' if an
Android hardware buffer external memory handle type is specified, but no
external image format properties output struct is supplied. This struct is
optional, so we should populate it if present, but return successfully
either way.

This fixes an error when using ANV with hwui, which otherwise prevents the
system from booting.[1]

[1] https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/libs/hwui/renderthread/VulkanSurface.cpp;l=271;drc=ad3fb95aa2fe0be59d3e991ddc883592ab5542bc

Signed-off-by: Chris Spencer <spencercw@gmail.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24844>

vulkan/wsi/wayland: add support for IMMEDIATE

Use the tearing-control-unstable-v1 protocol to indicate to the
Wayland compositor that tearing is acceptable.

Signed-off-by: Simon Ser <contact@emersion.fr>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18268>

wayland: enable use of wayland-protocols as a subproject

This allows developers to link subprojects/wayland-protocols/ to
a repository checkout. Useful when adding support for new protocols.

Signed-off-by: Simon Ser <contact@emersion.fr>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18268>

intel/dev: Add more RPL PCI IDs

Ref: https://patchwork.freedesktop.org/patch/553646/?series=122712&rev=1
Cc: mesa-stable
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24820>

intel/dev: Use RPL-U name on RPL-U devices

Ref: https://patchwork.freedesktop.org/patch/553646/?series=122712&rev=1
Cc: mesa-stable
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24820>

radv: stop declaring unused SGPR arguments for PS epilogs

ACO no longer requires these arguments.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24838>

radv: fix the per-patch data offset when TES isn't linked with TCS

When TCS and TES aren't linked together and TCS exports unused outputs,
the per-patch data offset needs to be adjusted. This is similar to the
LS-HS vertex stride when VS and TCS aren't linked together.

This fixes a bunch of failures by forcing the driver to use TCS epilogs.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24776>

mesa: fix some TexParameter and SamplerParameter cases

EXT extension was added without tests so these functions did
not work properly.

Fixes: 799710be889 ("mesa: Add EXT_texture_mirror_clamp_to_edge to extension table")
Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24845>

compiler/glsl: Move glsl_print_type from glsl_types.* to ir_print_visitor.cpp

glsl_print_type only referenced in ir_print_visitor.cpp
there is no need expose it

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24824>

compiler: use 4 instead ATOMIC_COUNTER_SIZE in glsl_types.h to avoid #include "mesa/main/config.h"

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24824>

d3d12: replace use of MAX_VERTEX_STREAMS with PIPE_MAX_VERTEX_STREAMS

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24824>

intel/brw: use 4 instead of MAX_VERTEX_STREAMS to avoid #include "mesa/main/config.h"

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24824>

sfn: Use 4 instead of ATOMIC_COUNTER_SIZE

../../src/gallium/drivers/r600/sfn/sfn_nir.cpp:458:59: error: ‘ATOMIC_COUNTER_SIZE’ was not declared in this scope

../../src/gallium/drivers/r600/sfn/sfn_shader.cpp:609:53: error: ‘ATOMIC_COUNTER_SIZE’ was not declared in this scope

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24824>

tu: Workaround bionic _SC_LEVEL1_DCACHE_LINESIZE

Bionic just returns a hard-coded 0, which isn't helpful.  But
fortunately on aarch64 it is easy enough just to read the value
ourselves.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24829>

intel/fs: Dump IR for pre-RA scheduler modes in DEBUG_OPTIMIZER

This lets us more easily compare and contrast the various scheduling
options that the compiler considered.

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24707>

intel/fs: Pick the lowest register pressure schedule when spilling

We try various pre-RA scheduler modes and see if any of them allow
us to register allocate without spilling.  If all of them spill,
however, we left it on the last mode: LIFO.  This is unfortunately
sometimes significantly worse than other modes (such as "none").

This patch makes us instead select the pre-RA scheduling mode that
gives the lowest register pressure estimate, if none of them manage
to avoid spilling.  The hope is that this scheduling will spill the
least out of all of them.

fossil-db stats (on Alchemist) speak for themselves:

    Totals:
    Instrs: 197297092 -> 195326552 (-1.00%); split: -1.02%, +0.03%
    Cycles: 14291286956 -> 14303502596 (+0.09%); split: -0.55%, +0.64%
    Spill count: 190886 -> 129204 (-32.31%); split: -33.01%, +0.70%
    Fill count: 361408 -> 225038 (-37.73%); split: -39.17%, +1.43%
    Scratch Memory Size: 12935168 -> 10868736 (-15.98%); split: -16.08%, +0.10%

    Totals from 1791 (0.27% of 668386) affected shaders:
    Instrs: 7628929 -> 5658389 (-25.83%); split: -26.50%, +0.67%
    Cycles: 719326691 -> 731542331 (+1.70%); split: -10.95%, +12.65%
    Spill count: 110627 -> 48945 (-55.76%); split: -56.96%, +1.20%
    Fill count: 221560 -> 85190 (-61.55%); split: -63.89%, +2.34%
    Scratch Memory Size: 4471808 -> 2405376 (-46.21%); split: -46.51%, +0.30%

Improves performance when using XeSS in Cyberpunk 2077 by 90% on A770.
Improves performance of Borderlands 3 by 1.54% on A770.

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24707>

intel/fs: Make helpers for saving/restoring instruction order

This moves a bit of code out of a large function, but also lets us reuse
it a few extra places in the next commit.

I opted to stop using ralloc here since this is short-lived data that
doesn't need to stick around for the rest of the compile, and it's easy
enough to free.

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24707>

intel/fs: Index scheduler mode string table by mode enum

pre_modes[] is an array with the modes ordered in our desired
preference.  scheduler_mode_name[] was also in that order, and the two
had to be kept in sync.  This is a little silly; we should just have
a mode enum -> string table and look it up via the enum.

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24707>

intel/compiler: Move SCHEDULE_NONE handling into schedule_instructions()

I'm going to introduce another call site for this function, and just
handling SCHEDULE_NONE in the scheduler itself makes more sense than
duplicating the logic.

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24707>

intel/fs: Account for payload GRFs when calculating register pressure

The register pressure analysis I wrote in 2013 only considered VGRFs,
and not other GRFs, such as payload registers and push constants.  We
need to consider those too, because payload registers definitely occupy
space and add to pressure.

In 2015, Connor already made the scheduler account for this, so the only
real use for this is in shader statistic dumps and optimizer printouts.
But we should make it more accurate.  (We will use it in more places
shortly, a few commits from now.)

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24707>

docs: Followup to !24636

Update docs to reflect that EGL_ANDROID_blob_cache is now available even
if on-disk caching is disabled.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24853>

Revert "radeonsi/vcn: add an exception of field case for h264 decoding"

This change causes page faults when playing corrupted video from the
bugreport. The original issue have now been resolved in firmware.

This reverts commit bfce57c7a5ba62d8e6f65addb2df136cab603a68.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9210

Reviewed-by: Ruijing Dong <ruijing.dong@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24836>

aco: combine a | ~b to bfi(b, a, -1)

Somehow I missed this when writing the a & ~b patch.

Foz-DB Navi21:
Totals from 1591 (1.20% of 132657) affected shaders:
Instrs: 2316379 -> 2315940 (-0.02%)
CodeSize: 12524240 -> 12528724 (+0.04%); split: -0.00%, +0.04%
Latency: 45393195 -> 45389285 (-0.01%); split: -0.01%, +0.00%
InvThroughput: 8658991 -> 8657944 (-0.01%); split: -0.01%, +0.00%
Copies: 135777 -> 135778 (+0.00%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24505>

nir: Remove dead nir_const_value variables

nir_const_value variables in nir_const_value_for_int and
nor_const_value_for_uint are unused resulting in unnecessary dead code.
The unused-variable warning has been suppressed by the memset following
their declarations.

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24851>

panfrost/ci: revert Disable T720

Tested, it was just some unlucky coincidence it didn't worked, the
device queue is almost empty.

This reverts commit b0f02973d7d12b2755666a8822cfc25c680bd27b.

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24850>

zink: copy some cs shader properties to the program struct

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24835>

lavapipe: Lock around CSO destroys

They can race in llvmpipe_register_shader.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9680
Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24827>

svga: fix stride used in vertex declaration

The stride for each vertex buffer should come from the corresponding
vertex element structure.

Fixes piglit/glretrace regressions running on svga vgpu9 device.

Fixes: 76725452239 ("gallium: move vertex stride to CSO")

Reviewed-by: Neha Bhende <bhenden@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24841>

asahi: Fix shader stage dirtying

Now this is actually doing what I expect. drawoverhead #1 score more than
doubles (6091->13375).

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Dirty the shader stage when the shader changes

We need to re-emit all descriptors in this case for correctness. Avoids
regressions from the following commit.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Dirty track VBOs + blend const separately

We're staging everything anyway.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Use proper dirty tracking for VBOs

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Use finer dirty tracking for blend constant

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Decouple sysval lowering from uniform assignment

For merging shader states, we'll need to lower sysvals separately for each
shader but assign uniforms together for the final merged shader. The easiest way
to do that is to decouple the lowering of sysvals to driver uniform reads, from
the assignment of driver uniform reads to actual uniform registers.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Put unuploaded uniforms on the batch

Less copying needed this way.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Extract sampler upload

Dirty track it.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Add real per-stage dirty flags

Instead of just using ~0 as a stub todo.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Upload a single draw_uniforms per draw

Not per stage per draw. This is less frequent.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Extract agx_upload_textures

By uploading textures ahead-of-time, we can upload uniforms ahead-of-time too.
This will also allow some overhead shaving optimizations, I guess.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Collapse grid_info

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Split out per-stage sysvals

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Add sysval tables for each shader stage

So we can model the descriptors of each shader stage independently, as required
for merged shaders.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Move UBO lowering into GL driver

In Vulkan, UBOs are lowered by nir_lower_explicit_io, and the ubo_base_agx
sysval is unused (since it doesn't handle descriptor sets). That makes the UBO
lowering GL-only and hence belongs with the GL driver rather than the compiler.
This lets us delete the ubo_base_agx sysval.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

nir,asahi: Remove texture_base_agx

Doing a descriptor crawl with binding tables requires a real binding table in
the shader, which won't work for VK or merged shader stages in GL. Instead,
let's lower anything that needs a crawl to bindless in the driver, so the
compiler code doesn't need to know anything about descriptor binding models.
That gets rid of the texture_base_agx sysval, which is problematic when there
are multiple descriptor sets worth of textures.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Add helper returning if a descriptor crawl is needed

For agx_nir_lower_texture to lower to a descriptor crawl, the driver needs to
make sure the address of the descriptor is available. This means a slightly
different code path should be used in the driver. Rather than the drivers
needing to know what exactly will be lowered, add a helper in the same file as
agx_nir_lower_texture that returns whether descriptor-based lowering will be
needed so the driver can act appropriately.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Do some texture lowering early

We want to make the implicit txs in operations explicit before lower_bindings so
lower_bindings knows to force bindless.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Add missing LOD source for agx_meta's txfs

These would be inserted by nir_lower_tex anyway, but we shouldn't be relying on
that behaviour for the meta shaders when we can just create the correct thing
from the start.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Do not fence write-only images

Reduces fencing significantly.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx/fence_images: Use intrinsics_pass

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Add get_query_address helper

This is the counterpart of get_oq_index for non-occlusion hardware queries.
These are not tracked with occlusion queries, since occlusion query allocations
are limited, and they are not based on indexing but rather general
batch-allocated space.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Add non-occlusion query tracking

For other GPU queries, handled similarly.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Sync when beginning a query

Otherwise batch->writer might be non-null. Fixes Piglit occlusion_query_conform
(which I think regressed when we added proper syncing).

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Only touch batch->occlusion_queries for occlusion

We will soon have other types of queries with non-null writers.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Refactor agx_get_query_result

In preparation for other types of GPU queries.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Simplify occlusion query batch tracking

Yes, this means we now lie to the app. There's nothing more in the spirit of
dumb OpenGL features than lying!

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Generalize query logic

We will need to do the same flushing dance for non-occlusion GPU queries.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Use 16-bit reg for pixel_coord

Mistake during IR translation, this is 16-bit in NIR.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Preserve atomic ops when rewriting image to bindless

Bug fix on its own, and prevents regressions from using bindless more.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Clear image_array after lowering

We lower to access to a non-array 2D image, so we need to update the image_array
flag when we lower or otherwise we get an incorrect 2D Array store to a 2D image
which the hardware doesn't want.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Clear sample count after lowering MSAA

Pedantic.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Pass layer stride in pixels, not elements

We do all the math in pixels and only multiply by the sample count at the end,
meaning the layer stride needs to be in terms of pixels (not samples) for
correct addressing of multisample array images in our texture lowering. This is
particularly used for lowering the multisample array stores we get from eMRT
with multisampled layered framebuffers.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Use local_size from compiler directly

This avoids an unnecessary trip through agx_uncompiled_shader.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Report local_size from compiler

So we can add more shared in the compiler.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi/decode: Turn assert into error

To allow us to debug broken fetches.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Advertise OpenGL ES 3.1!

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Implement imul_high

Like umul_high. Fixes
dEQP-VK.spirv_assembly.instruction.compute.mul_extended.signed_16bit

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Convert 8-bit comparisons

Fixes dEQP-VK.spirv_assembly.type.vec3.i8.slessthan_frag

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Handle b2i8

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Allow loop headers without later preds

These happen for loops that break after exactly 1 iteration, unconditionally.
Fixes validation splat in dEQP-VK.glsl.switch.conditional_fall_through_2_uniform_vertex

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx/lower_vbo: Handle nonzero component

Fixes dEQP-VK.glsl.440.linkage.varying.component.vert_in.vec2.as_float_float

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Ensure to lower 1D image load/store to 2D

This was missing from lower_images.
This fix a good chunk of Vulkan deQP failures with 1D images.

Signed-off-by: Mary <mary@mary.zone>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Don't blow up when lowering textures twice

Hard to avoid with GS.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Keep drawoverhead from OOMing itself

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Assert vertex_id, instance_id are VS-only

We can get them in other shaders transiently due to merging shader stages, but
they need to be lowered since the hardware versions only exist in the hardware
VS. So the compiler should never see them outside VS.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Copy CSO stride

Fixes: 76725452239 ("gallium: move vertex stride to CSO")
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Use more barriers

Fixes flakiness in
KHR-GLES31.core.texture_buffer.texture_buffer_atomic_functions.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Fix extraneous bits with b2b32

From expanding mov_imm to take a 64-bit immediate. this worked by accident
before. Fixes brief regression in
dEQP-GLES31.functional.compute.shared_var.basic_type.bvec3.

Fixes: dbd98aa24d1 ("agx: Fix 64-bit immediate moves")
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

nir: Add load_sysval_agx intrinsic

For merging shader stages, it will be useful to express a load from an explicit
GL "descriptor set", so we can represent things like UBO loads with merged
shaders where UBOs can come from either stage. To do so, we add an intrinsic
representing a load from the driver's uniform tables, indexed like "descriptor
sets" with "bindings".

In principle, a layered GL-on-Vulkan implementation would use literal descriptor
sets for each stage, so I feel comfortable with the analogy here.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>