intel/fs: Dump IR for pre-RA scheduler modes in DEBUG_OPTIMIZER

This lets us more easily compare and contrast the various scheduling
options that the compiler considered.

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24707>

intel/fs: Pick the lowest register pressure schedule when spilling

We try various pre-RA scheduler modes and see if any of them allow
us to register allocate without spilling.  If all of them spill,
however, we left it on the last mode: LIFO.  This is unfortunately
sometimes significantly worse than other modes (such as "none").

This patch makes us instead select the pre-RA scheduling mode that
gives the lowest register pressure estimate, if none of them manage
to avoid spilling.  The hope is that this scheduling will spill the
least out of all of them.

fossil-db stats (on Alchemist) speak for themselves:

    Totals:
    Instrs: 197297092 -> 195326552 (-1.00%); split: -1.02%, +0.03%
    Cycles: 14291286956 -> 14303502596 (+0.09%); split: -0.55%, +0.64%
    Spill count: 190886 -> 129204 (-32.31%); split: -33.01%, +0.70%
    Fill count: 361408 -> 225038 (-37.73%); split: -39.17%, +1.43%
    Scratch Memory Size: 12935168 -> 10868736 (-15.98%); split: -16.08%, +0.10%

    Totals from 1791 (0.27% of 668386) affected shaders:
    Instrs: 7628929 -> 5658389 (-25.83%); split: -26.50%, +0.67%
    Cycles: 719326691 -> 731542331 (+1.70%); split: -10.95%, +12.65%
    Spill count: 110627 -> 48945 (-55.76%); split: -56.96%, +1.20%
    Fill count: 221560 -> 85190 (-61.55%); split: -63.89%, +2.34%
    Scratch Memory Size: 4471808 -> 2405376 (-46.21%); split: -46.51%, +0.30%

Improves performance when using XeSS in Cyberpunk 2077 by 90% on A770.
Improves performance of Borderlands 3 by 1.54% on A770.

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24707>

intel/fs: Make helpers for saving/restoring instruction order

This moves a bit of code out of a large function, but also lets us reuse
it a few extra places in the next commit.

I opted to stop using ralloc here since this is short-lived data that
doesn't need to stick around for the rest of the compile, and it's easy
enough to free.

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24707>

intel/fs: Index scheduler mode string table by mode enum

pre_modes[] is an array with the modes ordered in our desired
preference.  scheduler_mode_name[] was also in that order, and the two
had to be kept in sync.  This is a little silly; we should just have
a mode enum -> string table and look it up via the enum.

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24707>

intel/compiler: Move SCHEDULE_NONE handling into schedule_instructions()

I'm going to introduce another call site for this function, and just
handling SCHEDULE_NONE in the scheduler itself makes more sense than
duplicating the logic.

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24707>

intel/fs: Account for payload GRFs when calculating register pressure

The register pressure analysis I wrote in 2013 only considered VGRFs,
and not other GRFs, such as payload registers and push constants.  We
need to consider those too, because payload registers definitely occupy
space and add to pressure.

In 2015, Connor already made the scheduler account for this, so the only
real use for this is in shader statistic dumps and optimizer printouts.
But we should make it more accurate.  (We will use it in more places
shortly, a few commits from now.)

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24707>

docs: Followup to !24636

Update docs to reflect that EGL_ANDROID_blob_cache is now available even
if on-disk caching is disabled.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24853>

Revert "radeonsi/vcn: add an exception of field case for h264 decoding"

This change causes page faults when playing corrupted video from the
bugreport. The original issue have now been resolved in firmware.

This reverts commit bfce57c7a5ba62d8e6f65addb2df136cab603a68.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9210

Reviewed-by: Ruijing Dong <ruijing.dong@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24836>

aco: combine a | ~b to bfi(b, a, -1)

Somehow I missed this when writing the a & ~b patch.

Foz-DB Navi21:
Totals from 1591 (1.20% of 132657) affected shaders:
Instrs: 2316379 -> 2315940 (-0.02%)
CodeSize: 12524240 -> 12528724 (+0.04%); split: -0.00%, +0.04%
Latency: 45393195 -> 45389285 (-0.01%); split: -0.01%, +0.00%
InvThroughput: 8658991 -> 8657944 (-0.01%); split: -0.01%, +0.00%
Copies: 135777 -> 135778 (+0.00%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24505>

nir: Remove dead nir_const_value variables

nir_const_value variables in nir_const_value_for_int and
nor_const_value_for_uint are unused resulting in unnecessary dead code.
The unused-variable warning has been suppressed by the memset following
their declarations.

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24851>

panfrost/ci: revert Disable T720

Tested, it was just some unlucky coincidence it didn't worked, the
device queue is almost empty.

This reverts commit b0f02973d7d12b2755666a8822cfc25c680bd27b.

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24850>

zink: copy some cs shader properties to the program struct

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24835>

lavapipe: Lock around CSO destroys

They can race in llvmpipe_register_shader.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9680
Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24827>

svga: fix stride used in vertex declaration

The stride for each vertex buffer should come from the corresponding
vertex element structure.

Fixes piglit/glretrace regressions running on svga vgpu9 device.

Fixes: 76725452239 ("gallium: move vertex stride to CSO")

Reviewed-by: Neha Bhende <bhenden@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24841>

asahi: Fix shader stage dirtying

Now this is actually doing what I expect. drawoverhead #1 score more than
doubles (6091->13375).

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Dirty the shader stage when the shader changes

We need to re-emit all descriptors in this case for correctness. Avoids
regressions from the following commit.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Dirty track VBOs + blend const separately

We're staging everything anyway.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Use proper dirty tracking for VBOs

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Use finer dirty tracking for blend constant

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Decouple sysval lowering from uniform assignment

For merging shader states, we'll need to lower sysvals separately for each
shader but assign uniforms together for the final merged shader. The easiest way
to do that is to decouple the lowering of sysvals to driver uniform reads, from
the assignment of driver uniform reads to actual uniform registers.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Put unuploaded uniforms on the batch

Less copying needed this way.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Extract sampler upload

Dirty track it.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Add real per-stage dirty flags

Instead of just using ~0 as a stub todo.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Upload a single draw_uniforms per draw

Not per stage per draw. This is less frequent.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Extract agx_upload_textures

By uploading textures ahead-of-time, we can upload uniforms ahead-of-time too.
This will also allow some overhead shaving optimizations, I guess.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Collapse grid_info

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Split out per-stage sysvals

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Add sysval tables for each shader stage

So we can model the descriptors of each shader stage independently, as required
for merged shaders.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Move UBO lowering into GL driver

In Vulkan, UBOs are lowered by nir_lower_explicit_io, and the ubo_base_agx
sysval is unused (since it doesn't handle descriptor sets). That makes the UBO
lowering GL-only and hence belongs with the GL driver rather than the compiler.
This lets us delete the ubo_base_agx sysval.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

nir,asahi: Remove texture_base_agx

Doing a descriptor crawl with binding tables requires a real binding table in
the shader, which won't work for VK or merged shader stages in GL. Instead,
let's lower anything that needs a crawl to bindless in the driver, so the
compiler code doesn't need to know anything about descriptor binding models.
That gets rid of the texture_base_agx sysval, which is problematic when there
are multiple descriptor sets worth of textures.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Add helper returning if a descriptor crawl is needed

For agx_nir_lower_texture to lower to a descriptor crawl, the driver needs to
make sure the address of the descriptor is available. This means a slightly
different code path should be used in the driver. Rather than the drivers
needing to know what exactly will be lowered, add a helper in the same file as
agx_nir_lower_texture that returns whether descriptor-based lowering will be
needed so the driver can act appropriately.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Do some texture lowering early

We want to make the implicit txs in operations explicit before lower_bindings so
lower_bindings knows to force bindless.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Add missing LOD source for agx_meta's txfs

These would be inserted by nir_lower_tex anyway, but we shouldn't be relying on
that behaviour for the meta shaders when we can just create the correct thing
from the start.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Do not fence write-only images

Reduces fencing significantly.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx/fence_images: Use intrinsics_pass

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Add get_query_address helper

This is the counterpart of get_oq_index for non-occlusion hardware queries.
These are not tracked with occlusion queries, since occlusion query allocations
are limited, and they are not based on indexing but rather general
batch-allocated space.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Add non-occlusion query tracking

For other GPU queries, handled similarly.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Sync when beginning a query

Otherwise batch->writer might be non-null. Fixes Piglit occlusion_query_conform
(which I think regressed when we added proper syncing).

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Only touch batch->occlusion_queries for occlusion

We will soon have other types of queries with non-null writers.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Refactor agx_get_query_result

In preparation for other types of GPU queries.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Simplify occlusion query batch tracking

Yes, this means we now lie to the app. There's nothing more in the spirit of
dumb OpenGL features than lying!

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Generalize query logic

We will need to do the same flushing dance for non-occlusion GPU queries.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Use 16-bit reg for pixel_coord

Mistake during IR translation, this is 16-bit in NIR.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Preserve atomic ops when rewriting image to bindless

Bug fix on its own, and prevents regressions from using bindless more.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Clear image_array after lowering

We lower to access to a non-array 2D image, so we need to update the image_array
flag when we lower or otherwise we get an incorrect 2D Array store to a 2D image
which the hardware doesn't want.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Clear sample count after lowering MSAA

Pedantic.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Pass layer stride in pixels, not elements

We do all the math in pixels and only multiply by the sample count at the end,
meaning the layer stride needs to be in terms of pixels (not samples) for
correct addressing of multisample array images in our texture lowering. This is
particularly used for lowering the multisample array stores we get from eMRT
with multisampled layered framebuffers.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Use local_size from compiler directly

This avoids an unnecessary trip through agx_uncompiled_shader.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Report local_size from compiler

So we can add more shared in the compiler.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi/decode: Turn assert into error

To allow us to debug broken fetches.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Advertise OpenGL ES 3.1!

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Implement imul_high

Like umul_high. Fixes
dEQP-VK.spirv_assembly.instruction.compute.mul_extended.signed_16bit

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Convert 8-bit comparisons

Fixes dEQP-VK.spirv_assembly.type.vec3.i8.slessthan_frag

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Handle b2i8

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Allow loop headers without later preds

These happen for loops that break after exactly 1 iteration, unconditionally.
Fixes validation splat in dEQP-VK.glsl.switch.conditional_fall_through_2_uniform_vertex

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx/lower_vbo: Handle nonzero component

Fixes dEQP-VK.glsl.440.linkage.varying.component.vert_in.vec2.as_float_float

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Ensure to lower 1D image load/store to 2D

This was missing from lower_images.
This fix a good chunk of Vulkan deQP failures with 1D images.

Signed-off-by: Mary <mary@mary.zone>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Don't blow up when lowering textures twice

Hard to avoid with GS.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Keep drawoverhead from OOMing itself

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Assert vertex_id, instance_id are VS-only

We can get them in other shaders transiently due to merging shader stages, but
they need to be lowered since the hardware versions only exist in the hardware
VS. So the compiler should never see them outside VS.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

asahi: Copy CSO stride

Fixes: 76725452239 ("gallium: move vertex stride to CSO")
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Use more barriers

Fixes flakiness in
KHR-GLES31.core.texture_buffer.texture_buffer_atomic_functions.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

agx: Fix extraneous bits with b2b32

From expanding mov_imm to take a 64-bit immediate. this worked by accident
before. Fixes brief regression in
dEQP-GLES31.functional.compute.shared_var.basic_type.bvec3.

Fixes: dbd98aa24d1 ("agx: Fix 64-bit immediate moves")
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

nir: Add load_sysval_agx intrinsic

For merging shader stages, it will be useful to express a load from an explicit
GL "descriptor set", so we can represent things like UBO loads with merged
shaders where UBOs can come from either stage. To do so, we add an intrinsic
representing a load from the driver's uniform tables, indexed like "descriptor
sets" with "bindings".

In principle, a layered GL-on-Vulkan implementation would use literal descriptor
sets for each stage, so I feel comfortable with the analogy here.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24847>

anv: Advertise Vulkan 1.3 on Android 13

Older versions of Android rejected newer versions of Vulkan,[1] but Android
13 devices are 'strongly recommended' to support Vulkan 1.3.[2]

[1] https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4781
[2] https://source.android.com/docs/compatibility/13/android-13-cdd#7142_vulkan

Signed-off-by: Chris Spencer <spencercw@gmail.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24816>

panfrost/ci: Disable T720

Timing out and preventing merges. The lab needs to fix the devices and likely
add more coverage before this can be re-enabled.

The rest of the Panfrost/Collabora farm seems ok, it's just the T720 devices.

See https://gitlab.freedesktop.org/mesa/mesa/-/jobs/47910575 which at the time
of pushing has been waiting for t720 job to start for 45 minutes. This will
allow us to merge code today.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

anv: Do fast clear color initialization more delicately

Fixes: b4198e79 ("anv/cmd_buffer: Initalize the clear color struct for CNL+")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9464
Signed-off-by: Sviatoslav Peleshko <sviatoslav.peleshko@globallogic.com>
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24768>

intel/isl: Don't over-allocate CLEAR_COLOR size to use whole cache line

At the time this was added to fix some test failures. But it seems that
the failures were happening due to missing cache flushes, so
this extra space is no longer neccessary.

Fixes: 37b4eacc ("intel/isl: Resize clear color buffer to full cacheline")
Signed-off-by: Sviatoslav Peleshko <sviatoslav.peleshko@globallogic.com>
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24768>

aco: fix u2f16 with 32bit input

The vulkan spec says all conversions are correctly rounded, so if the input
is larger than the largest fp16 value, we need to return MAX_FLOAT/inf
instead of cutting off the msbs.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24826>

aco: add adjust_bpermute_dst helper

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24693>

aco: clarify bpermute pseudo opcode names

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24693>

aco: fix p_bpermute_gfx6's exec save/restore with wave32

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24693>

aco: fix p_bpermute_gfx6 with input at non-zero byte

Same as the other bpermute pseudo instructions.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24693>

anv/android: Add support for AHARDWAREBUFFER_FORMAT_YV12

The default MediaCodec software video decoder returns frames in this
format.

Signed-off-by: Chris Spencer <spencercw@gmail.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24388>

anv/android: Fix importing hardware buffers with planar formats

Currently, we try to fetch the color aspect of the format and convert that
to an ISL format, which is then used to convert the pixel stride to bytes.
This does not work with planar formats because they don't have a color
aspect, and the planes can be of different sizes anyway, so may not have
the same byte stride. Change to calculate the stride individually for each
plane.

Signed-off-by: Chris Spencer <spencercw@gmail.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24388>

aco: rework printing shader stages

To avoid printing "unknown" for shader object when eg. VS and TCS
are compiled separately.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24810>

CODEOWNERS: Add @flynnjiang for VirGL video

Signed-off-by: Feng Jiang <jiangfeng@kylinos.cn>
Acked-by: Corentin Noël <corentin.noel@collabora.com>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24570>

radv: advertise NV_device_generated_commands_compute

This extension introduces a token for implementing DGC compute, it's
only intended to be used by vkd3d-proton.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24275>

radv: allow DGC on the compute queue

DGC cmdbuf on ACE are executed as IB1 without chaining because IB2
isn't supported on ACE.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24275>

radv: implement NV_device_generated_commands_compute

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24275>

radv: prepare radv_prepare_dgc() for DGC compute

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24275>

radv: prepare radv_get_sequence_size() for DGC compute

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24275>

radv: track the pipeline bind point for indirect commands layout

This will be used to implement DGC compute.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24275>

zink: sanitize optimal keys

shader keys represent pipeline states which trigger variants, but not
all shaders are affected by certain states

this adds some sanitizing for the optimal path to ignore shader variants
which won't have any effect for the currently bound shaders, thus reducing
the number of pipelines compiled (both unoptimized and optimized)

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24831>

zink: use the "set" optimal key for prog last_variant_hash for consistency

this is the key that has been set during program update for reuse
everywhere else, and everything else in the driver uses it

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24831>

zink: don't start multiple cache jobs for the same program

if there's already a cache job in flight then starting a second one
is illegal

cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24831>

frontends/va: Add BT.709 as supported postproc color standard

Reviewed-by: Leo Liu <leo.liu@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24788>

gallium/auxiliary/vl: Don't set csc matrix in video_buffer/rgb_to_yuv_layer

It's now handled in va frontend instead.

Reviewed-by: Leo Liu <leo.liu@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24788>

frontends/va: Set csc matrix in postproc

Set correct matrix according to format, color standard and range.

Change default value for color range when not explicitly specified.
Use limited range for YUV and full range for RGB.

This also adds support for converting from full range YUV to RGB.

Reviewed-by: Leo Liu <leo.liu@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24788>

gallium/auxiliary/vl: Add BT.709 full csc matrix

Used for converting from full range YUV.

Reviewed-by: Leo Liu <leo.liu@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24788>

blorp: Drop unnecessary assertions in blorp_can_hiz_clear_depth

We already checks for the alignment and the multislice surface, we don't
need to add assertions around those two.

fixes: 37fcbb375cb ("blorp: Disable unaligned partial HIZ fast clears for HIZ_CCS too")
closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9684

Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Tested-by: Mark Janes <markjanes@swizzler.org>
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24837>

util: Decouple disk cache from EGL_ANDROID_blob_cache

Just because the user / system-integrater doesn't want shader disk
cache, doesn't mean they don't want EGL_ANDROID_blob_cache to work.
We've kind of already solved this for the android case, so just
generalize that solution.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9520
Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24636>

intel/fs: Simplify compute_start_end().

Now that we have moved the screening up, we can simplify the code.  No
change in shader-db steam performance, n=10.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24702>

intel/fs: Move the defin[]/defout[] screening up to livein[]/liveout[] setup.

This keeps us from having to run the loop to propagate up quite so much.

steam shader-db time -1.86356% +/- 0.941498% (n=10).  There's a small
scheduling effect, since previously the scheduler wasn't considering
defin/defout:

cycles helped:   shaders/closed/steam/amnesia-the-dark-descent/high/241.shader_test FS SIMD16: 11428 -> 11422 (-0.05%) (scheduled: scheduled)
cycles helped:   shaders/humus-volumetricfogging2/1.shader_test FS SIMD32: 13832 -> 13800 (-0.23%) (scheduled: scheduled)
cycles helped:   shaders/tesseract/479.shader_test FS SIMD32:      9330 -> 8644 (-7.35%) (scheduled: scheduled)

cycles HURT:   shaders/robclark-shaders/android/angle/aztec_ruins/36.shader_test FS SIMD32: 7870 -> 7940 (0.89%) (scheduled: scheduled)
cycles HURT:   shaders/robclark-shaders/gfxbench5/gl_5_high_off/57.shader_test FS SIMD32: 7870 -> 7940 (0.89%) (scheduled: scheduled)
cycles HURT:   shaders/robclark-shaders/gfxbench5/gl_5_normal_off/54.shader_test FS SIMD32: 7870 -> 7940 (0.89%) (scheduled: scheduled)
cycles HURT:   shaders/robclark-shaders/android/angle/aztec_ruins/30.shader_test FS SIMD32: 8726 -> 8808 (0.94%) (scheduled: scheduled)
cycles HURT:   shaders/robclark-shaders/gfxbench5/gl_5_high_off/51.shader_test FS SIMD32: 8726 -> 8808 (0.94%) (scheduled: scheduled)
cycles HURT:   shaders/robclark-shaders/gfxbench5/gl_5_normal_off/48.shader_test FS SIMD32: 8726 -> 8808 (0.94%) (scheduled: scheduled)
cycles HURT:   shaders/robclark-shaders/gfxbench5/gl_4_off/129.shader_test TCS SIMD8: 3911 -> 3979 (1.74%) (scheduled: scheduled)
cycles HURT:   shaders/robclark-shaders/gfxbench5/gl_4_off/109.shader_test TCS SIMD8: 3911 -> 3979 (1.74%) (scheduled: scheduled)

total cycles in shared programs: 313096438 -> 313096306 (<.01%)
cycles in affected programs: 92200 -> 92068 (-0.14%)

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24702>

intel/fs: Move defin/defout setup to the start of the loop.

Refactor for the next commit.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24702>

nir/opt_move: fix handling of if-condition

By accident, this used the parent of the nir_src which is a nir_if
instead of the parent of the SSA value.

Totals from 10814 (8.10% of 133461) affected shaders: (GFX11)
Instrs: 21759185 -> 21757190 (-0.01%); split: -0.02%, +0.01%
CodeSize: 112320272 -> 112316008 (-0.00%); split: -0.02%, +0.01%
SpillSGPRs: 11220 -> 11212 (-0.07%)
SpillVGPRs: 911 -> 903 (-0.88%); split: -1.54%, +0.66%
Latency: 258334759 -> 258316073 (-0.01%); split: -0.02%, +0.01%
InvThroughput: 31428650 -> 31426394 (-0.01%); split: -0.02%, +0.01%
VClause: 309119 -> 309090 (-0.01%); split: -0.01%, +0.01%
SClause: 657028 -> 657150 (+0.02%); split: -0.03%, +0.04%
Copies: 1434209 -> 1432420 (-0.12%); split: -0.28%, +0.15%
Branches: 481804 -> 481801 (-0.00%)
PreSGPRs: 829995 -> 829966 (-0.00%)
PreVGPRs: 758249 -> 758253 (+0.00%)

Fixes: 8a78706643ecad8a1f303cc9358873abc29978b4 ('nir: refactor nir_opt_move')
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24695>

nir/lower_gs_intrinsics: Remove end primitive for points

EndPrimitive() for points is entirely pointless, so just remove it when lowering
EndPrimitive to simplify the IR. This is (maybe) an optimization everywhere, and
will be relied on for correctness on Asahi.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24798>

nir/print: Print access qualifiers for intrinsics

Instead of printing an opaque integer that needs to be manually decoded.
Example output:

    32x4   %7 = @image_load (%4 (0x0), %6, %5 (0x0), %4 (0x0)) (image_dim=2D, image_array=false, format=r8g8b8a8_snorm, access=readonly|reorderable, range_base=0, dest_type=float32)

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24798>

compiler/types: Use smaller keys for explicit_matrix_types table

Instead of using the name as key, use a shorter struct type.
Only build a name string if we are adding a new entry to the table.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23281>

compiler/types: Extract get_explicit_matrix_instance() function

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23281>