util/tests: adjust for new gtest

GTest deprecated the GTEST_ARRAY_SIZE_ macro.

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22814>

nv50: Fix return type of nv50_blit_is_array

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22997>

nvc0: Free blitter->vp

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22997>

nv50,nvc0: Free nir from blitter fp shader

Fixes: d11145e837 ("nv50,nvc0: Use nir in nv50_blitter_make_fp")
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22997>

dxil: Use unified atomics

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22993>

microsoft/compiler: Back-propagate type requirement information

For ALU ops where input types are known, we can store that info on
the input sources. This can be used to produce the correct overloads
of load instructions that don't immediately need to be followed by
bitcasts, or similarly to produce a constant value which can be directly
consumed by the relevant instruction without needing a bitcast.

Similarly for values that will be stored in an output, we know type
information. And using that info, we can use more-correct information
for phis instead of forcing all phi sources to be bitcast to int just
to be bitcast back to float on the other side for an alu or an output
store.

One missing piece is SSBO stores, where we can support int or float.
If the input is coming from a phi, we don't influence the phi's type,
so it'll be int, even though the incoming sources might've been float.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22972>

microsoft/compiler: Duplicate some SSA values to simplify SSA typing

For each phi src, ensure that it's only used as a phi src.
This lets us give each phi their own unique types without worrying
about them stomping on each other. Also scalarize phis.

For each constant, ensure that it's only used once. The DXIL backend
will already dedupe these consts within the module, but this lets a
single load_const have multiple types depending on how it's used.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22972>

microsoft/compiler: Remove alu type info from store_dest()

We pass in a *typed* value, we don't need to pass in additional
type info. That's just more opportunities to get it wrong.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22972>

ac/llvm: rewrite and unify how GLC, DLC, SLC are set

Use ACCESS_* flags in call sites instead of GLC/DLC/SLC.

ACCESS_* flags are extended to describe other aspects of memory instructions
like load/store/atomic/smem.

Then add a function that converts the access flags to GLC, DLC, SLC.

The new functions are also usable by ACO.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22770>

ac/llvm: don't treat ACCESS_NON_READABLE as ACCESS_COHERENT

... and expect it to behave like ACCESS_NON_TEMPORAL.
Handling ACCESS_NON_TEMPORAL is sufficient.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22770>

aco: don't treat ACCESS_NON_READABLE as ACCESS_COHERENT

... and expect it to behave like ACCESS_NON_TEMPORAL.
Handling ACCESS_NON_TEMPORAL is sufficient.

This was copied from ac_nir_to_llvm.c.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22770>

intel: Secondary CB print primary CB's renderpass

Reviewed-by: Mark Janes <markjanes@swizzler.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22723>

intel: batch consecutive dispatches into implicit renderpasses

Reviewed-by: Mark Janes <markjanes@swizzler.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22723>

intel: refactor INTEL_MEASURE pointer dumping

Refactor framebuffer to renderpass to mirror previous INTEL_MEASURE
changes.

We dump hashes/pointers for shaders and framebuffer/renderpass.
Reduce from 64bit to 32bit pointers. We don't benefit from the
extra precision and reduced output size is convenient.

Reviewed-by: Mark Janes <markjanes@swizzler.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22723>

anv: re-enable RT data in INTEL_MEASURE

Per-RenderTarget analysis was removed from anv's INTEL_MEASURE
previously, probably after switching to dynamic rendering model.
Restore capability by tracking count of beginRenderPass calls.

Reviewed-by: Mark Janes <markjanes@swizzler.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22723>

anv: fix INTEL_MEASURE on MTL

Ensure counter buffer is coherent. Required for MTL which changes
coherence policy.

Reviewed-by: Mark Janes <markjanes@swizzler.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22723>

nir/validate: Handle unified atomics

nir_validate checks that the format of an atomic (if specified) is compatible
with the atomic operation. For example, we can't fadd R64_UINT texels. The logic
can't be extended as-is to unified atomics because it's split across different
switch cases for different atomic-op intrinsics. So we add our own validation
case, porting over the logic from the separate existing cases below.

(The redundant logic will be deleted once we delete legacy atomics.)

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22914>

nir/opt_uniform_atomics: Handle unified atomics

This is the one place where using nir_atomic_op instead of nir_op directly is a
little annoying, since we need to translate between the two enums, but it's not
a big deal.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22914>

nir/lower_ssbo: Handle unified atomics

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22914>

nir/lower_io: Handle unified atomics

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22914>

nir/lower_task_shader: Handle unified atomics

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22914>

nir: Load/store atomic op indices when lowering image intrinsics

They might not be stored in the same const index after lowering

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22914>

nir: Handle unified atomics in simple cases

Lots of passes can be made unified-atomics-aware simply by adding extra cases in
their switch statements. This commit fixes a bunch of passes.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22914>

ir3: Use unified atomics

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Rob Clark <robclark@freedesktop.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22914>

nv50/ir: Use unified atomics

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: M Henning <drawoc@darkrefraction.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22914>

zink: Use unified atomics

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22914>

aco,radv: Use unified atomics

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22914>

ac/llvm: Use unified atomics

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22914>

ac/llvm: Don't handle atomic derefs

Should not be seen, already would be stubbed out.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Suggested-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22914>

ntt: Use unified atomics

Nice deduplication of the NIR->TGSI enum translation.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22914>

gallivm: Use unified atomics

This is a huge win because gallivm duplicated the translations in a zillion
places.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22914>

pan/mdg: Use unified atomics

This doesn't actually save anything, since Italo already introduced magic macros
for this, but it ticks off one more driver on the list to convert. It's also
more legible, so that's nice :-)

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22914>

pan/mdg: Fix icky formatting

clang-format butchered this initializer pretty badly.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22914>

pan/bi: Use unified atomics

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22914>

agx: Use unified atomics

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22914>

nir: Add pass to lower atomics to unified

In the future, we'd like to have all drivers only ingest unified atomics, and
all frontends only produce unified atomics, and garbage collect the existing
non-unified atomics. To get to that future, it's a lot nicer to convert drivers
one-by-one. Add a pass to translate old-style atomics to new-style atomics so
drivers can opt-in to the new form one-by-one. Once all drivers are converted,
we can convert producers one-by-one. Finally, we can just drop the calls to the
pass and garbage collect this pass and the old atomics. That's probably a while
out, though, so this will be out bridge to get there.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rob Clark <robclark@freedesktop.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22914>

nir: Add unified atomics

Currently, we have an atomic intrinsic for each combination of memory type
(global, shared, image, etc) and atomic operation (add, sub, etc). So for m
types of memory supported by the driver and n atomic opcodes, the driver has to
handle O(mn) intrinsics. This makes a total mess in every single backend I've
looked at, without fail.

It would be a lot nicer to unify the intrinsics. There are two obvious ways:

1. Make the memory type a constant index, keep different intrinsics for
   different operations. The problem with this is that different memory types
   imply different intrinsic signatures (number of sources, etc).  As an
   example, it doesn't make sense to unify global_atomic_amd with
   global_atomic_2x32, as an example. The first takes 3 scalar sources, the
   second takes 1 vector and 1 scalar. Also, in any single backend, there are a
   lot more operations than there are memory types.

2. Make the opcode a constant index, keep different intrinsics for different
   operations. This works well, with one exception: compswap and fcompswap
   take an extra argument that other atomics don't, so there's an extra axis of
   variation for the intrinsic signatures.

So, the solution is to have 2 intrinsics for each memory type -- for atomics
taking 1 argument and atomics taking 2 respectively. Both of these intrinsics
take an nir_atomic_op enum to describe its operation. We don't use a nir_op for
this purpose, as there are some atomics (cmpxchg, inc_wrap, etc) that don't
cleanly map to any ALU op and it would be weird to force it.

The plan is to transition to these new opcodes gradually. This series adds a
lowering pass producing these opcodes from the existing opcodes, so that
backends can opt-in to the new forms one-by-one. Then we can convert backends
separately without any cross-tree flag day. Once everything is converted, we can
convert the producers and core NIR as a flag day, but we have far fewer
producers than backends so this should be fine. Finally we can drop the old
stuff.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rob Clark <robclark@freedesktop.org>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22914>

aco/gfx11: fix VMEM/DS->VALU WaW/RaW hazard

Previously, we could safely read/write unused lanes of VMEM/DS
destination VGPRs without waiting for the load to finish. That doesn't
seem to be the case on GFX11.

fossil-db (gfx1100):
Totals from 6698 (4.94% of 135636) affected shaders:
Instrs: 11184274 -> 11199420 (+0.14%); split: -0.00%, +0.14%
CodeSize: 57578344 -> 57638928 (+0.11%); split: -0.00%, +0.11%
Latency: 198348808 -> 198382472 (+0.02%); split: -0.00%, +0.02%
InvThroughput: 24376324 -> 24378439 (+0.01%); split: -0.00%, +0.01%
VClause: 192420 -> 192559 (+0.07%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8722
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8239
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22965>

meson/vaon12: fix driver file name for mingw build

This fixes vaon12 driver file name to be consistent with libva
expectation - vaon12_drv_video.dll - without lib prefix.

Signed-off-by: Dmitry Rogozhkin <dmitry.v.rogozhkin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22995>

lvp: avoid accessing member of NULL ptr for global entries

Cc: mesa-stable
Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22979>

pipe-loader: avoid undefined memcpy behavior

If either dest or src is an invalid or null pointer, the behavior is
undefined, even if count is zero.

Cc: mesa-stable
Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22979>

zink: disable dynamic state exts if the previous ones aren't present

this would be weird if a driver did it

cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22996>

zink: disable have_EXT_vertex_input_dynamic_state without EDS2

this is disabled already in the draw paths but not the pipeline paths

cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22996>

nir_lower_system_values: Add ASSERTED to assert-only variable

Fixes: 1e0e4657 ("spirv/nir: wire ray interection triangle position fetch")
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22994>

ci: replace write + cat with tee

Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Martin Roukala <martin.roukala@mupuf.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22945>

ci: stop marking environment variable list as executable

We're only going to read it, not execute it.

Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Martin Roukala <martin.roukala@mupuf.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22945>

ci: drop GENERATE_ENV_SCRIPT

Added in 16b417b8d617df418ad1 ("ci, valve: Add the dEQP runners for
Valve CI") but never used.

Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Martin Roukala <martin.roukala@mupuf.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22945>

nir: Use nir_block_last_phi_instr more

We have a helper, don't open code it.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22967>

ac/llvm: Use nir_foreach_phi

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22967>

dxil: Use nir_foreach_phi_safe

This should be faster, since we're not iterating pointlessly over all the
non-phis after the phi.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22967>

nir: Use nir_foreach_phi(_safe)

The pattern shows up all the time open-coded. Use the macro instead.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22967>

nir: Add nir_foreach_phi(_safe) macro

Serious preprocessor voodoo here. There are two tricks here.

1. Iterating only phis. We know that phis come only at the beginning of a block,
   so all over the tree, we open-code iteration like:

      nir_foreach_instr(instr, block) {
         if (instr->type != phi)
            break;

         /* do stuff */
      }

   We can express this equivalently as

      nir_foreach_instr(instr, block)
         if (instr->type != phi)
            break;
         else {
            /* do stuff */
         }

   So, we can define a macro

      #define nir_foreach_phi(instr, block)
         if (instr->type != phi)
            break;
         else

   and then

      nir_foreach_phi(..)
         statement;

   and

      nir_foreach_phi(..) {
         ...
      }

   will expand to the right thing.

2. Automatically getting the phi as a phi. We want the instruction to go to some
   hidden variable, and then automatically insert nir_phi_instr *phi =
   nir_instr_as_phi(instr_internal); We can't do that directly, since we need to
   express the assignment implicitly in the control flow for the above trick to
   work. But we can do it indirectly with a loop initializer.

      for (nir_phi_instr *phi = nir_instr_as_phi(instr_internal); ...)

   That loop needs to break after exactly one iteration. We know that phi
   will always be non-null on its first iteration, since the original
   instruction is non-null, so we can use phi==NULL as a sentinel and express a
   one-iteration loop as for (phi = nonnull; phi != NULL; phi = NULL).

Putting these together gives the macros implemented used.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22967>

aco: don't apply dpp if the alu instr uses the operand twice

CP77 has a ton of fma(dpp(a), dpp(a), b).

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22698>

aco: use VOP3+DPP

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22698>

aco/ra: convert VOPC_DPP instructions without vcc to VOP3

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22698>

aco: add assembler tests for VOP3(P) with DPP

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22698>

aco/builder: support VOP3(P) with dpp

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22698>

aco/assembler: support VOP3P with DPP

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22698>

aco/optimizer: copy pass flags for newly created valu instructions

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22698>

aco/optimizer: don't use pass_flags for mad idx

fma can use DPP on GFX11+, so we want to keep the exec id in pass_flags

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22698>

aco/vn: compare all valu modifers

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Fixes: 9b4ea9ff901 ("aco/vn: hash opsel for VOP12C")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22698>

zink: add back some anv qbo flakes

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22987>

docs: Include ALU opcode descriptions

If we have a description for an ALU opcode in NIR, include it with our generated
HTML documentation so people don't need to go to nir_opcodes.py anyway because
the documentation is missing the documentation ;-)

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Jesse Natalie <jenatali@microsoft.com>
Tested-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22929>

nir: Make ALU descriptions machine-readable

We already document a lot of ALU opcodes, let's make this machine-readable so we
can put the descriptions in our generated HTML documentation.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22929>

nir: Allow adding descriptions to ALU opcodes

This will let us generate nicer documentation.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22929>

vc4/ci: re-enable VC4 testing

But keeps piglit's quick_gl disabled, as it contains the test causing
the GPU resets.

Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22982>

amd/drm-shim: Update docs for more devices.

We don't have to update the docs for every new entry.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22975>

amd/drm-shim: Add raphael&mendocino, polaris12 and gfx1100.

Decided to follow the chip names pretty much.This set happens to be
what is in my workstation currently.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22975>

amd/drm-shim: Add vangogh entry.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22975>

anv: handle missing astc for gfx125 in CreateImageView

Cc: mesa-stable
Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22818>

radv: configure PA_CL_VRS_CNTL entirely from the cmd buffer

We already have all the information needed to configure it.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22942>

radv: re-emit fragment shading rate state when PA_CL_VRS_CNTL changes

Found by inspection.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22942>

radv: dirty the dynamic vertex input state only when needed

This shouldn't be necessary when the VS doesn't have a prolog.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22944>

radv: reset the emitted VS prolog when a new vertex shader is bound

When a new vertex shader is bound, the VS prolog needs to be
re-emitted, and this allows us to avoid tracking if the pipeline is
dirty.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22944>

hasvk: avoid assert due to unsupported format

Fixes: 0a4c92b646f ("hasvk: Use the common vk_ycbcr_conversion object")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9011
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22978>

anv: Enable MCS init with ISL_AUX_OP_AMBIGUATE

Up until now, we have been initializing MCS with fast clears. This is
mostly safe, but there's a corner case that can be an issue.

The issue is with a workaround for MCS that requires the sampler not see
any fast-cleared blocks for certain surfaces (14013111325). Even though
we have been initializing MCS with fast clears, we expect most
applications to be safe because we expect that they would only sample
the samples they've rendered to previously (and the render would've
removed the fast-cleared blocks). In other words we don't expect that
apps would transition from VK_IMAGE_LAYOUT_UNDEFINED to
VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL and start sampling immediately.

If an application took the unexpected path of sampling undefined
samples, it's possible they'd hit the issue described in the workaround.

Fix this corner case by using an ambiguate to initialize MCS.

Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22545>

anv: Drop the MCS initialization performance warning

The comment above the warning explains that not all bit patterns are
necessarily valid. While we're at it, fix a typo in that comment.

Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22545>

iris: Enable MCS init with ISL_AUX_OP_AMBIGUATE

Add support for using BLORP's ambiguate pass to initialize MCS instead
of mapping and memsetting it on the CPU. Note that this won't be used if
the first operation on the MSAA layer is a fast clear.

Since we're no longer mapping, this removes a blocker towards getting
MCS_CCS enabled in small-BAR mode.

This functionality is difficult to test because of the way iris is set
up. It always tries to compress writes. So, a test would only read the
ambiguated MCS element if it tries to read from undefined samples.

To test this, I locally disabled fast clears and rendering with MCS (via
iris_resource_render_aux_usage). I continued to allow sampling with MCS
in iris_resource_texture_aux_usage. So, writes go directly to the main
surface and reads go through the ambiguated MCS surface.

When I then ran the test group, dEQP-GLES3.functional.multisample.*, all
48/64 supported tests passed on my Ice Lake. If I slightly changed
BLORP's ambiguate pass, I observed several tests failing.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22545>

intel: Implement ISL_AUX_OP_AMBIGUATE for MCS

Implement the ambiguate operation for MCS. This clears MCS layers with a
sample-dependent "uncompressed" value that tells the sampler to go look
at the main surface.

Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22545>

iris: Add a barrier to iris_mcs_partial_resolve

Partial resolves read from the MCS and write to the MSAA surface.
Add a texture barrier to prepare for the reads.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4179
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22545>

intel/isl: Bump the MCS halign value for BDW+

Select a horizontal alignment value that matches the main MSAA surface.

We need a valid horizontal alignment to perform MCS ambiguates. The
halign value doesn't actually affect test behavior, but it is validated
by isl_surf_fill_state. We currently have an invalid halign for gfx125.
This patch fixes that.

Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22545>

ail: Add MSAA tests

This tests the following matrix:
- Format: RGBA8Unorm, RGBA16Unorm, RGBA32Float
- Samples: 2 or 4
- Layers: 1 or 2
- Width: Interesting values 1..4097
- Height: Interesting values 1..4097

Compression is based on the dimensions (that is, everything that can be
compressed is). This test compares both the total texture size and the
compression metadata offset.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22971>

ail: Handle larger block sizes

We need to support up to 16 bytes/sample * 4 samples/pixel = 64 bytes/pixel for
multisampling to work with formats like RGBA32F.

Fixes dEQP-GLES3.functional.fbo.msaa.4_samples.rgba32f

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22971>

asahi: Use ail_can_compress() in agx_compression_allowed()

This moves the compression size threshold logic into ail, where
it belongs.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22971>

ail: Implement multisampling for compression meta calculation

For multisampled textures, the decision about whether to compress or not
is based on the effective width and height in samples, not pixels.

Introduce ail_can_compress() to encode this logic in ail, so the driver
can use it to decide whether to compress or not before the full layout
is determined.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22971>

asahi: Make bo->writer_syncobj atomic

BOs can be written from several contexts, so writing to this member is
racy. We only care about this for the purposes of exporting BOs after a
submission (and if the app is racing writers/submissions at that point
all bets are off), so just keeping track of the last written value is
sufficient.

Switch to atomic operations to eliminate the race, and drop the assert
in the batch cleanup path that no longer holds when the BO might have
been written to from another context.

Fixes: asahi/mesa#20

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22971>

asahi: Lazily initialize batch state on first draw

We track buffers written by batches, but this gets messy when we end up
with an empty batch that is never submitted, since then it might have
taken over writer state from a prior already submitted batch (for its
framebuffers).

Instead of trying to track two tiers of resource writers, let's just
defer initializing batch state until we know we have a draw (or compute
launch, or clear). This means that if a batch is marked as a writer for
a buffer, we know it will not be an empty batch.

This should be a small performance win for empty batches (no need to
emit initial VDM state or run the writer code), but more impontantly it
eliminates the empty batch writer state revert corner case.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22971>

asahi: Partially identify some missing index list stuff

Still unclear what the extra 2 blocks do, but at least we know the
size/order now.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22971>

asahi: Add some more system registers

Core and opfifo stuff from the compute helper blob, vm_slot because it
was the only one changing when I poked around yesterday and it hit me
what it was ^^

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22971>

asahi: Fix check for sprite coord mode in agx_bind_rasterizer_state

We need to set ctx->rast = so after comparing them.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22971>

asahi: Add missing stdbool include to lib/hexdump.h

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22971>

asahi: Disable tilebuffer write masking optimization

This seems to flake some dEQPs due to some kind of race/UB (which
doesn't even always cause the dEQPs to fail due to leeway in the image
comparison, since the problem is usually just a few pixels, but it's
there).

I spent a bunch of time trying other flags/things, and almost everything
changed the bad pixel pattern randomly but nothing fixed it. Let's
revisit this one later, since it looks like a pretty deep rabbit hole.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22971>

asahi: Make framebuffer texture barriers a no-op

Framebuffer fetch is coherent, so there is no need for barriers here.
This avoids pointless flushing if an app calls glBlendBarrier().

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22971>

asahi: Implement create_fence_fd and fence_server_sync

Apparently we were still missing some fence stuff, and it started
crashing Firefox in apitrace? I'm not sure why we never noticed this
before, but it's trivial enough. Cargo culted from Panfrost.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22971>

asahi: Implement memory_barrier

Cargo culted from panfrost.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22971>

intel: Disable shader cache when executing intel_clc during the build

With the shader cache enabled, intel_clc attempts to write to ~/.cache.
Many distributions' build systems limit file-system access, and will
kill the process thus causing the build to fail.

Fixes: 639665053fa ("anv/grl: Build OpenCL kernels")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22968>

radv: improve externalMemoryFeatures for android ahb

VK_EXTERNAL_MEMORY_FEATURE_DEDICATED_ONLY_BIT should always be set, as
required by the spec.

VK_EXTERNAL_MEMORY_FEATURE_EXPORTABLE_BIT should be set when
radv_ahb_format_for_vk_format knowns the format.  That is,
radv_create_ahb_memory should at least know how to call
AHardwareBuffer_allocate.

VK_EXTERNAL_MEMORY_FEATURE_IMPORTABLE_BIT is always set.  We can't know
if gralloc can allocate the format/flags/usage combo or not (gralloc
might use a private format for the combo).

Fixed
dEQP-VK.api.external.memory.android_hardware_buffer.image_formats.*.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22619>

anv,hasvk: android ahb is not always exportable

anv_ahb_format_for_vk_format needs to know the format at least.  There
is no guarantee that AHardwareBuffer_allocate will succeed, but we are
reluctant to check with AHardwareBuffer_isSupported which may
test-allocate internally and is expensive.

v2: add anv_ahb_format_for_vk_format to anv_android_stubs.c

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22619>

vulkan: add vk_image_format_to_ahb_format

There should be no functional change.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22619>

anv,hasvk,radv: do not fall back to AHARDWAREBUFFER_FORMAT_BLOB

When allocating a VkDeviceMemory exportable as AHB, it seems incorrect
to fall back to AHARDWAREBUFFER_FORMAT_BLOB when the image has no known
AHB format.  We should fail the allocation instead.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22619>