vc4: Convert to new-style NIR registers

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24153>

nir/trivialize: Move decl_reg to the start of the block

This makes it so we never find a reg_decl in between a reg_store and the def
for its value, which helps avid inserting copy movs.

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24153>

nir/trivialize: Handle more RaW hazards

Consider the snippet of NIR:

   div 32    %447 = @load_reg (%442) (base=0, legacy_fabs=0, legacy_fneg=0)
   div 32    %463 = @load_reg (%442) (base=0, legacy_fabs=0, legacy_fneg=0)
   con 32    %409 = iadd %17 (0x3), %447
                    @store_output (%182 (0x601), %463) (base=0, wrmask=x, component=0, src_type=invalid...
                    @store_reg (%409, %442) (base=0, wrmask=x, legacy_fsat=0)

The load_reg's are trivial, so the %442 read will get folded into store_output.
But under the old definition, the store_reg is also trivial so it gets folded
into the iadd... causing a read-after-write hazard and invalid code generation.

The fix is to amend our definition of store_reg triviality to account for loads
getting folded in. It's not good enough that there's no intervening load_reg,
there can also be no intervening source that gets chased to a load_reg. Handle
that case as well.

Identified in dEQP-VK.geometry.input.basic_primitive.triangles_adjacency on
V3DV.

Fixes: d313eba94ef0 ("nir: Add pass for trivializing register access")
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reported-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24153>

nir/trivialize: Trivialize cross-block loads

In order for a register load to be trivial, it cannot be used in any
block other than the one in which it is loaded.  We're not currently
explicitly doing anything to ensure this invariant holds.  It may be
that it holds regardless but I couldn't find any documented reason why
it should so let's explicitly handle that case.  Worst case, the newly
added code does nothing.

Fixes: d313eba94ef0 ("nir: Add pass for trivializing register access")
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24153>

nir/trivialize: Maintain divergence information

Because this pass is intended to be run after out-of-SSA and directly
before injesting the NIR into the back-end, it may come after divergence
analysis and needs to preserve the divergence information. Fortunately,
since all we ever do is insert nir_op_mov, this is easy.

Fixes: d313eba94ef0 ("nir: Add pass for trivializing register access")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24153>

nir: Properly handle divergence for load_reg

This commit makes three changes:

 1. Default all newly created registers divergent because this is the
    safer default.

 2. Make divergence analysis do something sane with register divergence.
    It's not perfect because divergence analysis isn't able to prove
    registers divergent based on stores but at least if someone uses
    registers a bit they'll end up with safe defaults.  This matches
    what they'd get with nir_ssa_def_init().

 3. Make the load_reg() helper automatically propagate divergence from
    the register.  Because the defaults for both nir_ssa_def_init() and
    nir_decl_reg() are to mark everything divergent, this only means
    that nir_load_reg() of a uniform reg is now uniform.

Putting all these together, nir_from_ssa should now be producing
load_reg intrinsics with the proper uniform information.

Fixes: 7229bffcb133 ("nir: Add intrinsics for register access")
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24153>

pan/bi: Remove leftover include

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24153>

intel/compiler/test: fix crashes when TEST_DEBUG is set

Dumping instructions requires that ISA info is not empty.

Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24274>

lavapipe: fixes indent of function lvp_inline_uniforms

The indent fixes are in separate patch is for easier to review

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24316>

lavapipe: Convert to use nir_foreach_function_impl

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24316>

zink: Convert to use nir_foreach_function_impl when possible

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24316>

freedreno: Switch to use nir_foreach_function_impl in tu_shader.cc

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24316>

lima: Convert to use nir_foreach_function_impl when possible

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Erico Nunes <nunes.erico@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24316>

rusticl/core: Make convert_spirv_to_nir output pair (KernelInfo, NirShader)

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23898>

rusticl/core: Delete KernelDevState and KernelDevStateInner

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23898>

rusticl/program: New helper functions to NirKernelBuild

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23898>

rusticl: Move NirKernelBuild to ProgramDevBuild

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23898>

rusticl/compiler: Remove unnecessary functions

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23898>

rusticl: Move Cso to Program

Commit got huge, but couldn't figure out a better way to split without
breaking stuff.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23898>

rusticl/compiler: Add NirPrintfInfo

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23898>

rusticl/kernel: Add CsoWrapper

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23898>

rusticl/kernel: Removing unnecessary clone in kernel launch

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23898>

ci/freedreno: add a530 flake vs-lessthanequal-uvec4-uvec4

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24314>

state_tracker: fix dereference before null check

Coverity error
CID 1528178 (#1 of 1): Dereference before null check (REVERSE_INULL)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Illia Polishchuk <illia.a.polishchuk@globallogic.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20893>

glx: fix dead code when gc var cannot be null due to earlier check

CID 1528170 (#1 of 1): Logically dead code (DEADCODE)
At condition gc, the value of gc cannot be NULL.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Illia Polishchuk <illia.a.polishchuk@globallogic.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20893>

s/Intel: fix/anv: fix: potentially overflowing expression in genX

CID 1528164 (#1 of 1): Unintentional integer overflow (OVERFLOW_BEFORE_WIDEN)
overflow_before_widen: Potentially overflowing expression
pool->n_passes * pool->khr_perf_preamble_stride with type
unsigned int (32 bits, unsigned) is evaluated using 32-bit arithmetic,
and then used in a context that expects an expression of type uint64_t (64 bits, unsigned).

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Illia Polishchuk <illia.a.polishchuk@globallogic.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20893>

iris: remove NULL check for already dereferenced pointer earlier

Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Illia Polishchuk <illia.a.polishchuk@globallogic.com>
Found by Coverity.

CID: 1528158
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20893>

i915: change format in dbg string
Actually, uintptr_t is of type unsigned long, but the
debug line uses the %d format specifier, which expects an int.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Illia Abernikhin <illia.abernikhin@globallogic.com>
Found by Coverity.

CID: 1515961
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20893>

state_tracker: moving initialisation of whandle out from if statement
whandle initialization inside if statement but used also outside

Signed-off-by: Illia Abernikhin <illia.abernikhin@globallogic.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Found by Coverity.

CID: 1516746
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20893>

lavapipe: Advertise samplerYcbcrConversion

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24295>

lavapipe: Implement samplerYcbcrConversion

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24295>

lavapipe: Fix binding immutable samplers with desc buffers

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24295>

lavapipe: Store immutable_samplers as lvp_sampler array

We will need this to access the ycbcr conversion.

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24295>

lavapipe: Remove dummy sampler ycbcr conversion

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24295>

gallivm: Ignore nir_tex_src_plane

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24295>

gallivm: Fix subsampled format sampling under Vulkan

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24295>

draw: Do not restart the primitive_id at 0

Otherwise the primitive_id will wrap around to 0 if more than 4096
patches are drawn.

cc: mesa-stable

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24295>

radv: pass submit info to radv_check_gpu_hangs()

This will allow to dump preambles/postambles CS and eventually even
more CS.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24191>

radv/amdgpu: rename old_ib to ib in radv_amdgpu_winsys_cs_dump()

Forgot this variable when I renamed the ib_buffers array.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24191>

radv/amdgpu: fix dumping CS with the chained IBs path

ib_buffer is now NULL in both paths, and the first IB is the beginning
of the chain.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24191>

radv: use next_stage for determining the stage to lower NGG

If the next stage is FS, it's also the last VGT API stage.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24273>

radv: simplify getting next VS stage for VS prologs

It's the VS shader info stage.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24273>

radv: determine as_ls earlier by using the next stage

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24273>

radv: determine ES info for VS/TES with GS earlier

By using the next stage, it's possible to compute these information
earlier without having to link shaders info.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24273>

radv: use the number of GS linked inputs to compute the ESGS itemsize

It's similar.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24273>

radv: add a helper to compute the ESGS itemsize

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24273>

radv: remove the pipeline dependency for creating a GS copy shader

This is unnecessary. While we are at it, stop passing the array of
shaders and use the GS stage only.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24273>

intel/common: Only set op mask on instructions in decoder

When a default value of a struct's field, which is in the
higher half of the first dword, is specified in a gen xml
file, setting op mask makes decoder treat the field as a
header (intel_field_is_header()). As a result, it won't
output the field in batch dump. This is not a common case
but can happen once a gen xml file includes such fields.

The op mask is only meaningful to instructions, so we fix
the above issue by not setting op mask of structs (also
registers).

Signed-off-by: Jianxun Zhang <jianxun.zhang@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24268>

iris: Handle clear color compatibility in prepare_render

Before this patch, iris_resource_render_aux_usage would disable
compression when the clear color did not support format
reinterpretation.

With this patch, iris now replaces the clear color with zero and keeps
compression enabled. Disabling fast clears would be enough for most aux
usages, but replacement is also done to handle ISL_AUX_USAGE_FCV_CCS_E.

Note that this also fixes a bug. Format reinterpretation with
incompatible clear colors previously was not handled for the MCS aux
usages.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23676>

iris: Create BLORP surfaces after resource preparation

iris_resource_prepare_render will soon gain the ability to change a
resource's clear color. iris_blorp_surf_for_resource will keep a copy of
that clear color, so make sure calls to it happen after the render
preparation helper. At the moment, this shouldn't have an impact besides
improving debugging.

While we're here, do the same for the generic access preparation helper.
We may convert those to more specific helpers at a later time.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23676>

iris: Pass the render format to prepare_render

This will be used in an upcoming patch.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23676>

iris: Reorder render_aux_usage parameters

Match the order of the parameters for iris_resource_texture_aux_usage.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23676>

intel/blorp: Ambiguate after CCS resolves on gfx7-8

ISL's state-machine of CCS_D describes full resolves as leaving the aux
buffer in the pass-through state. Hardware doesn't behave this way on
gfx8 however. On that platform, full resolves transition the aux buffer
to the resolved state. This was verified by dumping the CCS before and
after a full resolve on BDW (gfx7 is simply assumed to behave the same).
Ambiguate after resolving to match driver expectations.

Prevents iris from failing piglit's fcc-write-after-clear on BDW with a
future patch which relies on fast-clear encodings being removed after a
resolve. The avoided failure is:

   Testing implicit read of partial block UNORM -> SNORM
   Probe color at (0,1,0)
     Expected:  1.000000 1.000000 1.000000 1.000000
     Observed:  0.000000 0.000000 0.000000 0.000000

Cc: mesa-stable
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23676>

intel/fs: don't try to rebuild sequences of non ssa values

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 04777171e0 ("intel/fs: try to rematerialize surface computation code")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9378
Reviewed-by: Illia Polishchuk <illia.a.polishchuk@globallogic.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24228>

meson: Ensure that LLVMSPIRVLib is not required for Clover

Fixes: cb588d5d6ee ("compiler/clc: Move related NIR passes to the common mesa clc")
Closes: #9391
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24244>

ci/tgl: Improve the info for ANGLE's MSAA regression on TGL.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24200>

ci: Uprev ANGLE to 0518a3ff4d4e ("Android: Simplify power metrics collection")

There have been some fixes for our drivers that we'd like to bring in.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24200>

ci/radv: Clarify when the ANGLE GS failures started happening.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24200>

anv,hasvk,iris: sampler_prog_key::swizzles is only used on crocus

The field is no longer consumed by brw_complie_* and is instead handled
directly by the crocus driver.  Therefore, it's safe to leave it zero
and not even bother setting it.  This removes our reliance on the
SWIZZLE_* macros in prog_instructions.h.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24288>

etnaviv: nir: convert to new-style NIR registers

The initial plan was to use 'nir_legacy' helpers but it turns out
that our RA pass is hard to confince to be happy with it. So we are
useing the 'chasing' helpers now.

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Lucas Stach <l.stach@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24216>

etnaviv: nir: switch to etna_nir_lower_to_source_mods(..)

nir's source modifiers are going away soon and with it also the lowering
pass. Lets switch to our own lowering pass. We need to run our own
lowering pass almost at the end else opc_cse(..) etc. might do some
wrong needed opts as nir does not see our modifiers.

Also we need to remove the last nir_opt_dce(..) as it will remove not dead
code caused by the used load_const hack.

32    %15 = load_const (0x00000000 = 0.000000)
32     %4 = fabs %15 (0.000000)

nir_opt_dce is correct when it removes the two instructions. But in reality
the load_const is a uniform that should not be removed.

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Lucas Stach <l.stach@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24216>

etnaviv: nir: add etna_nir_lower_to_source_mods(..)

This is more or less a copy of nir_lower_to_source_mods(..) with
the following differences:

 - we store the source mods in pass_flags
 - we do not try to saturate the destination

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Lucas Stach <l.stach@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24216>

etnaviv: nir: look at parent instr in lower_alu(..)

When we switch to our own lower_to_source_mods pass we will start
to see such patterns:

32x4  %18 = fneg %5 (-5.125000, -30.000000, 5.500000, -6.500000)
32x4  %19 = ffma %18, %8, %4 (-6.500000, -7.750000, 6.500000, 6.000000)

This is a problem as we will generate instruction that accesses two
different uniforms, which is a problem on GPUs where has_no_oneconst_limit
is false.

Make lower_alu(..) smarter by looking in the parent for for the constant
value.

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Lucas Stach <l.stach@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24216>

etnaviv: do not clear all pass_flags before RA

We only need to clear the 'dead' bits. The others are
used for source mods.

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24216>

etnaviv: extend etna_pass_flags with source modifiers

As nir_lower_to_source_mods(..) will be deleted and with it the modifier storage
in nir's core we need to find an other way store the information.

We have have 6 bits left in nir's pass_flags - so lets go that route.

This also adds some small helpers that will be used later.

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24216>

etnaviv: add is_dead_instruction(..) helper

As we are going to extend the enum etna_pass_flags it makes sense
to add a small helper to test if an instruction is dead. An instruction
is dead if BYPASS_DST or BYPASS_SRC is set.

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24216>

etnaviv: name the enum used for pass_flags

This enum is used for the pass_flags that can be set on a
nir_instr. Name it to make the intention of its usage clear.

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24216>

etnaviv: make use of BITFIELD_BIT(..) macro

It helps to make the code easier to read.

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24216>

frontends/va: Add YUV420 to NV12 postproc conversion

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7853
Reviewed-by: Thong Thai <thong.thai@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24272>

gallium/auxiliary/vl: Fix blurry output of compute_shader_yuv

There is a linear sampler used, so add half texel offset
to avoid undesirable blur when input and output resolutions
are the same.

Reviewed-by: Thong Thai <thong.thai@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24272>

gallium/auxiliary/vl: Handle UV subsampling in compute_shader_yuv

Also remove the 1px vertical shift as it results
in a black line at the bottom of the picture.

Reviewed-by: Thong Thai <thong.thai@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24272>

aco: improve get_gfx11_true16_mask description

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24294>

aco/gfx11: fix get_gfx11_true16_mask with v_cmp_class_f16

The second operand is 16bit, so the we need to use VOP3 to address v128-v255.

Closes: #9413
Fixes: 6872f8d861b ("aco/gfx11: allow true 16-bit instructions to access v128+")
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24294>

nir/tests: add nir_opt_dead_cf_test.jump_before_constant_if

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24235>

nir/opt_dead_cf: remove nodes after a jump earlier

In the case of:
   halt
   // succs: b9
   if %618 {
       block b3:// preds:
       break
       // succs: b6
   } else {
       block b4:  // preds: , succs: b5
   }
   block b5:    // preds: b4
   32    %556 = iadd %617, %2 (0x1)
opt_constant_if() doesn't work because stitch_blocks() can't join blocks if the
before ends in a jump and the after isn't empty.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24235>

nir/tests: Use a single binary

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24249>

nir/tests: Refactor boilerplate into a common header

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24249>

tu,freedreno: Forbid blit event for R8G8_SRGB due to gpu faults

Same cause as for other R8G8 formats - msaa resolve via
blit event causes gpu fault.

Fixes:
 dEQP-VK.api.image_clearing.*.clear_color_attachment.*.r8g8_srgb_*

Fixes: 029919f3c83f379065515708188d5c439c3fa6bc
("tu: allow using resolve engine for SRGB MSAA resolves")

Cc: mesa-stable
Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24277>

panvk: Use 1.0 in ICD Manifest json

PanVK downgraded from supporting Vulkan 1.1 to 1.0, but did not change
their ICD Manifest api_version to reflect that. This cause the Vulkan-Loader
to interpret the ICD as a 1.1 driver erroneously. Originally discussed in this
issue https://github.com/KhronosGroup/Vulkan-Loader/issues/1242

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24289>

intel/compiler: load debug mesh compaction options once

Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20407>

intel/compiler,anv: put some vertex and primitive data in headers

Both per-primitive and per-vertex space is allocated in MUE in 8 dword
chunks and those 8-dword chunks (granularity of
3DSTATE_SBE_MESH.Per[Primitive|Vertex]URBEntryOutputReadLength)
are passed to fragment shaders as inputs (either non-interpolated
for per-primitive and flat vertex attributes or interpolated
for non-flat vertex attributes).

Some attributes have a special meaning and must be placed in separate
8/16-dword slot called Primitive Header or Vertex Header.

Primitive Header contains 4 such attributes (Cull Primitive,
ViewportIndex, RTAIndex, CPS), leaving 4 dwords (the rest of 8-dword
slot) potentially unused.

Vertex Header is similar - it starts with 3 unused dwords, 1 dword for
Point Size (but if we declare that shader doesn't produce Point Size
then we can reuse it), followed by 4 dwords for Position and optionally
8 dwords for clip distances.

This means we have an interesting optimization problem - we can put
some user attributes into holes in Primitive and Vertex Headers, which
may lead to smaller MUE size and potentially more mesh threads running
in parallel, but we have to be careful to use those holes only when
we need it, otherwise we could force HW to pass too much data to
fragment shader.

Example 1:
Let's assume that Primitive Header is enabled and user defined
12 dwords of per-primitive attributes.

Without packing we would consume 8 + ALIGN(12, 8) = 24 dwords of
MUE space and pass ALIGN(12, 8) = 16 dwords to fragment shader.

With packing, we'll consume 4 + 4 + ALIGN(12 - 4, 8) = 16 dwords of
MUE space and pass ALIGN(4, 8) + ALIGN(12 - 4, 8) = 16 dwords to
fragment shader.

16/16 is better than 24/16, so packing makes sense.

Example 2:
Now let's assume that Primitive Header is enabled and user defined
16 dwords of per-primitive attributes.

Without packing we would consume 8 + ALIGN(16, 8) = 24 dwords of
MUE space and pass ALIGN(16, 16) = 16 dwords to fragment shader.

With packing, we'll consume 4 + 4 + ALIGN(16 - 4, 8) = 24 dwords of
MUE space and pass ALIGN(4, 8) + ALIGN(16 - 4, 8) = 24 dwords to
fragment shader.

24/24 is worse than 24/16, so packing doesn't make sense.

This change doesn't affect vk_meshlet_cadscene in default configuration,
but it speeds it up by up to 25% with "-extraattributes N", where
N is some small value divisible by 2 (by default N == 1) and we
are bound by URB size.

Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20407>

intel/compiler/mesh: compactify MUE layout

Instead of using 4 dwords for each output slot, use only the amount
of memory actually needed by each variable.

There are some complications from this "obvious" idea:
- flat and non-flat variables can't be merged into the same vec4 slot,
  because flat inputs mask has vec4 stride
- multi-slot variables can have different layout:
   float[N] requires N 1-dword slots, but
   i64vec3 requires 1 fully occupied 4-dword slot followed by 2-dword slot
- some output variables occur both in single-channel/component split
  and combined variants
- crossing vec4 boundary requires generating more writes, so avoiding them
  if possible is beneficial

This patch fixes some issues with arrays in per-vertex and per-primitive data
(func.mesh.ext.outputs.*.indirect_array.q0 in crucible)
and by reduction in single MUE size it allows spawning more threads at
the same time.

Note: this patch doesn't improve vk_meshlet_cadscene performance because
default layout is already optimal enough.

Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20407>

radv: add radv_compile_cs() to compile a compute shader

This doesn't rely on the pipeline.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24280>

radv: stop using an array of binaries when compiling a compute shader

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24280>

Revert "intel/ci: disable iris-jsl-deqp because it always fails for an AMD MR"

This reverts commit da4b5b4a47ca727a7c8892d2bea50739df3b94ed.

Signed-off-by: Zhang Ning <zhangn1985@outlook.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23815>

nir/opt_copy_prop_vars: drop reuse of dynamic arrays

After the previous commit there are so few to reuse that this is no
longer worth doing and actually causes compilation to slow down.

The Blender shader compile time in issue #9326 improves as folows:
21.11 seconds -> 9.90 seconds

The CTS test dEQP-GLES31.functional.ubo.random.all_per_block_buffers.20
improves as follows:

0.92 seconds -> 0.68 seconds

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9326

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24227>

nir/opt_copy_prop_vars: skip cloning of copies arrays until needed

Most of the variables in the hash table will never actually be looked up
for any given block so cloning every possible value just creates a bunch
of unrequired memcpy calls.

Here we change the code to only clone the copies array once it is
actually looked up for the first time.

The Blender shader compile time in issue #9326 improves as folows:
151.09 seconds -> 21.11 seconds

The CTS test dEQP-GLES31.functional.ubo.random.all_per_block_buffers.20
improves as follows:

1.67 seconds -> 0.92 seconds

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24227>

nir/opt_copy_prop_vars: remove var hash entry on kill alias

If kill alias results in the hash table entry holding an empty
copies array then remove the hash entry and return the dynamic array
to the unused pool.

This helps avoid hash table size getting out of control in very large
shaders.

151.09 seconds -> 118.60 seconds

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24227>

nir/opt_copy_prop_vars: speedup cloning of copy tables

Here we change things to simply clone the entire hash table. This
is much faster than trying to rebuild it and is needed to avoid
slow compilation of very large shaders.

The Blender shader compile time in issue #9326 improves as folows:
251.29 seconds -> 151.09 seconds

The CTS test dEQP-GLES31.functional.ubo.random.all_per_block_buffers.20
improves as follows:

2.38 seconds -> 1.67 seconds

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24227>

nir/opt_copy_prop_vars: don't clone copies if branch empty

There is no point doing an expensive clone of the copies if the
if-branch is empty.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24227>

radeonsi: enable aco compile for mono merged ES/GS

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24204>

radeonsi: enable aco compile for mono merged LS/HS

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24204>

radeonsi: calculate lds size for merged shaders

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24204>

radeonsi: aco compile support merged mono shader

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24204>

radeonsi: refine si_llvm_es_build_end

1. merge si_set_es_return_value_for_gs into si_llvm_es_build_end
2. stop return value when mono mode in which case GS use ES input as
   input instead of ES output

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24204>

radeonsi: refine si_llvm_ls_build_end

1. merge si_set_ls_return_value_for_tcs into si_llvm_ls_build_end because they
   do the same job to return value
2. stop return value when mono mode with different thread count, in which case
   TCS use LS input as its input instead of LS output
3. use si_insert_input_ret_float

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24204>

radeonsi: remove param type check in wrapper function

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24204>

radeonsi: move vertex shader vb desc input sgpr args to last

ACO use same args for merged shader stages, but vb desc input sgpr args
is not present when second stage of merged shader. In order to share
same shaders args, move it to last so other args have same index.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24204>

radeonsi: simplify si_build_wrapper_function

We only need it to merge LS/HS or ES/GS now, prolog and epilog have
been lowered in nir already. So we just need to handle two parts and
they are sure to be first and second stage of a merged shader.

This also remove the needs SGPRs must be before VGPRs, which is required
by following commits to move some SGPRs after VGPRs.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24204>

radeonsi: init aco shader info for merged LS/HS

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24204>