freedreno: Use explicit QCOM_TILED3 modifier

Now that this is in upstream drm_fourcc.h we can drop the private
internal modifier.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25465>

freedreno: Indentation fix

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25465>

tu/kgsl: Support u_trace and perfetto

Raw GPU time is retrieved via kgsl_cmdbatch_profiling_buffer,
offseted GPU time is retrieved via KGSL_PERFCOUNTER_GROUP_ALWAYSON.
This allows to calculate GPU time offset for each submission and
synchronize CPU/GPU time domains.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12805>

tu/kgsl: Fix memory leak of tmp allocations during submissions

cc: mesa-stable

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12805>

tu/perfetto: Allow gpu time to be passed into tu_perfetto_submit

In preparation to support perfetto on KGSL, on KGSL GPU time is
retrieved on submission and requires minimal post-processing.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12805>

tu/perfetto: Remove now unnecessary tu_perfetto_util

Since Turnip now uses C++ we can directly include tu_device.h
into perfetto code.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12805>

nir/lower_subgroups: use intrinsic builder more

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25501>

nir: make quad intrinsic dst bit size match src0

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25501>

isaspec: Add BitSetEnumValue object

There might be cases where you describe an enum in isaspec and want it to use
for decoding but also for codegen with e.g. mako.

Lets have a look at the following exmaple:

<enum name="#cond">
<value val="0" display=""/>    <!-- always: display nothing -->
<value val="1" display=".gt"/>
...
</enum>

In the decoding case we want that nothing gets displayed if #cond has the value of "0". For
codegen with mako this could result in the following C code:

enum PACKED cond {
   COND_ = 0,
   COND_GT = 1,
   ...
};

What you really want is this:

enum PACKED cond {
   COND_ALWAYS = 0,
   COND_GT = 1,
   ...
};

To make this possible introduce BitSetEnumValue class which represents
an isaspec xml enum. It holds the value, displayname and now a name.

With the  __str__ method the old behaviour is still intact.

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25451>

isaspec: Add support for custom meta information

Allows every user of isaspec to add custom meta information
to any bitset. This can be quite handy if you want to know
how many sources your instruction have, if this instruction has
a dest or provide some names for sources that can be used
by some debug tools.

All you need is to sprinkle <meta> tags in the xml file and
provide custom attributes.
  <meta has_dest="1" num_sources="3" .. />

With get_meta() method of any bitset you can get access to
the dict with all the attributes. get_meta() walks the whole
tree to collect all <meta> tags on the way to the root node.

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25451>

isaspec: Add method to get all instrustions

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25451>

isaspec: encode: Correct used regex

The current regex misses the = sign and therefore fails to match
DST:align=16.

Fixes: 9e56f69edf5 ("isaspec: encode: handle special fieldname properties")
Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25451>

isaspec: Add support for templates

If you have a repeating <display> substring you can replace this
substring with a template and reference the template name instead.

Saves from doing lot of copy&paste and makes general changes to the
substring much easier.

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25451>

v3dv/ci: update expected list

Remove dEQP-VK.api.driver_properties.conformance_version, as the version
it required is now the one used in the CI.

Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25518>

radv: enable DCC for MSAA images on GFX11

This seems to be working now! I suspect either the "recent" addrlib
update fixed it or recent comp-to-single fast clear fixes.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8326
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25448>

tu: add a TU_DEBUG=rd option for cmdstream dumping

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25246>

freedreno: move redump.h to common code + cleanup

remove the unused parts and add an implementation of rd_write_section

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25246>

Revert "ci: disable Collabora's LAVA lab for maintance"

This reverts commit https://gitlab.freedesktop.org/mesa/mesa/-/commit/ccd3e68146fd7ee8732d497f367a0574e4cf84a8

Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25369>

isl: Use 16-bit instead of 8-bits for surface format info fields

Comparing uint8_t max value 255 with devinfo->verx10 will work fine for
now but for future platforms, comparison will fail. To avoid this
let's switch the field data type from 8-bits to 16-bits.

v1: (Jordan)
- Use 16 bits instead of 32 and add assertion.

Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25478>

radv/amdgpu: fix alignment of command buffers

Fixes other recent regressions.

Fixes: 4f660f99 ("ac/gpu_info: pad IBs according to ib_size_alignment")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25502>

ci: make B2C_JOB_VOLUME_EXCLUSIONS to all .b2c-test jobs

Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25429>

aco: remove -0.0 for 32 bit fsign with mul_legacy/omod when denorms are flushed

v_mul_legacy_f32 and omod return +0.0 if any operand is +0.0/-0.0.

Foz-DB Navi21:
Totals from 4289 (5.60% of 76572) affected shaders:
Instrs: 8100571 -> 8099319 (-0.02%); split: -0.02%, +0.00%
CodeSize: 43433200 -> 43435088 (+0.00%); split: -0.01%, +0.01%
Latency: 88151566 -> 88147232 (-0.00%); split: -0.00%, +0.00%
InvThroughput: 22966705 -> 22965192 (-0.01%); split: -0.01%, +0.00%
VClause: 190010 -> 190009 (-0.00%)
SClause: 269697 -> 269689 (-0.00%)
Copies: 687294 -> 687296 (+0.00%); split: -0.00%, +0.00%

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25347>

aco/optimizer: copy propagate to output modifier instructions

Foz-DB Navi21:
Totals from 847 (1.11% of 76572) affected shaders:
Instrs: 2331245 -> 2330335 (-0.04%); split: -0.04%, +0.00%
CodeSize: 12451040 -> 12451736 (+0.01%); split: -0.00%, +0.01%
Latency: 26230953 -> 26229153 (-0.01%); split: -0.01%, +0.00%
InvThroughput: 6297802 -> 6296788 (-0.02%); split: -0.02%, +0.00%
VClause: 64527 -> 64528 (+0.00%); split: -0.00%, +0.01%
SClause: 73150 -> 73121 (-0.04%); split: -0.06%, +0.02%
Copies: 180083 -> 179172 (-0.51%); split: -0.53%, +0.02%
PreSGPRs: 62311 -> 62316 (+0.01%)
PreVGPRs: 51720 -> 51710 (-0.02%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25347>

aco/optimizer: check if we can use omod before labeling it

Allows to use omod for v_mul_legacy_f32 regardless of signedZeroInfNaNPreserve

Foz-DB Navi21:
Totals from 15 (0.02% of 76572) affected shaders:
Instrs: 20131 -> 20113 (-0.09%)
CodeSize: 107100 -> 107144 (+0.04%)
Latency: 400789 -> 400470 (-0.08%)
InvThroughput: 62342 -> 62278 (-0.10%)
Copies: 1194 -> 1176 (-1.51%)
PreVGPRs: 787 -> 785 (-0.25%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25347>

radv/ci: update list of flakes for NAVI10/VEGA10

This one is fixed since CTS 1.3.6.3.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25505>

radv/ci: update list of expected failures on PITCAIRN

This one is fixed since CTS 1.3.6.3.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25505>

radv: fix alignment of DGC command buffers

Otherwise, DGC command buffers might not be correctly aligned.

This fixes a regression with the vkd3d-proton DGC tests.

Fixes: 4f660f99378 ("ac/gpu_info: pad IBs according to ib_size_alignment")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25500>

intel/genxml: remove HDC from gen11.xml, it is not available

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25399>

iris: HDC flush is available only for GFX_VER 12+

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25399>

anv: HDC flush is available only for GFX_VER 12+

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25399>

iris: flush data cache when flushing HDC on GFX < 12

This matches what anv driver does.

Fixes: a969ad1d ("iris: Demote DC flush to HDC flush in cache tracker")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6314
Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25399>

aco: remove zero offset optimization

This is done in nir_opt_constant_folding now.

No fossil-db changes on navi31.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25477>

nir/constant_folding: remove zero texel offset

fossil-db (navi31):
Totals from 7 (0.01% of 79330) affected shaders:
Instrs: 7001 -> 6993 (-0.11%)
CodeSize: 35736 -> 35692 (-0.12%)
InvThroughput: 3232 -> 3229 (-0.09%)
Copies: 552 -> 549 (-0.54%)
PreSGPRs: 277 -> 273 (-1.44%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25477>

aco: disable zero offset optimization for strict WQM coords

If we try to do this, we end up using {undef,coordx} as the coordinates
for an image_sample instruction, because we can't shrink the linear VGPR.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9767
Fixes: 859e059aa912 ("radv: use fix_derivs_in_divergent_cf")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25477>

nir: scalarize masked_swizzle_amd created from shuffle_xor

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9901
Fixes: 0ef87f148df ("nir/lower_subgroups: Don't do multiple lowerings at once")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25468>

iris/anv: move Wa_14018912822 as a drirc workaround

This should be toggled on only for applications that hit the issue.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9886
Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25424>

iris: correct dst alpha blend factor in Wa_14018912822

Fixes: 0e9a26372bb4 ("iris: implement Wa_14018912822")
Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25424>

anv: fix internal compute copy shader build

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9907
Fixes: 2cc5b3b1e0 ("anv: add a memcpy compute internal kernel")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25480>

docs: Move isaspec out of drivers/freedreno

Lets put it under 'Developer Topics'.

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Acked-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25452>

v3d: get rid of shader_state pointer in v3d_key

Having this pointer in the key is undesirable since it makes
copying keys difficult and error prone (as seen in previous
patches), also, it is only there for convenience and we don't
strictly need it (in fact the vulkan driver doesn't use it at
all), so let's just get rid of it so our v3d_key is fully
static.

Reviewed-by: Juan A. Suarez <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25418>

v3d: fix RAM shader cache

The RAM shader cache was using the v3d_key for hashes and
comparisons which is not correct. Particularly, this struct
has a void pointer where we store a reference to an uncompiled
shader with the NIR code, and that is of course not accounted
for when hashing and comparing keys, which can lead to bogus
cache hits.

This patch introduces a v3d_cache_key that has both the v3d key
and a sha1 of the uncompiled NIR. Now key hashing and comparison
is done on the static part of the v3d key (that is, excluding the
uncompiled shader pointer, which may be invalid in the cache if
the original shader was deleted) and taking the sha1 from the
uncompiled shader. This also makes sure the shader key we store
in the cache has a NULL shader_state pointer to make it more
clear that this field may not be used at all for caching purposes.

This fixes GPU hangs with some OpenCL tests (through Rusticl)
caused by incorrect RAM cache hits.

Reviewed-by: Juan A. Suarez <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25418>

v3d: use pre-computed shader sha1 for disk cache

Reviewed-by: Juan A. Suarez <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25418>

v3d: compute nir sha1 for uncompiled shader state

Reviewed-by: Juan A. Suarez <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25418>

broadcom/compiler: add a couple of shader key helpers

Our shader key includes a void pointer that we can't just memcmp,
so add helpers that allow us toget the 'static' portion and size
of a key. We will use this to fix up the shader cache in v3d in
a later patch.

Reviewed-by: Juan A. Suarez <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25418>

ci: disable Collabora's LAVA lab for maintance

This is to inform you of some planned downtime in the LAVA lab as follows:
* Start: 2023-10-02 08:00 BST (07:00 UTC)
* End: 2023-10-02 12:00 BST (11:00 UTC)

Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25368>

ci/vkcts-navi21: mark more of the RT handles checks as flakes

We keep hitting more and more of them, so let's be more inclusive.

Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25495>

ci/vkcts-vangogh: mark dEQP-VK.dynamic_rendering.primary_cmd_buff.basic.* as flake

This mirrors what we did on navi21, as there are just too many of these tests.

Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25495>

asahi: decode: Fix uint64_t format modifiers in agxdecode_stateful()

Fixes i386 build.

Fixes: acd5ed0451d6 ("asahi: decode: Implement VDM call/ret")
Signed-off-by: Janne Grunau <j@jannau.net>

asahi: Handle layered background programs

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

asahi: Generate layered EOT programs

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

asahi: Use a 2D Array texture for array render targets

Fixes KHR-GLES31.core.geometry_shader.layered_framebuffer.blending_support with
eMRT forced.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

asahi: Write to cubes/etc attachments as 2D array

To reduce shader variants, the tilebuffer lowering code does not know the
actual texture targets of the spilled render targets, only whether they are
layered or not. As such, all layered targets (3D, cube map, etc) get written out
uniformly as 2D Arrays. For that to work, the driver needs to do the
corresponding transform.

Regular imageStore() instructions are not affected by any of this.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

asahi: Predicate layer ID reads

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

asahi: Add pass to predicate layer ID reads

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

asahi: Assume LAYER is flat-shaded

It can't be anything else, this makes sure the varyings are sorted properly.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

asahi: Account for layering for attachment views

Do not force a single-layer view, use an actual array attachment when there are
multiple layers, since this corresponds to a layered framebuffer that will write
to an array with the eMRT path.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

asahi: Expose VS_LAYER_VIEWPORT behind a flag

We can't technically expose the extension without a higher GL version, but the
implemented subset should work and this lets us test with piglit.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

asahi: Use layered layouts

For correct eMRT code.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

agx/lower_tilebuffer: Support spilled layered RTs

If we spill render targets with a layered framebuffer, our spilled targets are
assumed to be 2D Arrays (in general). We need to use arrayed image operations to
load/store from these. The layer is given by the layer as read in the fragemnt
shader. This handles the eMRT portion of layered rendering.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

agx/tilebuffer: Support layered layouts

Just add a flag for it. We don't care about the actual # of layers when
calculating the layout, only the boolean fact of being layered or not. The
reason we need this at all is because the eMRT implementation needs to
account for layering and that is only keyed off the tilebuffer layout.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

agx: Support packed layered rendering writes

With the new pass.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

asahi,agx: Select layered rendering outputs

These 2 are together

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

asahi: Add helper to get layer id in internal program

For background/EOT only.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

agx: Add pass to lower layer ID writes

The hardware needs the layer ID and the viewport index packed together. That
consumes an entire varying slot, if we want those available in the frag shader
we need a separate slot. Add a pass to insert the extra packed write.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

agx: Handle layered block image stores

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

agx: Pack block image store dim correctly

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

agx/nir_lower_texture: Allow disabling layer clamping

For background program with layered.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

nir: Support arrays in block_image_store_agx

For layered rendering, runs once per layer.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

nir: Add layer_id_written_agx sysval

We'll implement layer ID reads in the frag shader with a varying read, but if
the VS doesn't write the varying we need to return 0 per the spec. Add a sysval
to detect that case so we can handle it at runtime without keys.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

agx: Insert jmp_exec_none instructions

With the exception of the backwards branch for loops, all the control flow we
insert during instruction selection just predicates instructions rather than
actually jumping around. That means, for example, we execute both sides of the
if even for a uniform condition! That's inefficient. The solution is insert
jmp_exec_none instructions after control flow in order to skip unexecuted
regions, which is much faster than predicating them out. However, jmp_exec_none
is costly in itself, so we need to use a heuristic to determine when it's
actually beneficial.

This uses a very simple heuristic for this purpose. However, it is a massive
performance speed-up for Dolphin uber shaders: 39fps -> 67fps at 2x resolution.
Nearly a doubling of performance!

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

agx: Add agx_prev_block helper

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

agx: Add jumps to block ends

jmp_exec_none variant that jumps to the last instruction of the target block,
rather than the beginning. This is convenient for skipping over elses, while
still executing the block-final pop_exec instruction. Similarly for skipping
over loop bodies while still executing the block-final pop_exec, after break
instructions.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

agx: Augment if/else/while_cmp with a target

Add an optional pointer to a target block for these instructions. This does NOT
act like a logical branch, and does NOT get added to the logical control flow.
It is ignored wholesale until after RA, when physical edges may be inserted by a
pass we add later in this series.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

agx: Set PIPE_SHADER_CAP_CONT_SUPPORTED

So we get adequate testing of continues, rather than lowering them in GLSL. We
don't really /want/ to see continues but lowering them away will just make them
harder to test... and besides, we should be optimizing them in NIR (not GLSL) so
we can get the win on Vulkan too.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

agx: Hoist sample_mask/zs_emit

Although this is well-motivated, perf effect seems to be neglible for Dolphin.
It does prevent the scheduler from making things worse by sinking these
instructions though, so as a way to prevent future problems this seems sensible.

The kind of problem this affects (late discard) isn't modelled in shader-db.
Nevertheless, nothing concerning there:

   total instructions in shared programs: 1756699 -> 1756722 (<.01%)
   instructions in affected programs: 10106 -> 10129 (0.23%)
   helped: 21
   HURT: 41
   Inconclusive result (value mean confidence interval includes 0).

   total bytes in shared programs: 11525404 -> 11525452 (<.01%)
   bytes in affected programs: 72900 -> 72948 (0.07%)
   helped: 27
   HURT: 41
   Inconclusive result (value mean confidence interval includes 0).

   total halfregs in shared programs: 483394 -> 483286 (-0.02%)
   halfregs in affected programs: 4945 -> 4837 (-2.18%)
   helped: 88
   HURT: 78
   Inconclusive result (value mean confidence interval includes 0).

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

agx: Align the reg file for 256-bit vectors

This fixes live range splitting with 3D textureGrad(), which involves vectors
larger than the natural 128-bit maximum and hence requires special handling.
Fixes this assert with a combination of debug flags and new patches:

   unsigned int find_best_region_to_evict(struct ra_ctx *, unsigned int,
   unsigned int *, unsigned int *):
   Assertion `(rctx->bound % size) == 0 && "register file size must be aligned
   to the maximum vector size"' failed

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

asahi: Clamp index buffer extent to what's read

This makes for cleaner agxdecodes, I think this matches what I've seen on the
macOS side but I might be misremembering. Certainly shouldn't hurt.

This only applies for direct draws.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>

radv: Fix check in insert_block

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25496>

radv: Initialize shader freelist on allocation

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25496>

r600: Replace R600_BIG_ENDIAN with UTIL_ARCH_BIG_ENDIAN

In particular, removes the dependency of r600_formats.h on r600_pipe.h so
it can be shared between Gallium and Vulkan.

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24513>

winsys/amdgpu: pad gfx and compute IBs with a single NOP packet

to minimize CP overhead

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25043>

ac/gpu_info: pad IBs according to ib_size_alignment

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25043>

ac/gpu_info: replace ib_alignment with per-IP IB base and size alignments

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25043>

ci/deqp-runner: restore exit-on-error after getting deqp-runner's exit code

Signed-off-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24738>

ci/deqp-runner: fix indentation

Signed-off-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24738>

amd: rename GFX110x to NAVI31-33

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25492>

ac/llvm: fix flat PS input corruption

Fixes: 0a54fbb5b40fc4e4a - radeonsi/gfx11: interp changes for 32bit

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25492>

ac/gpu_info: don't align IBs to the GL2 cache line size

PAL doesn't do it. If drivers want IBs not to share cache lines with other buffers,
they should align the size manually.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25492>

ac/llvm: set !fpmath 3.0 for llvm.sqrt

LLVM will be changed to increase precision with additional instructions,
which we don't want.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25492>

ac/perfcounter: remove a bogus assert to fix an assertion failure on gfx11

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25485>

ci: move shader-db clone/build into its own script

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24504>

ci: drop unused shader-db clone + build from fedora image

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24504>

ci: drop unused shader-db clone + build from alpine image

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24504>

radv: Do not sync after radv_update_buffer_cp

WR_CONFIRM is set.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24846>

radv: Merge the sync_data and header initialization

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24846>

radv/bvh/ploc: Load child bounds from LDS

The bounds are already in LDS so there is no need to load them from
VRAM.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24846>

docs: document the merging process and what is allowed or not

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25117>

virgl:Fix ITEM_CPY macro pointer copy bug

The ITEM_CPY macro uses the memcpy function to copy the item variable.
When item is a pointer, the memcpy function will copy the value of the
pointer, not the address that the pointer points to.

Signed-off-by: Cong Liu <liucong2@kylinos.cn>
Reviewerd-by: Feng Jiang <jiangfeng@kylinos.cn>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25453>

ci/anv: Catch some flakes

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25469>

ci/zink: Found some flakes

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25469>