svga: set clear_texture to NULL for vgpu9

With PIPE_CAP_CLEAR_TEXTURE removed, we need to set clear_texture to NULL
on svga vgpu9 device so it can use the fallback path.

Fixes: a1eabeff660 ("gallium: remove PIPE_CAP_CLEAR_TEXTURE")

Reviewed-by: Neha Bhende <bhenden@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24264>

ci/venus: update venus-lavapipe expectations

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24263>

lvp: avoid reading immutable sampler from desc write info

Lavapipe has switched to layer push descriptor support atop descriptor
updates internally since 12a7fc51c77925a5562fd104a8fbd664a46ffc8b, so
it must skip retrieving immutable samplers from the write info even if
the update call itself is blessed by the spec to not hit that case.

Fixes: 12a7fc51c77 ("lavapipe: Rework descriptor handling")
Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24263>

vulkan: bump header register to 1.3.258

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24279>

tu, freedreno/a6xx: Remove has_ccu_flush_bug

Based on the previous commit, this isn't actually a bug and is expected
behavior. Turnip should already be handling it correctly for user
flushes, we just have to make sure to handle it for flushes we insert
ourselves in turnip and freedreno.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24162>

tu: Fix and simplify execution dependency handling

When I wrote this code, I was under the impression that at most one
context from each cluster could be executing at a time. This would mean
that we could treat clusters as pipeline stages and only insert a WFI if
there was a bubble where an earlier stage depends on the result of a
later stage.

This mental model was wrong, though. Experiments on a6xx show it's
possible for two contexts to be executing simultaneously, even though
there are only two contexts - register writing is just stalled until the
earliest-launched context finishes.

This means that the mental model is now much simpler. Any draw can, in
theory, execute in parallel with any previous draw, blit, flush, etc,
although it seems that flushes do wait for any earlier work to finish.
Clusters are mostly just an implementation detail that only matter in
some corner cases, like setting a non-context register (written in the
last cluster) that is used by an earlier cluster that can race ahead of
the write.

An example where this makes a difference is a fragment shader that
writes an image via stib followed by a blit from that same image.
Because both operations happen in the same cluster and use the same
cache, we wouldn't emit anything in the barrier, however actually we
still need to WFI.

This was getting worse on a7xx because later clusters now have 4
contexts, making it easier for draws to be executed in parallel. However
AFAICT it was already a problem on a6xx.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24162>

tu: Fix vk2tu_*_stage flag type

New flags were silently getting dropped.

Fixes: 59259a01671 ("tu: Convert to sync2 entrypoints")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24162>

docs: update calendar for 23.1.4

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24281>

docs: add sha256sum for 23.1.4

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24281>

docs: add release notes for 23.1.4

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24281>

gallium/auxiliary/vl: Fix RGB->YCbCr full range matrix

Also rename it to bt_709_rev_full as there already
is bt_709 which is used for YCbCr->RGB.

Fixes: 8a21efce3a2 ("frontends/va: Add postproc support for converting to full range")

Reviewed-by: Leo Liu <leo.liu@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24238>

radeonsi/vcn: num bs_bufs must be proportional to num jpeg engines

using limited number of bs buffers constraints the simultaneous
use of all available jpeg engines especially when count is lesser than
that of the available engines. make sure the number of buffers
available are more than or equal to the number of jpeg engines on the asic.

Signed-off-by: Sathishkumar S <sathishkumar.sundararaju@amd.com>
Reviewed-by: Leo Liu <leo.liu@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24240>

radeonsi/vcn: support variable number of bs_bufs

add support to use variable number of bitstream buffers for decode

v2: remove the always true if condition (CI report)

Signed-off-by: Sathishkumar S <sathishkumar.sundararaju@amd.com>
Reviewed-by: Leo Liu <leo.liu@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24240>

radv: bind the pre-compiled PS epilog to the cmdbuf state

For PS epilogs we have two paths, the first one is to pre-compile PS
epilogs at pipeline creation time, while the second one is to compile
PS epilogs on-demand when some dynamic states are used.

Binding the pre-compiled PS epilog to the cmdbuf state allows us to
remove one more pipeline dependency when recording cmdbufs (for shader
objects).

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24254>

radv: pass a shaders array for computing ia_multi_vgt_param

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24251>

radv: remove unused param in radv_pipeline_emit_vgt_gs_out()

Also rename the function.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24251>

radv: stop using a pipeline for emitting VGT_VERTEX_REUSE_BLOCK_CNTL

Passing a TES is enough.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24251>

nir: Remove register arrays

Nothing produces them any more, so remove them from NIR. This massively reduces
the size of nir_src, which should improve performance all over.

nir_src size reduced from 56 bytes -> 40 bytes (pahole results on arm64, x86_64
should be similar.)

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24253>

nir: Rename lower_locals_to_reg_intrinsics back

The short name is freed up.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24253>

nir: Remove nir_lower_locals_to_regs

No more users, all switched to the intrinsic version.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24253>

panfrost: Remove unused helpers

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24253>

intel/fs: Don't read reg.base_offset

It's not set in the new intrinsics path.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24253>

anv: drop CFE state validation checks

anv no longer needs to track if the CFE state is valid since we ensure
that the state is valid at pipeline creation time.

Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23934>

anv,iris: program the maximum number of threads on compute queue init

Fixes: 90a39cac87 ("intel/blorp: Emit compute program based on BLORP_BATCH_USE_COMPUTE")
Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23934>

etnaviv: nir: lower nir_texop_txs

Non of the GPU models know at this time have hardware support to
retrieve the dimensions of a level of a texture. Do almost the
same as the binary blob and store the needed values as uniforms.

Passes dEQP-GLES3.functional.shaders.texture_functions.texturesize.*

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24217>

etnaviv: nir: support intrinsic used for txs lowering

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24217>

nir: add enta specific intrinsic used for txs lowering

Non of the know etnaviv GPUs support this feature in hardware
and the binary blob provides sizes via uniforms too.

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24217>

etnaviv: move nir texture lowerings into one pass

This is just a prep commit to keep all texture related
lowerings in one c file.

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24217>

anv: merge cases leading to the same code

Added in: 688968e8880 ("anv: add support for direct descriptor in allocation/writes")

Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24260>

anv: drop unused function

Added in: 02cecffe2bb ("anv: add a pass to partially lower resource_intel")

Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24260>

intel/compiler: remove redundant code

has_lsc is checked few lines above, so this code doesn't matter.

Added in: a358b97c586 ("intel/fs: optimize uniform SSBO & shared loads")

Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24260>

anv: use ycbcr_info for P010 format

Since !24096 landed, we can just use ycbcr_info to get information
of an image of the P010 format.

Signed-off-by: Hyunjun Ko <zzoon@igalia.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24265>

nouveau: Delete the nouveau_compiler tool

It's TGSI only.

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24175>

nouveau: Drop BuildUtil::Location

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24175>

nouveau: Drop BuildUtil::DataArray

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24175>

nouveau: Drop ConverterCommon::Subroutine

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24175>

nouveau: Drop tgsi support from nv50_ir_prog_info

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24175>

nouveau: Delete nv50_ir_from_tgsi.cpp

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24175>

nv50: Keep nir directly in nv50_program

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24175>

nvc0: Keep nir directly in nvc0_program

instead of under pipe_shader_state.

This makes it obvious that we never produce tgsi shaders since
c3cbe610 "nouveau: Delete the NV50_PROG_USE_TGSI env var."

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24175>

zink: emit SpvCapabilitySampleMaskPostDepthCoverage with SpvExecutionModePostDepthCoverage

can't have one without the other

cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24183>

zink: be even dumber about buffer refs when replacing storage

these extra checks can cause issues when multiple contexts and transfer
ops are involved

cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24141>

nvc0: fix num_gprs for Volta+

Overallocating by 2 gprs for ugprs is a wild guess by me. It does make
sense though as each subgroup shares 64 ugprs and that's 2 per thread.

Signed-off-by: Karol Herbst <git@karolherbst.de>
Reviewed-by: M Henning <drawoc@darkrefraction.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24261>

llvmpipe: enable system SVM

The API bits are already implemented in clover and rusticl and by
definition a CPU driver implements SVM.

This should allow anybody to work on proper SyCL/CHIP-SPV support for
rusticl running llvmpipe.

Signed-off-by: Karol Herbst <git@karolherbst.de>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24092>

rusticl/mesa: make svm_migrate optional

It's just a hint and drivers might want to ignore implementing it for now.

Signed-off-by: Karol Herbst <git@karolherbst.de>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24092>

intel/isl: Add a score for DG2_RC_CCS

This enables the DG2 render compression modifier in anv. When I tested
this against vkcube, I observed that the full resolve which happened at
the end of every frame was converted to a partial resolve, allowing the
framebuffer to retain compression.

According to Caleb Callaway's testing, enabling this modifier positively
impacts the FPS of the following game benchmarks:

 - Strange Brigade.vk-g6              +12.78%
 - Strange Brigade.dx12vk-g6          + 9.33%
 - Shadow of the Tomb Raider.vk-g6-lx + 2.37%
 - Dota 2 (replay Jul 2020).vk-g6     + 2.28%

Thanks to Felix Degrood for pointing out that Strange Brigade would
benefit from this optimization.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24120>

intel/isl: Move the Tile4 modifier score case down

Group modifiers by platform first, then the score. I find it easier to
read this way.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24120>

intel: Describe modifier compression with booleans

Replace the aux_usage field with two booleans: one for render
compression and one for media compression.

This more accurately describes how CCS_E is used on gfx12. On those
platforms, the FCV feature may be enabled or disabled, but ISL's
modifier table has been using the FCV aux-usage for every gfx12 render
compression modifier. Instead, set the newly-added render compression
boolean to true.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24120>

iris: Swap stencil and modifier aux assignment order

Makes the next patch clearer.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24120>

hasvk: Delete modifier with aux code

Modifiers with compression are not supported.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24120>

crocus: Delete modifier with aux code

Modifiers with compression are not supported.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24120>

iris: Reduce accesses of mod_info->aux_usage

This field will be replaced in an upcoming patch.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24120>

anv: Reduce accesses of isl_mod_info->aux_usage

This field will be replaced in an upcoming patch.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24120>

anv: Handle explicit surface layout of DG2_RC_CCS

We're going to enable the DG2 modifier. Account for the reduced plane
count that exists with it.

Also add an assert to make it clearer that the aux in use is CCS.
Otherwise, it may not be obvious because of the generic compression
names being used here.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24120>

intel: Add and use isl_drm_modifier_get_plane_count

We're going to enable the DG2_RC_CCS modifier in anv. Add and use this
function to prepare for the new plane count that comes with that
modifier.

iris is left alone for now because it supports more modifiers than
isl_drm_modifier_get_score is aware of.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24120>

anv: Don't support ASTC images with modifiers

Before this change, anv_get_image_format_features2 reported support for
ASTC formats with any modifier (even those not supported by anv). But,
we didn't intend to support that compressed image format with modifiers.

With this change, the format feature function reports no support for
modifiers on ASTC-formatted images.

This prevents the next patch from causing assertion failures due to
unsupported modifiers.

Fixes: 355f3188438 ("anv: Allow transfer-only linear ASTC images")
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24120>

iris: Remap DRM_FORMAT_MOD_INVALID more often during import

We'd eventually like to use an ISL helper that doesn't support
DRM_FORMAT_MOD_INVALID. Prepare for this by replacing the invalid value
with the modifier associated with the BO's tiling in more cases.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24120>

anv: use the correct GFX_VERx10 macro for WA

Fixes: 60b0d2c2cbea ("add required invalidate/flush for Wa_14014427904")
Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23937>

anv: use the WA infrastructure where possible when generating state

Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23937>

radeonsi: fix a CDNA regression breaking compute

reported internally

Fixes: 315231b5a519bdc14 - radeonsi: eliminate redundant compute SH register changes

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24169>

nir/print: print instr pass_flags

From time to time it can be helpful to "see" the pass_flags.

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24234>

turnip: flush cache for dstBuffer in vkCmdCopyQueryPoolResults

There can be other writes to the dstBuffer gated by proper barriers
beforehand.

TEST=dEQP-VK.pipeline.*.timestamp.* with Venus on Turnip

Fixes: 487aa807bd1b ("tu: Rewrite flushing to use barriers")
Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24245>

ir2: Switch to nir_legacy

Addresses a2xx portion of #9051.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Rob Clark <robclark@freedesktop.org>
Tested-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24118>

asahi: Advertise Z16_UNORM

This works (on the downstream kernel, anyway).

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

asahi: Execute preambles for background programs

This will be useful when spilling render targets.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

asahi: Offset clear colour uniform by 4

Frees up u0_u1 for a bindless base address which will make render target
spilling easier to implement.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

asahi: Ignore spilled render targets for background load

Nothing to reload.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

asahi: Permit meta shaders to use preambles

Preambles are occassionally useful with background programs.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

asahi: Lower multisample image stores

These will be used for spilling multisampled render targets.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

asahi: Lower tilebuffer access for spilled RTs

Conceptually similar, we just don't have the tilebuffer available this time.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

asahi: Extract some tilebuffer lowering code

In prep for spilling. No functional change.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

asahi: Ignore spilled render targets with partial renders

Partial renders exist to the spill the tilebuffer to memory, there's nothing to
do if it's already spilled (and would just waste memory bandwidth and create a
feedback loop).

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

asahi: Ignore spilled render targets in EOT shaders

Regardless whether we implement Apple-style eMRT or something simpler, the EOT
shader isn't involved here.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

asahi: Do not support masking with spilled RTs

Extra complexity for this interaction, not worth it until we have an actual use
case IMHO.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

asahi: Add agx_tilebuffer_spills query

We can skip various work in the driver if we're not spilling render targets.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

asahi: Introduce concept of spilled render targets

To accommodate framebuffers which exceed tilebuffer limits, we'll need to spill
render targets to main memory. In effect, we need to emulate an immediate-mode
renderer for some render targets. This decision is made on a per-render target
basis. In our tilebuffer layout calculation, rather than asserting that all
render targets fit, introduce a notion of spilling.

This doesn't actually implement spilling -- it just pushes the assert failure
down to the users. But it's progress.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

asahi: Extract sampler_view_for_surface

We'll reuse this logic for the spilled RT case.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

agx: Plumb in coverage mask

This is internally used by the hardware when writing to the tilebuffer. We need
to use it externally to spill multisample render targets.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

agx: Require tag writes with side effects

Otherwise the fragment shader might be skipped entirely. (Possibly this is the
wrong approach to this though...)

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

agx: Add simple image fencing pass

Minimum needed to pass CTS.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

agx: Implement fence_*_to_tex_agx intrinsics

We need these fencing intrinsics because our image caches aren't coherent with
memory. Furthermore, we need some sync intrinsics for imageblocks (which are
spicy images). These are a stub of what the final fragment shader interlock
implementation will look like, or what a real Metal-grade imageblock
implementation needs, but this is good enough for handling the sync requirements
with spilled render targets.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

agx: Don't emit silly barriers

Trust in the scoped_barrier.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

agx: Emit global memory barriers for images

This is part of image atomics, since those go through the regular memory path.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

agx: Implement image_load

Texture loads can be reordered freely but image loads can't be (since there
could be writes). Implement image_load natively to avoid subtle problems with
CSE and scheduling.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

agx: Extract texture write mask handling

image_load will share the logic.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

agx: Add image_load opcode

This is equivalent to texture_load but cannot be reordered, since it might be
writeable.

It also sets bit 43. This needs more investigation, but it fixes
KHR-GLES31.core.shader_image_load_store.basic-glsl-misc-fs. Some sort of cache
control bit.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

asahi,agx: Fix txf sampler

Bizarrely, the clamps/wrap modes are respected so we need to set them
appropriately for correct out-of-bounds behaviour (returning all zero). That in
turn means we can't use whatever sampler is already there, instead we need to
allocate a dedicated sampler just for txf. Good news is we have an extra sampler
state register available for the purpose.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

agx: Lower buffer images

Similar to buffer reads, we need to implement buffer images as 2D images with
fixed width and some lowering code.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

agx: Lower image atomics

Lower image atomics to texel address loads, and lower texel address loads to
arithmetic and descriptor reads. This implements image atomics.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

agx: Extract texture_descriptor_ptr_for_* helpers

For implementing image_texel_address, when there's no point in creating an
internal texture instruction just to lower immediately.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

agx: Extract coords_for_buffer_texture helper

The mapping of 1D -> 2D coordinates for indexing into buffer textures (lowered
to fixed-width 2D images) will be shared between both texture load and image
store code paths, so pull it out.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

agx: Add interleave opcode

We'll use it for texture atomics.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

agx: Handle early_fragment_tests

Simply doing nothing fixes
dEQP-GLES31.functional.image_load_store.early_fragment_tests.*. However, we need
to actually insert the sample_mask instruction to make sure the shader runs at
all (I think), doing that fixes:

   KHR-GLES31.core.shader_image_load_store.basic-glsl-earlyFragTests

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

agx: Implement image barriers

Or cache flushes or whatever these actually are. Probably could be optimized
once we understand what the 4 individual instructions are actually doing. Fixes
dEQP-GLES31.functional.image_load_store.2d.qualifiers.*.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

agx: Wait for outstanding stores before barriers

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

agx: Handle frag side effects without render targets

We still need to insert our lowering code, though this case could probably be
optimized somehow. Fixes a massive number of KHR-GLES3 and KHR-GLES31 tests,
including
KHR-GLES31.core.shader_atomic_counters.advanced-usage-many-draw-calls2 and lots
of PBO tests.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

agx: Translate image_store from NIR

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

agx: Translate texture bindless handles

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

agx: Pack bindless textures

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>

agx: Handle bindless properly for txs lowering

When I wrote this pass I mostly guessed what our bindless handles would look
like. Now that we know we can do it right.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24258>