v3dv: fix event synchronization

Since we now implement events in the GPU we need to be more careful
and insert barriers to honor the dependencies provided by the API
as well as ensuring we are synchronizing these with the compute
queue, since that is how we implement GPU event functionality.

Fixes: ecb01d53fdb413121ce2 ("v3dv: refactor events")
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19458>
(cherry picked from commit e6884df088a1e172cd87956fe9afdac63069eb2c)

v3dv: make the helper to emit pipeline barriers public to other files

Fixes: ecb01d53fdb413121ce2 ("v3dv: refactor events")
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19458>
(cherry picked from commit 8113f973b3946e465d508627a4796d94cec1f1d8)

v3dv: always check VK_ACCESS_2_MEMORY_READ_BIT for read accesses

Fixes: a981ac053964 ('v3dv: skip binning sync if binning shaders don't access external resources')

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19458>
(cherry picked from commit 67e82fd1f2c334da2e888553967ebae04c08146a)

v3dv: increase limit for active event objects

Fixes: ecb01d53fdb413121ce2 ("v3dv: refactor events")
Fixes: dEQP-VK.api.command_buffers.execute_large_primary

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19458>
(cherry picked from commit 4c861cf22a63b892cec74bc132313dcd17d77584)

v3dv: do better cleanup on failure during pipeline cache operation

Fixes (with disk cache enabled):
dEQP-VK.api.device_init.create_instance_device_intentional_alloc_fail.basic
dEQP-VK.api.object_management.alloc_callback_fail.device
dEQP-VK.api.object_management.alloc_callback_fail.device_group

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19458>
(cherry picked from commit 5e97150e21a75920c3521717f70f9f92702729b4)

v3dv: handle allocation failure during pipeline initialization

Fixes (with disk cache disabled):
dEQP-VK.api.device_init.create_instance_device_intentional_alloc_fail.basic
dEQP-VK.api.object_management.alloc_callback_fail.device
dEQP-VK.api.object_management.alloc_callback_fail.device_group

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19458>
(cherry picked from commit 1f5966397aa208eb574394a4d0a592b66b776b44)

v3dv: fix incorrect return type

Fixes: ecb01d53fdb413121ce2 ("v3dv: refactor events")

Partially fixes:
dEQP-VK.api.device_init.create_instance_device_intentional_alloc_fail.basic
dEQP-VK.api.object_management.alloc_callback_fail.device
dEQP-VK.api.object_management.alloc_callback_fail.device_group

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19458>
(cherry picked from commit 7f905a8117e9dc85088fe23429594537ee240c1e)

v3dv: do a better job at cleaning up the device on init failure

These leaks on device creation failure have been there before, but
were only exposed as CTS failures after the recent event refactoring.

Partially fixes:
dEQP-VK.api.device_init.create_instance_device_intentional_alloc_fail.basic
dEQP-VK.api.object_management.alloc_callback_fail.device
dEQP-VK.api.object_management.alloc_callback_fail.device_group

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19458>
(cherry picked from commit b78fd50e90c700466213e4b3b40ce10547b1e14e)

v3dv: avoid freeing already-freed memory

Fixes: ecb01d53fdb413121ce2 ("v3dv: refactor events")
Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19458>
(cherry picked from commit aff368fe83bd4bcb4620a1a19e9b71eb67a05af7)

v3dv: remove unnecessary check for NULL

We are initializing the device, so we know this will be NULL.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19458>
(cherry picked from commit c793d384c12ebaa13487aeee35532e52b888c049)

vk/runtime: drop incorrect UNUSED annotation

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19458>
(cherry picked from commit c6c5949ff70a47c47795fe9161a7514173b5be24)

radv: fix fallback for extreme geometry with tessellation on GFX11

It would assert anyways. Found by inspection.

Cc: 22.3 mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19495>
(cherry picked from commit fab87b0f4115bd360ed9334aa7a72518613c2014)

anv: Reduce RHWO optimization (Wa_1508744258)

Implement Wa_1508744258:

   Disable RHWO by setting 0x7010[14] by default except during resolve
   pass.

Disable the RCC RHWO optimization at all times except when resolving
single sampled color surfaces.

v2: Move stalling to genX(cmd_buffer_apply_pipe_flushes) for clarity (Mark)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Mark Janes <markjanes@swizzler.org>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19450>
(cherry picked from commit ba0336ab3f69f958e4346c1465c2fe0859d4ac59)

radv: Make the compute scratch waves per SE as well.

Fixes: 278e533ec9c ("radv: update scratch buffer registers on GFX11")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19488>
(cherry picked from commit d876ddc920f1aaf48219f2060de6cc99c003bc5d)

radv: Fix compute scratch buffer emission.

Copied wrong from radeonsi. The registers following the scratch
buffer address are the shader rsrc1/rsrc2. Not the user SGPR0
containing the ring resource word 1.

Fixes: 278e533ec9c ("radv: update scratch buffer registers on GFX11")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19488>
(cherry picked from commit b8865ad04643cd7e8faa8a863d81fedbc1e29231)

iris: invalidate sysvals if grid dimension changes

Cc: mesa-stable
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18581>
(cherry picked from commit 9ccdd86b9040b265893712f4125d30552e0fe1cd)

glx: Fixes apple/apple_visual.c that include non-exist util/debug.h

Fixes: aa4ac5ff8bb ("utils: Merge util/debug.* into util/u_debug.* and remove util/debug.*")

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Illia Abernikhin <illia.abernikhin@globallogic.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19486>
(cherry picked from commit 287435134c6685a71f5c247778017c8b66a93627)

panfrost: Replace resource shadowing flush

The entire point of resource shadowing is to avoid unnecessary flushing.
Flushing readers after shadowing is counterproductive. A refresher on
how resource shadowing is supposed to work:

First, we determine if it's beneficial to shadow resources. If so, we
create a new backing buffer object. We flush the current writer of the
resource, if there is one, so the current contents become known to the
CPU. If we are not discarding the original resource, we then copy the
existing contents of the buffer to the new shadow buffer on the CPU.
Finally, we swap the resource's backing buffer for our shadow. Any batch
that reads the resource will continue to read the old copy of the
resource, and any future draw calls will see the new copy with the
change implemented.

Where did we go wrong?

In 988d5aae74d ("panfrost: Flush resources when shadowing"), we started
flushing all readers. We didn't actually need to flush, we just needed
to avoid dangling references on the batches reading the old copy of the
resource. But that's easily enough avoided: just remove the references.
The batches still hold a reference to the underlying BO, which will be
freed at the right time regardless.

Originally motivated by glmark2 -bbuffer:update-method=subdata, which
has some pathological access paterns.

Firefox is a lot faster anecdotally (now scrolling at 60fps in firefox).

But what actually motivated this is an apitrace from Duckstation's GLES
renderer. With this patch, the in-game portion is improved 3fps to 21fps.

Closes: #4028
Fixes: 988d5aae74d ("panfrost: Flush resources when shadowing")
Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19361>
(cherry picked from commit 2d8f28df731638e1270b0ae273c7bfd2b29b7993)

panfrost: Don't copy resources if replaced

If a synchronized transfer_map is going to overwrite an entire resource,
there's no need to memcpy in the original contents ahead-of-time. This
memcpy is particularly bad for large buffers where it's copying WC->WC,
although that could be mitigated with threaded_context's cpu_storage in
the future if needed.

Prevents a performance regression in glmark2's buffer scenes from the
next patch, hence the Cc.

Cc: mesa-stable
Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19361>
(cherry picked from commit 0b26a9f773956fc00a77b0d4a7aafee5795ce935)

zink: add missing build_id linker args

Fixes: 4e14da056d623ababf7c ("zink: Enable mesa/st frontend shader caching.")
Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19485>
(cherry picked from commit dfa8600a2d39aabc88e5a5b87ab86b2b0ea30191)

.pick_status.json: Update to 1174f376096ed6ceebb0fb2810456f1501a68df7

VERSION: bump to 22.3.0-rc1

Revert "etnaviv: completely turn off MSAA"

This reverts commit 044b238507bb54ff8b6129c9754f1efc0f0834b9 and extends it with
 - putting the comments directly in front of the if's
 - do not support 2x MSAA on SMALL_MSAA hardware
 - checking if blt/rs supports the format

MSAA should work as expected now. Tested with kmscube and qt5.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19013>

etnaviv: blt: Directly call etna_try_blt_blit(..)

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19013>

etnaviv: blt: Add color resolve support

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19013>

etnaviv: Update headers from rnndb

Update to rnndb commit fff078b61ceb.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19013>

etnaviv: rs: msaa: Set cache mode for SMALL_MSAA

If the GPU supports SMALL_MSAA and does not support CACHE128B256BPERLINE we
need to set the cache mode to CACHE_MODE_256B.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19013>

etnaviv: rs: Directly call etna_try_rs_blit(..)

Also remove some old comments.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19013>

etnaviv: rs: Color resolve works

No special treatment is needed and is helps MSAA.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19013>

etnaviv: Create MSAA surfaces with PE compatible tiling

MSAA surfaces are always used for rendering, and only as blit sources,
so the need to be allocated with PE compatible tiling.

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19013>

etnaviv: Remove possibility to force MSAA

st/mesa does not know anything about the forced MSAA and we end with
the following assert:
etna_try_rs_blit: Assertion `(blit_info->src.box.x + blit_info->src.box.width) * msaa_xscale <= src_lev->padded_width' failed

Let the application do its thing regarding MSAA and remove this 'debug'
feature.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19013>

d3d12: Hook up driver-private interop data

Reviewed-by: Adam Jackson <ajax@redhat.com>
Acked-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19242>

mesa/st: Query driver for interop data

Reviewed-by: Adam Jackson <ajax@redhat.com>
Acked-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19242>

gallium: Add screen methods for driver-private data on interop interface

Reviewed-by: Adam Jackson <ajax@redhat.com>
Acked-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19242>

include: Add driver data to v2 of mesa_glinterop_device_info

Reviewed-by: Adam Jackson <ajax@redhat.com>
Acked-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19242>

egl/wgl: Implement interop methods

Reviewed-by: Adam Jackson <ajax@redhat.com>
Acked-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19242>

wgl: Implement interop methods

Reviewed-by: Adam Jackson <ajax@redhat.com>
Acked-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19242>

egl/dri2: Implement the new flush method

Reviewed-by: Adam Jackson <ajax@redhat.com>
Acked-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19242>

glx: Implement the new flush method

Reviewed-by: Adam Jackson <ajax@redhat.com>
Acked-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19242>

dri: Implement the new flush method

Reviewed-by: Adam Jackson <ajax@redhat.com>
Acked-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19242>

mesa/st: Implement the new flush method

Reviewed-by: Adam Jackson <ajax@redhat.com>
Acked-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19242>

mesa/st: Split interop object lookup from export

Reviewed-by: Adam Jackson <ajax@redhat.com>
Acked-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19242>

mesa: Make fence_sync not static

Reviewed-by: Adam Jackson <ajax@redhat.com>
Acked-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19242>

include: Add a flush interop method

Reviewed-by: Adam Jackson <ajax@redhat.com>
Acked-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19242>

mesa, dri2: Move interop helpers from dri2 to state tracker

They're not at all dri2 specific and we'll want to use them from wgl.

Reviewed-by: Adam Jackson <ajax@redhat.com>
Acked-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19242>

include: Add WGL version of GL interop extensions

Reviewed-by: Adam Jackson <ajax@redhat.com>
Acked-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19242>

docs/new_features: Add Panfrost disk cache

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Acked-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19474>

radeonsi: increase gfx1100/gfx1101 physical vgprs

https://reviews.llvm.org/D134522

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18825>

aco/gfx11: increase gfx1100/gfx1101 physical vgprs

https://reviews.llvm.org/D134522

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18825>

ac/gpu_info: increase gfx1100/gfx1101 physical vgprs

https://reviews.llvm.org/D134522

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18825>

radv: increase gfx1100/gfx1101 physical vgprs

https://reviews.llvm.org/D134522

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18825>

panfrost: Implement a disk cache

Wire up the Mesa shader disk cache into Panfrost. Coupled with the
precompiles from the previous patch, this should greatly reduce shader
recompile jank.

This is a bare bones implementation. Obvious future work includes:

- Caching internal (outside of Gallium) shaders
- Implement finalize_nir to reduce on disk size of shaders

That doesn't need to come in this patch.

This patch does shuffle some allocation patterns around to avoid extra
nir_shader_clones, but the result should be pretty clean.

---

Consider dEQP-GLES31.functional.ssbo.layout.basic_unsized_array.* in the CTS.
With a cold cache:

   44.11user 0.66system 0:45.44elapsed 98%CPU (0avgtext+0avgdata 267804maxresident)
   k 0inputs+0outputs (130major+74725minor)pagefaults 0swaps

But with this commit and a warm cache:

   4.07user 0.35system 0:04.56elapsed 96%CPU (0avgtext+0avgdata 211012maxresident)
   k0inputs+0outputs (1major+49489minor)pagefaults 0swaps

That's an 11x improvement!

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19363>

panfrost: Precompile shaders

We have no vertex shader key, and unless legacy GL features are used, the
fragment shader key is known ahead-of-time. That means we can precompile shaders
at CSO create time, hopefully avoiding some draw-time jank.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19363>

panfrost: Precompile transform feedback program

This avoids the weird compiled_shader pointer inside of compiled_shader. Because
we don't have a nonempty vertex shader key, there will only ever be a single
transform feedback program per CSO.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19363>

panfrost: Make fixed_varying_mask a fragment-only key

This makes it clear that there are no VS variants.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19363>

panfrost: Use u_dynarray for variants

No need to open code our own "special" dynarray. Unify the graphics/compute CSO
creation to make this work without duplicating more code.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19363>

panfrost: Remove uncompiled_shader->active_variant

The active compiled shader (variant) is context state, it is inappropriate to
stash it on the uncompiled shader. Add compiled shader pointers to the context
and get rid of the active_variant mutation. Names from iris.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19363>

panfrost: Rename structs to panfrost_(un)compiled_shader

Consistency with other drivers, this makes the language less variant-centric.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19363>

panfrost: Remove unused req_input_mem copy

Cloverism.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19363>

panfrost: Merge pan_assemble.c into pan_shader.c

We now have a common place for the driver side of shader compilation. As a bonus
this gets rid of the old "assemble" name which hasn't been accurate since 2018
or so.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19363>

panfrost: Consolidate all shader compiling code

Compute and graphics shaders will need similar paths for the disk cache. Let's
consolidate the code to make it easier to work with.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19363>

panfrost: Remove bogus assert

Nothing enforces this except perhaps the implicit structure of shader keys.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19363>

panfrost: Move small compute functions to pan_context.c

So we can use pan_compute.c for just programs.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19363>

panfrost: Delete set_global_resources

Cloverism.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19363>

panfrost: Don't use nir_variable to link varyings

NIR deemphasizes nir_variable. We want to transition off it. Instead of walking
the list of variables and playing games with the GLSL types to collect varying
information, walk the list of instructions and use the I/O semantics to collect
similar information.

In addition to avoiding the reliance on nir_variable, this fixes handling of
struct varyings under certain circumstances. Such programs are compiled by the
GLES3.1 CTS but not used, so without this fix, the affected tests would regress
when precompiling.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19363>

panfrost: Don't allocate space for empty varyings

PIPE_FORMAT_NONE has a block size of 1, oddly, but we don't actually
need to allocate any space for it. This acts as a small optimization for
a few shaders with the new varying linker.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19363>

pan/mdg: Use bifrost_nir_lower_store_component

Move the pass from the Bifrost compiler to the Midgard/Bifrost common code
directory, and take advantage of it on Midgard, where it fixes the same
tests as it fixed originally on Bifrost.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19363>

pan/mdg: Use .u32 for flat shading

This is simple and matches what we do on Bifrost.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19363>

pan/mdg: Don't pair ST_VARY.a32 with other instrs

For some reason, LD_ATTR/ST_VARY.a32 bundles raise INSTR_INVALID_ENC, at
least on Mali-T860. Don't construct such pairs. This is a blunt hack but
I don't know where this curveball requirement is coming from and this
unblocks the rest of this series.

total instructions in shared programs: 99879 -> 99788 (-0.09%)
instructions in affected programs: 3179 -> 3088 (-2.86%)
helped: 49
HURT: 9
helped stats (abs) min: 1.0 max: 6.0 x̄: 2.04 x̃: 2
helped stats (rel) min: 0.93% max: 10.53% x̄: 5.46% x̃: 4.88%
HURT stats (abs)   min: 1.0 max: 1.0 x̄: 1.00 x̃: 1
HURT stats (rel)   min: 0.61% max: 2.13% x̄: 1.41% x̃: 1.14%
95% mean confidence interval for instructions value: -1.93 -1.20
95% mean confidence interval for instructions %-change: -5.37% -3.41%
Instructions are helped.

total bundles in shared programs: 43778 -> 45102 (3.02%)
bundles in affected programs: 10737 -> 12061 (12.33%)
helped: 10
HURT: 369
helped stats (abs) min: 1.0 max: 3.0 x̄: 1.50 x̃: 1
helped stats (rel) min: 2.90% max: 18.75% x̄: 6.93% x̃: 5.21%
HURT stats (abs)   min: 1.0 max: 10.0 x̄: 3.63 x̃: 4
HURT stats (rel)   min: 0.82% max: 44.44% x̄: 15.27% x̃: 13.33%
95% mean confidence interval for bundles value: 3.29 3.69
95% mean confidence interval for bundles %-change: 13.68% 15.69%
Bundles are HURT.

total quadwords in shared programs: 76783 -> 77914 (1.47%)
quadwords in affected programs: 18633 -> 19764 (6.07%)
helped: 9
HURT: 370
helped stats (abs) min: 1.0 max: 2.0 x̄: 1.22 x̃: 1
helped stats (rel) min: 0.87% max: 8.33% x̄: 3.71% x̃: 3.85%
HURT stats (abs)   min: 1.0 max: 7.0 x̄: 3.09 x̃: 3
HURT stats (rel)   min: 0.82% max: 35.00% x̄: 7.82% x̃: 6.11%
95% mean confidence interval for quadwords value: 2.82 3.15
95% mean confidence interval for quadwords %-change: 7.02% 8.06%
Quadwords are HURT.

total registers in shared programs: 7266 -> 7076 (-2.61%)
registers in affected programs: 1224 -> 1034 (-15.52%)
helped: 171
HURT: 25
helped stats (abs) min: 1.0 max: 3.0 x̄: 1.27 x̃: 1
helped stats (rel) min: 8.33% max: 50.00% x̄: 21.85% x̃: 20.00%
HURT stats (abs)   min: 1.0 max: 2.0 x̄: 1.12 x̃: 1
HURT stats (rel)   min: 10.00% max: 100.00% x̄: 35.73% x̃: 33.33%
95% mean confidence interval for registers value: -1.10 -0.84
95% mean confidence interval for registers %-change: -17.69% -11.32%
Registers are helped.

total threads in shared programs: 4956 -> 5019 (1.27%)
threads in affected programs: 99 -> 162 (63.64%)
helped: 43
HURT: 6
helped stats (abs) min: 1.0 max: 2.0 x̄: 1.74 x̃: 2
helped stats (rel) min: 100.00% max: 100.00% x̄: 100.00% x̃: 100.00%
HURT stats (abs)   min: 2.0 max: 2.0 x̄: 2.00 x̃: 2
HURT stats (rel)   min: 50.00% max: 50.00% x̄: 50.00% x̃: 50.00%
95% mean confidence interval for threads value: 0.91 1.66
95% mean confidence interval for threads %-change: 67.36% 95.90%
Threads are helped.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19363>

pan/mdg: Disassemble the .a32 bit

Corresponds to .auto32 on Bifrost. This is helpful for a conformant
implementation of flat shading.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19363>

freedreno/a6xx: Mark gl45 supported

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19400>

freedreno/a6xx: ARB_query_buffer_object support

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19400>

freedreno: Core ARB_query_buffer_object support

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19400>

freedreno/batch: Add a global epilogue

Rename the existing one to make it clear that it is per-tile, and add a
new one that runs after all the tile passes.  Will be needed in the next
commit.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19400>

freedreno/a5xx+a6xx: Add base class for query samples

For PIPE_CAP_QUERY_BUFFER_OBJECT we'll need to write on the GPU a flag
when the query result is available, which means the buffers used for
query results should have a header with availability flag.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19400>

freedreno/a6xx: Remove unused field

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19400>

freedreno/a6xx: Fix occlusion queries

WFI is not a strong enough barrier, which shows up in piglit qbo tests
which do a single draw.

Fixes: 13fc03f4c0e ("freedreno/a6xx: Avoid stalling for occlusion queries")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19400>

freedreno/a6xx: Enable ARB_shader_group_vote

Already supported for at least a6xx.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19400>

freedreno/ir3: Unconditionally lower subgroup ops

For devices that don't support getfiberid, we force the subgroup size
to 1 for things other than compute stage.  This matches what zink does.
And fixes spec@arb_shader_group_vote@vs-eq-uniform once we expose
ARB_shader_group_vote.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19400>

freedreno/a3xx+: Enable ARB_derivative_control

Also already supported by ir3.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19400>

freedreno/a3xx+: Enable ARB_shader_texture_image_samples

This is already supported for ir3

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19400>

freedreno/a6xx: Implement ARB_clear_texture

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19400>

freedreno/a6xx: Use box to pass 2d clear params

Simplifies the interface slightly and makes it possible to re-use the
path for pctx->clear_texture() in the next commit.  The z dimensions
still come from the surface.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19400>

freedreno: Add ARB_gl_spirv support

All the heavy lifting is done in nir.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19400>

docs: update new_features.txt for non-extension additions to RADV

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19469>

zink: put union fields into structs named by the shader-stages

This makes it easier to see that a field is only valid in a given stage,
to avoid undefined behavior.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19457>

zink: do not read is_generated unless in tcs shader

It's undefined behavior in C to read a union member if another member
has been written to more recently. Let's be more careful here!

Fixes: a9d2b86c2c8 ("zink: store the spirv_shader to the zink_shader struct for generated tcs")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19457>

zink: consider polygon-mode for rast_prim

But because polygon-offset needs to consider the primitive-type *before*
overriding the type, add a zink_prim_type()-helper for the partially
resolved state.

Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19438>

zink: only set line-width if drawing lines

This might seem like a premature optimization, but it's going to make a
bit more sense with the next commit, to prevent needlessly regressing
performance.

Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19438>

zink: correct depth-bias enable condition

This should be based on the fill_mode, not on the primitive type. We
*also* need to check if we'll rasterize triangles in the end, though.

Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19438>

rusticl: meson devenv support

This gets 'meson devenv -C build clinfo' working on iris for me.

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19451>

radv/llvm: fix GS shaders on GFX8/9

6698753cdb6 switched our GS output stores to use MUBUF.

The stride doesn't matter for the ESGS descriptor (because idxen=false and
the index stride is 64), but this fixes it anyway.

This also changes ACO to use MUBUF store too, since MTBUF doesn't seem to
work correctly with an invalid data format in the descriptor.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Fixes: 6698753cdb6 ("ac/llvm: don't use tbuffer_store as a fallback for swizzled stores")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18885>

nv50/ir/nir: ignore sampler for TXF/TXQ ops.

Recently, a regression was reported where videos in Firefox had shifted/
glitched colors on certain Kepler hardware. This was bisected to
bf02bffe156214dad942f56ee68c380d1968704f, however, the issue already
existed but didn't hit users until TGSI was switched to NIR as default.

The issue was traced to a YUV-to-RGB fragment shader used by Firefox,
which uses three samplers for the Y/U/V components. The Y component was
handled correctly, but the U/V components were bogus, causing the issue.

After analysis, it appears the TXF/TXQ ops. should only handle the texture
(r) but not the sampler (s), see 63b850403c90f33c295d3ad6be4ad749d4ea6274
and 346ce0b98832e33d5411200002571b3edea9e2bb.
Similarly, handleTXQ/handleTXF on nv50_ir_from_tgsi always sets s=0.
Only Kepler was affected because other hardware ignores s at codegen.

Always set s=0 on NIR for TXF/TXQ, to keep TGSI behavior and fix the
regression.

Thanks: Karol Herbst and M Henning for help diagnosing the issue.
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7416
Cc: mesa-stable
Suggested-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: M Henning <drawoc@darkrefraction.com>
Signed-off-by: Joan Bruguera <joanbrugueram@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19453>

android: -Ddri-drivers= is not needed anymore

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19462>

docs: There is no more swrast driver

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19459>

docs: There is only glx=xlib option now

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19459>

docs:  -Ddri-drivers= option is deprecated, do not use it anymore

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19459>

docs: fixup rebase mistake

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19296>

radeonsi: update db_eqaa even if msaa is disabled

This seems to fix rendering in application toggling MSAA on and
off between draw calls.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7537
Cc: mesa-stable
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19326>

radeonsi/gfx11: enable sdma copy DRI_PRIME

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19326>