gallium: add the u_vbuf pointer into pipe_context

This will allow removing the draw_vbo wrapping in cso_context.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20025>

asahi: Implement occlusion queries

While the hardware supports both counter and boolean occlusion queries, the
programming model is quite different from OpenGL. In AGX (and in Metal),
there is a single "visibility result buffer" associated with the render pass.
Each draw that uses occlusion queries writes into this render pass global
visibility result buffer at a particular index. By contrast, the OpenGL
occlusion query model supposes that each query has independent state that can be
mixed and matched within a render pass. We can't simply allocate backing memory
for a query and write to it from a job. We can't allocate visibility result
buffers for each batch up front and statically assign OpenGL queries to indices,
because the OpenGL query can span multiple batches. Finally we can't use a
global visibility result buffer without introducing additional synchronization,
given that we now support multiple batches in-flight at once.

In this patch, I've elected to use a simple solution: allocate visibility result
buffers and indices on the fly as needed, and accumulate the results on the CPU
at the end of the render pass. When we have proper synchronization we'll want to
revisit this, but as everything is stalling at submit time now, I'm not inclined
to "optimize" something I can't test.

Passes dEQP-GLES3.functional.occlusion_query.* and the relevant piglit tests.
The piglits are considerably more thorough, checking lots of "we hate tilers"
conditions that dEQP skips over.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Move query functions to agx_query.c

New file. They're just stubs now but will get nontrivial in a moment.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Don't upload samplers for clears/stores

Unlikely to help but makes the traces neater.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Avoid reloads with staging blits

Noticed by inspection. Not likely to matter unless these staging blits are in a
hot path, but it's an easy win.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Don't reload uninitialized surfaces

Pointless. This should save some bandwidth in some cases (possibly mipmap
generation?)

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Fix Z32S8 harder

Fixes dEQP-GLES3.functional.texture.format.sized.2d.depth32f_stencil8_pot after
stencil texturing broke it.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Don't shadow idle resources

Pointless allocation+memcpy.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Model alignment of occlusion query indices

8-byte offsets.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Identify some more fields used with layered

These values depend on the framebuffer width/height and maybe other
stuff. Maybe strides.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Identify XML for anisotropic filtering

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Check-box implement rasterizer discard

Passes dEQP-GLES3.functional.rasterizer_discard.*

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Wire in 1D (array) textures

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Dirty track VS/FS key updates

drawoverhead 1 score doubled to 7668.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Separate VS/FS shader keys

First remove agx_shader_key from asahi_shader_key. It's trivial. agx_shader_key
is going to go away soon now that we lower everything in NIR.

Then everything else is mutually exclusive between stages. That means much
less to hash. drawoverhead test 1 from 2331 to 3443.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Implement invalidate_resource

From Panfrost. This lets us avoid storing depth/stencil attachments at the end
of the frame in GLES.

On my 4K monitor, glmark2 -btexture at fullscreen goes from 705fps to 1150fps. I
assume gains on real workloads will be smaller.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Track batch masks on ZS/blend CSO

Adapted from panfrost, with the work happening at CSO create time instead of
draw time allowing us to do more sophisticated analysis. We'll use these for
accurate masks in a moment.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Dirty track blend state

We'll want this to reduce variant lookups eventually.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Enable dirty tracking

Whoops. drawoverhead test 1 score from 496 -> 2377.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

ail: Save level_offsets_compressed_B

So we can bind specific mip levels for rendering into compressed Z/S.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

panfrost: implement clear_depth_stencil

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20238>

nine: enable on panfrost

Also, enable required kmsro dependencies.

Tested-by: Aleksey Komarov <q4arus@ya.ru>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20238>

panfrost: implement clear_render_target

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Co-authored-by: Aleksey Komarov <q4arus@ya.ru>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Aleksey Komarov <q4arus@ya.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20238>

panfrost: Handle resources without depth in batch_to_fb_info

Prevent preloading data from resources which doesn't exist.

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Aleksey Komarov <q4arus@ya.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20238>

panfrost: Implement GL_EXT_clip_control

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Co-authored-by: Aleksey Komarov <q4arus@ya.ru>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Aleksey Komarov <q4arus@ya.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20238>

intel: add devinfo->has_64bit_float_via_math_pipe

Unusual hardware features that require special hanlding usually get a
devinfo field, so do this for MTL's unordered DF types. This will
guarantee that any platform based on MTL (thus inheriting from
MTL_FEATURES) will automatically be handled in these special cases.

v2: s/has_unordered_64bit_float/has_64bit_float_via_math_pipe/ (Curro).

Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20072>

intel/compiler: fix intel_swsb_decode for newer platforms

In the previous patch we adjusted the scoreboard pass to take into
consideration a new case of unordered operations for TGL. Fix the
decoding as well.

v2: use intel_device_info_is_mtl()  (Curro, Jordan)
v3: the part where we export num_sources_from_inst() is now a separate patch
    (Curro).
v4: Work around false positive maybe-unitialized warning since Marge
    uses -Werror=maybe-uninitialized (Marge).

Reviewed-by: Francisco Jerez <currojerez@riseup.net> (v3)
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20072>

intel/compiler: export brw_num_sources_from_inst

We want to call this from brw_disasm.c, so move it out to brw_eu.c
since it's about to become more of a shared utility function than
something specific to the EU validator.

Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20072>

intel/compiler: avoid 64bit SEL_EXEC on MTL

On MTL, instructions with DF type are unordered, executed in the math
pipe. This means that they require different SWSB dependency handling,
and also that in some cases such as MOVs it's generally faster to
simply use 2 smaller ordered moves than a single unordered MOV.

One problem we have with the current code is that generate_code() is
not setting the proper SWSB dependencies for the generated DF MOVs,
causing some tests to fail.

One solution would be to fix generate_code() by making it set the
appropriate dependencies. This was the first patch I wrote. Another
solution to this problem, pointed to us by Curro, is to change
required_exec_type() so we use UD instructions instead of DF, just
like we do with platforms that don't have 64 bit instructions, which
means there won't be anything to fix in generate_code(). The second
solution is what this patch implements.

This fixes at least:
 - dEQP-VK.subgroups.arithmetic.framebuffer.subgroupmin_double_vertex

Thanks to Francisco Jerez for all the major help provided with this
problem.

Credits-to: Francisco Jerez <currojerez@riseup.net>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20072>

intel/compiler: avoid (RegDist, SBID) on DF instructions on MTL

When we use this form there's no way to specify which pipe RegDist
refers to, so there are a few rules to figure this out, which is what
inferred_sync_pipe() implements. But for MTL there's no long pipe and
the documentation does not explicitly explain what should be the
inferred type for its long (DF) instructions - which are out-of-order,
by the way.  One way to interpret this is that such case should be
avoided.  So add the extra check to entirely avoid this case.

Notice that this is not actually fixing any bug, since returning
TGL_PIPE_LONG (what we do today) will actually make these DF
instructions incompatible with every in-order instruction, so we'll
never opt to use the (RegDist, SBID) form anyway. But still, it's
better to have this case explicitly documented instead of having it
covered by a semi coincidence.

v2: use intel_device_info_is_mtl()  (Curro, Jordan)

Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20072>

intel/compiler: on MTL, DF instructions run in the math pipe

Adjust the scoreboard code to take that into account.

Fixes at least:
  - dEQP-VK.glsl.builtin.precision_double.refract.compute.vec3
  - dEQP-VK.glsl.builtin.precision_double.matrixcompmult.compute.mat4

v2: use intel_device_info_is_mtl()  (Curro, Jordan)

Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20072>

intel/fs: Make the result of is_unordered() dependent on devinfo.

Reviewed-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20072>

Revert "anv: compile anv_acceleration_structure.c"

This reverts commit 74d0be27ae9eb666df948874a20a3b4464df7db1.

Also remove anv_acceleration_structure.c, it was meant to be removed
earlier. There was probably a rebase issue somewhere.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20248>

freedreno/a6xx: fix blend all_mrt_write_mask

Fix all_mrt_write_mask when independent_blend_enable is false.
Otherwise, lrz write is always diabled with MRT when
independent_blend_enable is false.  This fixes a 2% perf regression for
multiple gfxbench benchmarks.

Fixes: 0132c22de75 ("freedreno/a6xx: Don't disable LRZ for invalid channels")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20254>

iris: Improve direct CPU map heuristics

We were promoting reads with a valid primary to direct CPU maps even
if the mmap mode was IRIS_MMAP_WC, which would mean uncached reads from
VRAM.  In that case, GPU blits are in fact useful!

We were also only checking for !DISCARD_RANGE rather than MAP_READ,
which isn't a great idea for image maps, given the discussion in the
previous commit about image map semantics.

The original code was also just confusingly structured.  Make a helper
function with clearly defined cases where we want to bail on CPU maps.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19209>

iris: Only copy existing data into staging images with PIPE_MAP_READ

When performing transfer maps on images that require staging buffers
(say, for presenting a linear view of tiled memory), we were reading
the existing contents of the buffer into the staging resource on map
unless PIPE_MAP_DISCARD_RANGE was set.

The thinking was to support partial writes.  If you map a subrectangle
of an image, but then only write selective pixels - should it preserve
the existing contents of the mapped region?  I believed that it should,
unless you pass PIPE_MAP_DISCARD_RANGE to explicitly say that that it's
okay to invalidate the destination region.

However, that does not appear to be the interpretation favored by other
Mesa developers (in particular Michel Dänzer and Marek Olšák).  The
radeonsi driver does not do this readback from the destination region
to the staging buffer unless you pass PIPE_MAP_READ.  If you want to
do a partial write and preserve contents, you need to pass both flags:
(PIPE_MAP_READ | PIPE_MAP_WRITE).  Passing READ is expected to come
with an associated cost.

OpenGL defines GL_MAP_INVALIDATE_RANGE_BIT for mapping buffer objects,
which is translated to PIPE_MAP_DISCARD_RANGE.  However, unextended
OpenGL doesn't define mapping textures.  There are two main sources of
image maps: our internal MapTextureImage() hook, and gbm_bo_map().

I've audited our internal MapTextureImage() calls, and while some do
pass PIPE_MAP_DISCARD_RANGE, almost all of them wholly overwrite the
mapped region, and those that care about combining with existing image
contents all pass PIPE_MAP_READ.  So this should work there.

GBM defines three flags: GBM_BO_TRANSFER_READ, WRITE, and READ_WRITE.
There is no defined "invalidate range" bit.  In issue #6020, Matthias
Treydte notes that this extra readback can cause performance problems,
and with iris's current interpretation, there's no way to avoid it.
During that discussion, Michel and Matthias both argued that
GBM_BO_TRANSFER_WRITE should invalidate the destination contents and
avoid the readback, while GBM_BO_TRANSFER_READ_WRITE would preserve it.

This patch makes iris follow that model for image mappings, removing
readback on staging maps for both detiling and stall avoidance, unless
PIPE_MAP_READ is passed.  I believe we can change this with impunity.

For buffer objects, Ian Romanick and I both agree that partial writes
should be supported, and GL_MAP_INVALIDATE_RANGE_BIT exists precisely
to indicate that you should spend effort preserving existing contents.
So we continue doing readback for buffers unless PIPE_MAP_DISCARD_RANGE
is flagged, for now.  While I think this is work, it also seems to be
undertested in the CTS and Piglit.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6020
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19209>

iris: Return idle status from iris_invalidate_buffer, skip busy checks

If we successfully replace the backing storage for a buffer, we know
that it's idle, and the transfer map code can mark it unsynchronized
right away, letting us skip redundant resource_is_busy() checks.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19209>

iris: Don't replace backing storage for exported buffers.

We already gave out the old BO...or acquired it from somewhere which
may be affecting it.  We simply can't replace the backing store.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19209>

iris: Promote DISCARD_RANGE to DISCARD_WHOLE_RESOURCE where possible

This allows us to replace the backing storage for a buffer, which
means we'd have an idle buffer and thus could do an unsynchronized
mapping where we otherwise wouldn't.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19209>

iris: Use persistent mappings for pinned memory (userptr)

This is a port of Nicolai's b52721e3b693e113aa537d163c8a855169e7b75d
from radeonsi.  Because GL_AMD_pinned_memory guarantees that mappings
will refer to the same underlying page, we need to avoid using staging
maps.  Using a persistent map is a reasonable way to accomplish this.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19209>

iris: Delete map->dest_had_defined_contents

Dead since commit 6cc09699cd77286f7601a1afab60dc9fa0bafd48.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19209>

nir/gather_info: use nir_ssa_scalar_resolved

This lets us skip copies.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19597>

vtn: add mesh output and task_payload to vtn_mode_is_cross_invocation

This fixes a potential race condition, and removes output loads (which
should not exist in the EXT_mesh_shader).

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7391
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19597>

nir: add task_payload and shader_out to nir_var_vec_indexable_modes

Since these can be cross-invocation, we need this to write individual
components without race conditions or loads.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Gitlab: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7391
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19597>

radv: fix task payload lowering when shared_memory_explicit_layout=true

If shared_memory_explicit_layout=true, we would have skipped lowering task
payload variables to explicit types.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19597>

radv: fix mesh shaders with null winsys

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19597>

panvk: Delete panvk_CmdSetDeviceMask, panvk_GetDeviceGroupPeerMemoryFeatures

Delete panvk_CmdSetDeviceMask and panvk_GetDeviceGroupPeerMemoryFeatures
so that the vk_common_* version will be used instead. This will avoid
repeated code.

Signed-off-by: Rebecca Mckeever <rebecca.mckeever@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20218>

hasvk: Delete VK_KHR_device_group provided entrypoints

Delete anv_CmdDispatch, anv_CmdSetDeviceMask, and
anv_GetDeviceGroupPeerMemoryFeatures so that the vk_common_*
versions will be used instead. This will avoid repeated code.

Signed-off-by: Rebecca Mckeever <rebecca.mckeever@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20218>

anv: Delete VK_KHR_device_group provided entrypoints

Delete anv_CmdDispatch, anv_CmdSetDeviceMask, and
anv_GetDeviceGroupPeerMemoryFeatures so that the vk_common_*
versions will be used instead. This will avoid repeated code.

Signed-off-by: Rebecca Mckeever <rebecca.mckeever@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20218>

tu: Delete VK_KHR_device_group provided entrypoints

Delete tu_CmdDispatch, tu_CmdSetDeviceMask, and
tu_GetDeviceGroupPeerMemoryFeatures so that the vk_common_*
versions will be used instead. This will avoid repeated code.

Signed-off-by: Rebecca Mckeever <rebecca.mckeever@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20218>

v3dv: Delete VK_KHR_device_group provided entrypoints

Delete v3dv_CmdDispatch, v3dv_CmdSetDeviceMask, and
v3dv_GetDeviceGroupPeerMemoryFeatures so that the vk_common_*
versions will be used instead. This will avoid repeated code.

Signed-off-by: Rebecca Mckeever <rebecca.mckeever@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20218>

radv: Delete VK_KHR_device_group provided entrypoints

Delete radv_CmdDispatch, radv_CmdSetDeviceMask, and
radv_GetDeviceGroupPeerMemoryFeatures so that the vk_common_*
versions will be used instead. This will avoid repeated code.

Signed-off-by: Rebecca Mckeever <rebecca.mckeever@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20218>

vulkan/runtime: Add VK_KHR_device_group provided entrypoints

Add entrypoints vk_common_CmdDispatch, vk_common_CmdSetDeviceMask,
and vk_common_GetDeviceGroupPeerMemoryFeatures in Mesa Vulkan runtime so
that they are available to all drivers.

Signed-off-by: Rebecca Mckeever <rebecca.mckeever@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20218>

gallium: add new variable for video frame statistics

Video encoder previously reuses the associated_data variable to output
encoding statistics, but it ended up breaking when transcoding. This
commit adds a new variable just for statistics.

Fixes: 2d1bd619dfd ("frontends/va: add ability for encoder to output statistics")
Signed-off-by: Thong Thai <thong.thai@amd.com>
Reviewed-by: Boyuan Zhang <Boyuan.Zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20190>

radv: remove some unnecessary 64-bit IO handling

nir_lower_io() lowers these to 32-bit.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20137>

radv: fix streamout with different streams in the same varying slot

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20137>

aco/ra: don't swap create_vector operand with definition blocker for SGPRs

There is no SGPR swap instruction, we always need 3 XORs.

fossil-db (navi21):
Totals from 76 (0.06% of 135636) affected shaders:
Instrs: 58400 -> 58347 (-0.09%); split: -0.10%, +0.01%
CodeSize: 312580 -> 312368 (-0.07%); split: -0.08%, +0.01%
Latency: 843333 -> 843180 (-0.02%); split: -0.02%, +0.00%
InvThroughput: 126431 -> 126412 (-0.02%)
Copies: 4008 -> 3955 (-1.32%); split: -1.47%, +0.15%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20240>

aco/ra: don't swap p_create_vector operand with definition blocker for scc

SCC is 1-bit, and we can't copy a 32-bit value into it.

Fixes dEQP-VK.spirv_assembly.type.scalar.i32.iequal_tesse with
ACO_DEBUG=noopt.

No fossil-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: 9476986e6f6 ("aco/ra: special-case get_reg_for_create_vector_copy()")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20240>

nir/opt_algebraic: Optimize open coded bfm.

Foz-DB Navi21:
Totals from 1553 (1.15% of 134913) affected shaders:
SpillVGPRs: 2246 -> 2223 (-1.02%); split: -1.42%, +0.40%
CodeSize: 10409156 -> 10410720 (+0.02%); split: -0.03%, +0.04%
Instrs: 1899725 -> 1898773 (-0.05%); split: -0.07%, +0.02%
Latency: 71225814 -> 71118314 (-0.15%); split: -0.21%, +0.06%
InvThroughput: 13384926 -> 13330369 (-0.41%); split: -0.47%, +0.06%
VClause: 38309 -> 38284 (-0.07%); split: -0.17%, +0.11%
SClause: 70743 -> 70706 (-0.05%)
Copies: 167296 -> 167230 (-0.04%); split: -0.28%, +0.24%
Branches: 42446 -> 42444 (-0.00%); split: -0.01%, +0.00%
PreVGPRs: 95191 -> 95188 (-0.00%)

Some minor instructions count regressions in parallel-rdp
because v_bfm_b32 can't use SDWA, but overall an improvement.

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18887>

frontends/va: fix gst videotestsrc h264 enc fail issue.

problem:
when doing "gst-launch-1.0 -v videotestsrc num-buffer=10 !
   vaapih264enc ! fakeink"

The command will fail due to gst will fetch the first
available supported format in the list, it becomes P010_LE
due to the commit in

[0b02db3007]
frontends/va: fixed av1 decoding 10bit ffmpeg output YUV issue

fix:
move the P010_LE code block to the end of the function, the sequence
of the supported formats restored to its original.

cc: mesa-stable

Reviewed-by: Boyuan Zhang <Boyuan.Zhang@amd.com>
Signed-off-by: Ruijing Dong <ruijing.dong@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20242>

frontend/osmesa: inherit pipe_frontend_drawable instead of allocating separately

This is required by st/mesa now.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

frontend/hgl: inherit pipe_frontend_drawable instead of allocating separately

This is required by st/mesa now.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

gallium: rename st_framebuffer_iface -> pipe_frontend_drawable, etc.

Also rename:
    iface -> drawable
    stfb -> drawable (where it means dri_drawable and not st_framebuffer)
    stfbi -> drawable or pdrawable (if drawable exists)

pipe_frontend_drawable* is really just dri_drawable* for DRI, and WGL/GLX
have their own variants. This makes it easier to understand what kind of
object is being used.

I always wondered what st_framebuffer_iface, iface, stfbi, iface_stamp,
and iface_ID actually mean. Now those terms are gone forever.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

gallium: remove pipe_frontend_screen::destroy callback, call it directly

This is the only one implemented by mesa/state_tracker.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

gallium: clean up comments in api.h, cosmetic changes

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

gallium: remove st_framebuffer_iface::st_manager_private

It's always equal to self.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

frontend/glx: inherit st_framebuffer_iface instead of allocating separately

All other frontends do it like this. This enables the next cleanup.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

gallium: remove unused ST_CONTEXT_ERROR_*

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

gallium: rename st_manager_private -> st_screen

Oh you didn't know that st_context had a screen object too?
Yeah, this code is full of surprises.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

gallium: remove st_profile_type in favor of gl_api, move out st_context_attribs

st_context_attribs is moved into st_context.h.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

gallium: remove unnecessary ST_PROFILE_*_MASK definitions

This didn't do anything.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

gallium: use PIPE_CONTEXT_* flags instead of ST_CONTEXT_FLAG_*

where PIPE_CONTEXT_* flags are available

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

gallium: move st function declarations from api.h into st_context.h

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

gallium: remove enum st_texture_type

just use GLenum

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

st/mesa: remove st_context callbacks for dri/glx/wgl frontends

Use direct calls.

Other changes:
- kopper called thread_finish twice in the same function.
  Remove the second call.
- Context creation had no-op initializations of cso_context and pipe.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

gallium: remove st_context_iface, use st_context directly

st_context_iface was the base class that st_context inherited.
Just use st_context.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

gallium/pp: don't use st_context_iface, use an explicit callback

st_context_iface will be removed.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

gallium/hud: don't use st_context_iface, use an explicit callback

st_context_iface will be removed.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

gallium: rename st_context_iface::st_manager_private to frontend_context

It's really dri_context for DRI.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

gallium: remove duplicated st_context_iface::st_context_private

It's equal to fscreen, which is almost next to it.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

gallium: rename st_manager to pipe_frontend_screen (think dri_screen)

st_manager is really the base class of dri_screen, so let's call it
pipe_frontend_screen. That's a much better name than the vague "st_manager".

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

zink: do not leave needless shader temps around

This used to not matter, but since we started emitting shader-temps
properly, this causes issues where we end up with samplers and images as
shader-temps. That causes asserts while emitting them.

So let's remove the unused vars as well.

This fixes a piglit regression that somehow went unnoticed on CI.

Fixes: 85964945e7c ("zink: emit vars with nir_var_shader_temp mode")
Tested-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20227>

intel/compiler: Delete sampler key handling for planar format stuff

i965 used these, but Gallium drivers do this lowering via a separate
nir_lower_tex call from st/mesa.  Vulkan drivers don't use these at all.

Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20223>

intel/compiler: Delete key->msaa_16

None of the drivers have used this since we dropped i965, and BLORP
no longer uses it as of the previous commit.  We can also drop the
former compressed_multisample_tex_mask (now padding) field so that
things remain 64-bit aligned.

Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20223>

intel/blorp: Set key->msaa_16 unconditionally on Gfx9+

This will result in us using the TXF_CMS_W message rather than the
TXF_CMS message on Skylake through Tigerlake for 2/4/8x MSAA blits,
which is technically slightly worse.  However, it shouldn't be that
much worse: the TXF_CMS message was removed altogether on Alchemist.

iris and anv set key->msaa_16 unconditionally, to avoid paying the
cost of shader recompiles for a miniscule gain.  crocus and hasvk
don't need to set it as they don't support 16x MSAA.  BLORP already
recompiles based on the sample count, so it could easily keep doing
this for the minor benefit.  But avoiding it will let us drop the
entire msaa_16 key field out of the compiler, which is nice.

Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20223>

intel: Drop compressed_multisample_layout_mask from the compiler keys

The compiler looks at this key field to determine whether to perform
an MCS fetch for a txf_ms or samples_identical texture message, if a
nir_tex_src_ms_mcs_intel source wasn't provided.  If it isn't set,
it instead uses constant 0 (nothing is compressed).

All of the drivers (iris, crocus, anv, hasvk) unconditionally set this
to ~0 because we don't want to pay for costly shader recompiles (which
can cause nasty stuttering).  Most textures are compressed anyway, and
the hardware ignores the l2dms MCS parameter if MCS is disabled.

The only user was BLORP, which sets the key field based on whether the
texture's aux usage has MCS.  But if it has MCS, it also does the MCS
fetch itself and supplies it directly.  Otherwise, it relies on the
compiler to fill in the 0 value.  But it could easily just provide the
0 value itself in that case and not rely on the compiler at all.

With that fixed, we can just drop the key fields entirely.  We leave
them as padding for now to avoid repacking structures; we won't need
to after the next commits anyway.

Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20223>

intel/common: use format struct in aux mapping

Refactor aux mapping with the new format struct and helpers.

Signed-off-by: Jianxun Zhang <jianxun.zhang@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20167>

intel/common: initialize format of aux mapping on GFX12

Signed-off-by: Jianxun Zhang <jianxun.zhang@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20167>

intel/common: fix style of some comments in intel_aux_map.h

Signed-off-by: Jianxun Zhang <jianxun.zhang@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20167>

intel/common: Add a new struct to describe AUX mapping format

The new struct and some helper functions are for further
refactoring.

Reworks:
 * Jordan: Refactor code around aux format array

Signed-off-by: Jianxun Zhang <jianxun.zhang@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20167>

intel/common: Add an enum of formats of AUX mapping

The new enum allows us to support multiple formats in the future.

Signed-off-by: Jianxun Zhang <jianxun.zhang@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20167>

intel: add MTL performance metrics

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20228>

hasvk: Allow aliasing with modifiers for WSI images

Ignore ALIAS_BIT when format comes from WSI because
we have the ability to bind the MEMORY_BINDING_PRIVATE
from the other WSI image.

This commit is the same as f350b78b but for hasvk.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19840>

r600/sfn: implement GDS op channel mask evaluation

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20205>

r600/sfn: allocate pinned registers as SSA

Don't allocate the helper_invocation register as pinned, because it is
not an SSA value.

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20205>

r600/sfn: nir_op_vec results don't need channel pinning

This will be handled by the op that uses the vector

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20205>

r600/sfn: implement TF allowed channel mask override

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20205>

r600/sfn: clean up TF emission in TCS

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20205>

r600/sfn: Cleanup copy-prop into vec4 source values

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20205>

r600/sfn: scheduler allow more lookahead on ALU instr

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20205>