radv: Handle NULL miss shaders

Fixes reflections in DOOM Eternal.

Fixes: 85580faa ("radv: Add ray traversal loop.")
Closes: #6210
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20270>

radv: Change shaderGroupBaseAlignment to RADV_RT_HANDLE_SIZE

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20270>

radv: Force non uniform texture sampling with NV Remix

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20243>

radv: Add a driconf option to force non uniform texture sampling

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20243>

spirv: Add a debug option to force non uniform texture sampling

Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20243>

radv/rt: Hash the pipeline key

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20243>

radv/rt: Propagate radv_pipeline_key

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20243>

radv/rt: Work around incorrect NOT_DEFERRED handling in Portal RTX

Allows Portal RTX to reach the main menu.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20243>

tu: Implement VK_EXT_descriptor_buffer

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19849>

vk/descriptor_set_layout: Add optional destructor

Drivers implementing descriptor buffers will want to allocate and free
descriptors with the layout for embedded samplers, so we need a hook to
allow them to free any GPU buffers.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19849>

tu: Fall back to ldg for variable-size inline uniform blocks

With descriptor buffers, we have no way to know how big the descriptor
set actually is, so we have no idea how many constants we can safely
push. If we use a UBO then it will still get pushed, because we normally
assume that we can freely access UBOs without any fear of faults due to
the range checking. This does the easiest thing of using raw pointer
loads, although performance will fall off a cliff, because we don't have
many better options.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19849>

ir3/a6xx: Fix immediate offset stg/ldg path

The immediate offset is in units of bytes, whereas the register offset
is in dwords. We need to compensate for that.

Also, fix an off-by-one when checking the range - the offset field is 13
bits, but the sign bit means we can only represent up to 1 << 12 in
bytes or 1 << 10 in dwords.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19849>

microsoft/compiler: Fix codegen when a loop ends in a jump

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7792
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20255>

microsoft/compiler: Handle holes in driver_location when adding sysvals

All of the full runtime+compiler stacks reassign these driver_location
values to compact them and sort between shader stages, but for the
spirv2dxil tool, we leave the original shader's "location" intact. That
means that there can be holes in the driver_location space, and simply
counting how many inputs there are can lead to collisions. So instead
place all sysvals after the last-used driver_location.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7811
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20253>

ci: Bump virglrenderer version

Update virglrenderer to the latest version on time.

Signed-off-by: Corentin Noël <corentin.noel@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20277>

ir3/freedreno: Find regs for FS inputs when printing info

FS inputs are not directly loaded into regs, but require additional
instruction to do so. So in order to print in which reg the input
is loaded we have to scan the shader for the instruction
which loads the input.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20247>

radv: fix assertion on gpu hang detection

fixes assert in RADV_DECL_PIPELINE_DOWNCAST when bound pipline is a compute
pipeline

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20276>

intel/compiler: Remove leftover declarations of old NIR passes

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19805>

intel/nir/rt: fixup primitive id

There is a delta index value in the hit structure, we forgot to add it
to the base value.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 046571479028 ("intel/nir/rt: add more helpers for ray queries")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7565
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19346>

radv: fix hashing descriptor set layout

Shouldn't have pointers.

Fixes: 19f8d338761 ("radv: Use vk_descriptor_set_layout")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20250>

nir/lower_shader_calls: Remove phis after dead control flow

This potentially gets rid of some more phis without sources.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19960>

nir: Do not consider phis with incompatible dests equal

CSE tries to collapse equal instructions, and collapsing two phis with incompatible dests is illegal.

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Fixes: 6bdce55c ("nir: Add a basic CSE pass")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19960>

docs/release-calendar: add 22.3.x dates

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20131>

zink: Don't set dynamic color attachment state for 0 attachments.

Fixes some validation failures like:

VUID-vkCmdSetColorBlendEquationEXT-attachmentCount-arraylength(ERROR / SPEC): msgNum: -175001922 - Validation Error: [ VUID-vkCmdSetColorBlendEquationEXT-attachmentCount-arraylength ] Object 0: handle = 0xaaaae7632fa0, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0xf591aebe | vkCmdSetColorBlendEquationEXT: parameter attachmentCount must be greater than 0. The Vulkan spec states: attachmentCount must be greater than 0 (https://www.khronos.org/registry/vulkan/specs/1.3-extensions/html/vkspec.html#VUID-vkCmdSetColorBlendEquationEXT-attachmentCount-arraylength)

However, we still have some around dynamic color attachment state:

    Objects: 1
        [0] 0xaaaafcab4150, type: 6, name: NULL
VUID_Undefined(ERROR / SPEC): msgNum: 2044605652 - Validation Error: [ VUID_Undefined ] Object 0: handle = 0xaaaafcab4150, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x79de34d4 | VkCommandBuffer 0xaaaafcab4150[]: Dynamic color blend enable state not set for this command buffer.
    Objects: 1
        [0] 0xaaaafcab4150, type: 6, name: NULL
VUID_Undefined(ERROR / SPEC): msgNum: 2044605652 - Validation Error: [ VUID_Undefined ] Object 0: handle = 0xaaaafcab4150, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x79de34d4 | VkCommandBuffer 0xaaaafcab4150[]: Dynamic color blend equation state not set for this command buffer.
    Objects: 1
        [0] 0xaaaafcab4150, type: 6, name: NULL
VUID_Undefined(ERROR / SPEC): msgNum: 2044605652 - Validation Error: [ VUID_Undefined ] Object 0: handle = 0xaaaafcab4150, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x79de34d4 | VkCommandBuffer 0xaaaafcab4150[]: Dynamic color write mask state not set for this command buffer.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20186>

radv: Use correct watermark for early loop exit.

The previous check assumed the stack starts at offset=0, which isn't
necessarily true for ray queries.

Note that this didn't cause correctness issues, just made an optimization
not apply. Found when I accidentally made this load-bearing in a
refactor.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20265>

radv: Rename stack_base to stack_low_watermark.

Better covers the purpose.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20265>

Revert "r600/sfn: Make use of variable length DOT"

This reverts commit fcafe1ffc85573b6a0df2d64a438f1bac2bae3a2.

Variable length DOT products are not supported for pre EG
cards, and the read port evaluation is not correctly
checked, so that scheduling might fail. Revert for now to
fix the issues below and get gack with a better implementation
later.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7876
Related: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7878

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20268>

cso: remove cso_draw_vbo from all draws, call the driver or u_vbuf directly

Instead of calling like this:
    st_draw_gallium -> cso_draw_vbo -> driver_draw_vbo

Do it like this:
    st_draw_gallium -> driver_draw_vbo
OR
    st_draw_gallium -> u_vbuf_draw_vbo

It's accomplished by adding a draw_vbo function pointer into cso_context.
The pointer is equal to pipe_context::draw_vbo when needed, so there is
no call overhead from this if cso's draw_vbo callback is indeed equal to
driver_draw_vbo. We just call cso_context_base::draw_vbo to jump into
the driver directly, or u_vbuf if needed.

The cso function with the indirect function call is inlined, so draws
don't actually visit any cso_context function.

Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20025>

cso: add a base class cso_context_base holding pipe_context*

We'll add more stuff there. The first change is that we need pipe_context*
there.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20025>

cso: unify cso_draw_vbo and cso_multi_draw

This is going to be inlined.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20025>

gallium/u_vbuf: change u_vbuf_draw_vbo to accept pipe_context as first param

This makes the parameters equal to pipe_context::draw_vbo.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20025>

gallium: add the u_vbuf pointer into pipe_context

This will allow removing the draw_vbo wrapping in cso_context.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20025>

asahi: Implement occlusion queries

While the hardware supports both counter and boolean occlusion queries, the
programming model is quite different from OpenGL. In AGX (and in Metal),
there is a single "visibility result buffer" associated with the render pass.
Each draw that uses occlusion queries writes into this render pass global
visibility result buffer at a particular index. By contrast, the OpenGL
occlusion query model supposes that each query has independent state that can be
mixed and matched within a render pass. We can't simply allocate backing memory
for a query and write to it from a job. We can't allocate visibility result
buffers for each batch up front and statically assign OpenGL queries to indices,
because the OpenGL query can span multiple batches. Finally we can't use a
global visibility result buffer without introducing additional synchronization,
given that we now support multiple batches in-flight at once.

In this patch, I've elected to use a simple solution: allocate visibility result
buffers and indices on the fly as needed, and accumulate the results on the CPU
at the end of the render pass. When we have proper synchronization we'll want to
revisit this, but as everything is stalling at submit time now, I'm not inclined
to "optimize" something I can't test.

Passes dEQP-GLES3.functional.occlusion_query.* and the relevant piglit tests.
The piglits are considerably more thorough, checking lots of "we hate tilers"
conditions that dEQP skips over.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Move query functions to agx_query.c

New file. They're just stubs now but will get nontrivial in a moment.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Don't upload samplers for clears/stores

Unlikely to help but makes the traces neater.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Avoid reloads with staging blits

Noticed by inspection. Not likely to matter unless these staging blits are in a
hot path, but it's an easy win.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Don't reload uninitialized surfaces

Pointless. This should save some bandwidth in some cases (possibly mipmap
generation?)

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Fix Z32S8 harder

Fixes dEQP-GLES3.functional.texture.format.sized.2d.depth32f_stencil8_pot after
stencil texturing broke it.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Don't shadow idle resources

Pointless allocation+memcpy.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Model alignment of occlusion query indices

8-byte offsets.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Identify some more fields used with layered

These values depend on the framebuffer width/height and maybe other
stuff. Maybe strides.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Identify XML for anisotropic filtering

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Check-box implement rasterizer discard

Passes dEQP-GLES3.functional.rasterizer_discard.*

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Wire in 1D (array) textures

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Dirty track VS/FS key updates

drawoverhead 1 score doubled to 7668.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Separate VS/FS shader keys

First remove agx_shader_key from asahi_shader_key. It's trivial. agx_shader_key
is going to go away soon now that we lower everything in NIR.

Then everything else is mutually exclusive between stages. That means much
less to hash. drawoverhead test 1 from 2331 to 3443.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Implement invalidate_resource

From Panfrost. This lets us avoid storing depth/stencil attachments at the end
of the frame in GLES.

On my 4K monitor, glmark2 -btexture at fullscreen goes from 705fps to 1150fps. I
assume gains on real workloads will be smaller.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Track batch masks on ZS/blend CSO

Adapted from panfrost, with the work happening at CSO create time instead of
draw time allowing us to do more sophisticated analysis. We'll use these for
accurate masks in a moment.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Dirty track blend state

We'll want this to reduce variant lookups eventually.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Enable dirty tracking

Whoops. drawoverhead test 1 score from 496 -> 2377.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

ail: Save level_offsets_compressed_B

So we can bind specific mip levels for rendering into compressed Z/S.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

panfrost: implement clear_depth_stencil

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20238>

nine: enable on panfrost

Also, enable required kmsro dependencies.

Tested-by: Aleksey Komarov <q4arus@ya.ru>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20238>

panfrost: implement clear_render_target

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Co-authored-by: Aleksey Komarov <q4arus@ya.ru>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Aleksey Komarov <q4arus@ya.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20238>

panfrost: Handle resources without depth in batch_to_fb_info

Prevent preloading data from resources which doesn't exist.

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Aleksey Komarov <q4arus@ya.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20238>

panfrost: Implement GL_EXT_clip_control

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Co-authored-by: Aleksey Komarov <q4arus@ya.ru>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Aleksey Komarov <q4arus@ya.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20238>

intel: add devinfo->has_64bit_float_via_math_pipe

Unusual hardware features that require special hanlding usually get a
devinfo field, so do this for MTL's unordered DF types. This will
guarantee that any platform based on MTL (thus inheriting from
MTL_FEATURES) will automatically be handled in these special cases.

v2: s/has_unordered_64bit_float/has_64bit_float_via_math_pipe/ (Curro).

Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20072>

intel/compiler: fix intel_swsb_decode for newer platforms

In the previous patch we adjusted the scoreboard pass to take into
consideration a new case of unordered operations for TGL. Fix the
decoding as well.

v2: use intel_device_info_is_mtl()  (Curro, Jordan)
v3: the part where we export num_sources_from_inst() is now a separate patch
    (Curro).
v4: Work around false positive maybe-unitialized warning since Marge
    uses -Werror=maybe-uninitialized (Marge).

Reviewed-by: Francisco Jerez <currojerez@riseup.net> (v3)
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20072>

intel/compiler: export brw_num_sources_from_inst

We want to call this from brw_disasm.c, so move it out to brw_eu.c
since it's about to become more of a shared utility function than
something specific to the EU validator.

Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20072>

intel/compiler: avoid 64bit SEL_EXEC on MTL

On MTL, instructions with DF type are unordered, executed in the math
pipe. This means that they require different SWSB dependency handling,
and also that in some cases such as MOVs it's generally faster to
simply use 2 smaller ordered moves than a single unordered MOV.

One problem we have with the current code is that generate_code() is
not setting the proper SWSB dependencies for the generated DF MOVs,
causing some tests to fail.

One solution would be to fix generate_code() by making it set the
appropriate dependencies. This was the first patch I wrote. Another
solution to this problem, pointed to us by Curro, is to change
required_exec_type() so we use UD instructions instead of DF, just
like we do with platforms that don't have 64 bit instructions, which
means there won't be anything to fix in generate_code(). The second
solution is what this patch implements.

This fixes at least:
 - dEQP-VK.subgroups.arithmetic.framebuffer.subgroupmin_double_vertex

Thanks to Francisco Jerez for all the major help provided with this
problem.

Credits-to: Francisco Jerez <currojerez@riseup.net>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20072>

intel/compiler: avoid (RegDist, SBID) on DF instructions on MTL

When we use this form there's no way to specify which pipe RegDist
refers to, so there are a few rules to figure this out, which is what
inferred_sync_pipe() implements. But for MTL there's no long pipe and
the documentation does not explicitly explain what should be the
inferred type for its long (DF) instructions - which are out-of-order,
by the way.  One way to interpret this is that such case should be
avoided.  So add the extra check to entirely avoid this case.

Notice that this is not actually fixing any bug, since returning
TGL_PIPE_LONG (what we do today) will actually make these DF
instructions incompatible with every in-order instruction, so we'll
never opt to use the (RegDist, SBID) form anyway. But still, it's
better to have this case explicitly documented instead of having it
covered by a semi coincidence.

v2: use intel_device_info_is_mtl()  (Curro, Jordan)

Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20072>

intel/compiler: on MTL, DF instructions run in the math pipe

Adjust the scoreboard code to take that into account.

Fixes at least:
  - dEQP-VK.glsl.builtin.precision_double.refract.compute.vec3
  - dEQP-VK.glsl.builtin.precision_double.matrixcompmult.compute.mat4

v2: use intel_device_info_is_mtl()  (Curro, Jordan)

Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20072>

intel/fs: Make the result of is_unordered() dependent on devinfo.

Reviewed-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20072>

Revert "anv: compile anv_acceleration_structure.c"

This reverts commit 74d0be27ae9eb666df948874a20a3b4464df7db1.

Also remove anv_acceleration_structure.c, it was meant to be removed
earlier. There was probably a rebase issue somewhere.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20248>

freedreno/a6xx: fix blend all_mrt_write_mask

Fix all_mrt_write_mask when independent_blend_enable is false.
Otherwise, lrz write is always diabled with MRT when
independent_blend_enable is false.  This fixes a 2% perf regression for
multiple gfxbench benchmarks.

Fixes: 0132c22de75 ("freedreno/a6xx: Don't disable LRZ for invalid channels")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20254>

iris: Improve direct CPU map heuristics

We were promoting reads with a valid primary to direct CPU maps even
if the mmap mode was IRIS_MMAP_WC, which would mean uncached reads from
VRAM.  In that case, GPU blits are in fact useful!

We were also only checking for !DISCARD_RANGE rather than MAP_READ,
which isn't a great idea for image maps, given the discussion in the
previous commit about image map semantics.

The original code was also just confusingly structured.  Make a helper
function with clearly defined cases where we want to bail on CPU maps.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19209>

iris: Only copy existing data into staging images with PIPE_MAP_READ

When performing transfer maps on images that require staging buffers
(say, for presenting a linear view of tiled memory), we were reading
the existing contents of the buffer into the staging resource on map
unless PIPE_MAP_DISCARD_RANGE was set.

The thinking was to support partial writes.  If you map a subrectangle
of an image, but then only write selective pixels - should it preserve
the existing contents of the mapped region?  I believed that it should,
unless you pass PIPE_MAP_DISCARD_RANGE to explicitly say that that it's
okay to invalidate the destination region.

However, that does not appear to be the interpretation favored by other
Mesa developers (in particular Michel Dänzer and Marek Olšák).  The
radeonsi driver does not do this readback from the destination region
to the staging buffer unless you pass PIPE_MAP_READ.  If you want to
do a partial write and preserve contents, you need to pass both flags:
(PIPE_MAP_READ | PIPE_MAP_WRITE).  Passing READ is expected to come
with an associated cost.

OpenGL defines GL_MAP_INVALIDATE_RANGE_BIT for mapping buffer objects,
which is translated to PIPE_MAP_DISCARD_RANGE.  However, unextended
OpenGL doesn't define mapping textures.  There are two main sources of
image maps: our internal MapTextureImage() hook, and gbm_bo_map().

I've audited our internal MapTextureImage() calls, and while some do
pass PIPE_MAP_DISCARD_RANGE, almost all of them wholly overwrite the
mapped region, and those that care about combining with existing image
contents all pass PIPE_MAP_READ.  So this should work there.

GBM defines three flags: GBM_BO_TRANSFER_READ, WRITE, and READ_WRITE.
There is no defined "invalidate range" bit.  In issue #6020, Matthias
Treydte notes that this extra readback can cause performance problems,
and with iris's current interpretation, there's no way to avoid it.
During that discussion, Michel and Matthias both argued that
GBM_BO_TRANSFER_WRITE should invalidate the destination contents and
avoid the readback, while GBM_BO_TRANSFER_READ_WRITE would preserve it.

This patch makes iris follow that model for image mappings, removing
readback on staging maps for both detiling and stall avoidance, unless
PIPE_MAP_READ is passed.  I believe we can change this with impunity.

For buffer objects, Ian Romanick and I both agree that partial writes
should be supported, and GL_MAP_INVALIDATE_RANGE_BIT exists precisely
to indicate that you should spend effort preserving existing contents.
So we continue doing readback for buffers unless PIPE_MAP_DISCARD_RANGE
is flagged, for now.  While I think this is work, it also seems to be
undertested in the CTS and Piglit.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6020
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19209>

iris: Return idle status from iris_invalidate_buffer, skip busy checks

If we successfully replace the backing storage for a buffer, we know
that it's idle, and the transfer map code can mark it unsynchronized
right away, letting us skip redundant resource_is_busy() checks.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19209>

iris: Don't replace backing storage for exported buffers.

We already gave out the old BO...or acquired it from somewhere which
may be affecting it.  We simply can't replace the backing store.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19209>

iris: Promote DISCARD_RANGE to DISCARD_WHOLE_RESOURCE where possible

This allows us to replace the backing storage for a buffer, which
means we'd have an idle buffer and thus could do an unsynchronized
mapping where we otherwise wouldn't.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19209>

iris: Use persistent mappings for pinned memory (userptr)

This is a port of Nicolai's b52721e3b693e113aa537d163c8a855169e7b75d
from radeonsi.  Because GL_AMD_pinned_memory guarantees that mappings
will refer to the same underlying page, we need to avoid using staging
maps.  Using a persistent map is a reasonable way to accomplish this.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19209>

iris: Delete map->dest_had_defined_contents

Dead since commit 6cc09699cd77286f7601a1afab60dc9fa0bafd48.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19209>

nir/gather_info: use nir_ssa_scalar_resolved

This lets us skip copies.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19597>

vtn: add mesh output and task_payload to vtn_mode_is_cross_invocation

This fixes a potential race condition, and removes output loads (which
should not exist in the EXT_mesh_shader).

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7391
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19597>

nir: add task_payload and shader_out to nir_var_vec_indexable_modes

Since these can be cross-invocation, we need this to write individual
components without race conditions or loads.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Gitlab: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7391
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19597>

radv: fix task payload lowering when shared_memory_explicit_layout=true

If shared_memory_explicit_layout=true, we would have skipped lowering task
payload variables to explicit types.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19597>

radv: fix mesh shaders with null winsys

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19597>

panvk: Delete panvk_CmdSetDeviceMask, panvk_GetDeviceGroupPeerMemoryFeatures

Delete panvk_CmdSetDeviceMask and panvk_GetDeviceGroupPeerMemoryFeatures
so that the vk_common_* version will be used instead. This will avoid
repeated code.

Signed-off-by: Rebecca Mckeever <rebecca.mckeever@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20218>

hasvk: Delete VK_KHR_device_group provided entrypoints

Delete anv_CmdDispatch, anv_CmdSetDeviceMask, and
anv_GetDeviceGroupPeerMemoryFeatures so that the vk_common_*
versions will be used instead. This will avoid repeated code.

Signed-off-by: Rebecca Mckeever <rebecca.mckeever@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20218>

anv: Delete VK_KHR_device_group provided entrypoints

Delete anv_CmdDispatch, anv_CmdSetDeviceMask, and
anv_GetDeviceGroupPeerMemoryFeatures so that the vk_common_*
versions will be used instead. This will avoid repeated code.

Signed-off-by: Rebecca Mckeever <rebecca.mckeever@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20218>

tu: Delete VK_KHR_device_group provided entrypoints

Delete tu_CmdDispatch, tu_CmdSetDeviceMask, and
tu_GetDeviceGroupPeerMemoryFeatures so that the vk_common_*
versions will be used instead. This will avoid repeated code.

Signed-off-by: Rebecca Mckeever <rebecca.mckeever@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20218>

v3dv: Delete VK_KHR_device_group provided entrypoints

Delete v3dv_CmdDispatch, v3dv_CmdSetDeviceMask, and
v3dv_GetDeviceGroupPeerMemoryFeatures so that the vk_common_*
versions will be used instead. This will avoid repeated code.

Signed-off-by: Rebecca Mckeever <rebecca.mckeever@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20218>

radv: Delete VK_KHR_device_group provided entrypoints

Delete radv_CmdDispatch, radv_CmdSetDeviceMask, and
radv_GetDeviceGroupPeerMemoryFeatures so that the vk_common_*
versions will be used instead. This will avoid repeated code.

Signed-off-by: Rebecca Mckeever <rebecca.mckeever@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20218>

vulkan/runtime: Add VK_KHR_device_group provided entrypoints

Add entrypoints vk_common_CmdDispatch, vk_common_CmdSetDeviceMask,
and vk_common_GetDeviceGroupPeerMemoryFeatures in Mesa Vulkan runtime so
that they are available to all drivers.

Signed-off-by: Rebecca Mckeever <rebecca.mckeever@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20218>

gallium: add new variable for video frame statistics

Video encoder previously reuses the associated_data variable to output
encoding statistics, but it ended up breaking when transcoding. This
commit adds a new variable just for statistics.

Fixes: 2d1bd619dfd ("frontends/va: add ability for encoder to output statistics")
Signed-off-by: Thong Thai <thong.thai@amd.com>
Reviewed-by: Boyuan Zhang <Boyuan.Zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20190>

radv: remove some unnecessary 64-bit IO handling

nir_lower_io() lowers these to 32-bit.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20137>

radv: fix streamout with different streams in the same varying slot

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20137>

aco/ra: don't swap create_vector operand with definition blocker for SGPRs

There is no SGPR swap instruction, we always need 3 XORs.

fossil-db (navi21):
Totals from 76 (0.06% of 135636) affected shaders:
Instrs: 58400 -> 58347 (-0.09%); split: -0.10%, +0.01%
CodeSize: 312580 -> 312368 (-0.07%); split: -0.08%, +0.01%
Latency: 843333 -> 843180 (-0.02%); split: -0.02%, +0.00%
InvThroughput: 126431 -> 126412 (-0.02%)
Copies: 4008 -> 3955 (-1.32%); split: -1.47%, +0.15%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20240>

aco/ra: don't swap p_create_vector operand with definition blocker for scc

SCC is 1-bit, and we can't copy a 32-bit value into it.

Fixes dEQP-VK.spirv_assembly.type.scalar.i32.iequal_tesse with
ACO_DEBUG=noopt.

No fossil-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: 9476986e6f6 ("aco/ra: special-case get_reg_for_create_vector_copy()")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20240>

nir/opt_algebraic: Optimize open coded bfm.

Foz-DB Navi21:
Totals from 1553 (1.15% of 134913) affected shaders:
SpillVGPRs: 2246 -> 2223 (-1.02%); split: -1.42%, +0.40%
CodeSize: 10409156 -> 10410720 (+0.02%); split: -0.03%, +0.04%
Instrs: 1899725 -> 1898773 (-0.05%); split: -0.07%, +0.02%
Latency: 71225814 -> 71118314 (-0.15%); split: -0.21%, +0.06%
InvThroughput: 13384926 -> 13330369 (-0.41%); split: -0.47%, +0.06%
VClause: 38309 -> 38284 (-0.07%); split: -0.17%, +0.11%
SClause: 70743 -> 70706 (-0.05%)
Copies: 167296 -> 167230 (-0.04%); split: -0.28%, +0.24%
Branches: 42446 -> 42444 (-0.00%); split: -0.01%, +0.00%
PreVGPRs: 95191 -> 95188 (-0.00%)

Some minor instructions count regressions in parallel-rdp
because v_bfm_b32 can't use SDWA, but overall an improvement.

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18887>

frontends/va: fix gst videotestsrc h264 enc fail issue.

problem:
when doing "gst-launch-1.0 -v videotestsrc num-buffer=10 !
   vaapih264enc ! fakeink"

The command will fail due to gst will fetch the first
available supported format in the list, it becomes P010_LE
due to the commit in

[0b02db3007]
frontends/va: fixed av1 decoding 10bit ffmpeg output YUV issue

fix:
move the P010_LE code block to the end of the function, the sequence
of the supported formats restored to its original.

cc: mesa-stable

Reviewed-by: Boyuan Zhang <Boyuan.Zhang@amd.com>
Signed-off-by: Ruijing Dong <ruijing.dong@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20242>

frontend/osmesa: inherit pipe_frontend_drawable instead of allocating separately

This is required by st/mesa now.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

frontend/hgl: inherit pipe_frontend_drawable instead of allocating separately

This is required by st/mesa now.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

gallium: rename st_framebuffer_iface -> pipe_frontend_drawable, etc.

Also rename:
    iface -> drawable
    stfb -> drawable (where it means dri_drawable and not st_framebuffer)
    stfbi -> drawable or pdrawable (if drawable exists)

pipe_frontend_drawable* is really just dri_drawable* for DRI, and WGL/GLX
have their own variants. This makes it easier to understand what kind of
object is being used.

I always wondered what st_framebuffer_iface, iface, stfbi, iface_stamp,
and iface_ID actually mean. Now those terms are gone forever.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

gallium: remove pipe_frontend_screen::destroy callback, call it directly

This is the only one implemented by mesa/state_tracker.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

gallium: clean up comments in api.h, cosmetic changes

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

gallium: remove st_framebuffer_iface::st_manager_private

It's always equal to self.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

frontend/glx: inherit st_framebuffer_iface instead of allocating separately

All other frontends do it like this. This enables the next cleanup.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

gallium: remove unused ST_CONTEXT_ERROR_*

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>

gallium: rename st_manager_private -> st_screen

Oh you didn't know that st_context had a screen object too?
Yeah, this code is full of surprises.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20027>