turnip: consider shader's immediates size for sub-stream allocation

Otherwise we could exceed pre-allocated space.

Fixes:
 dEQP-VK.spirv_assembly.instruction.compute.opphi.wide

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8841>

zink: add a piglit ci job for lazy descriptors

this fills out lavapipe's descriptor coverage in ci

Acked-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Acked-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12708>

lavapipe: unbreak push descriptor templates

the stride/offset calculation on this was all wrong, so just unroll the
template before enqueuing to present a uniform stream of descriptors
without trying to copy potentially megabytes of data

Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12708>

radv: expose VK_KHR_shader_integer_dot_product

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12617>

aco/ra: allow v1b operands with 16-bit instructions

Instruction selection can create these.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Fixes: ec1bbfa6088 ("aco/ra: refactor subdword operand stride")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12617>

aco: implement udot_4x8/sdot_4x8/udot_2x16/sdot_2x16 opcodes

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12617>

aco: implement nir_op_pack_32_4x8

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12617>

radv,aco: implement iadd_sat

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12617>

radv: refactor handling of nir_options

Make it easier to change them depending on chip_class and family.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12617>

ac/llvm: implement udot_4x8/sdot_4x8/udot_2x16/sdot_2x16 opcodes

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12617>

ac/llvm,radv: implement uadd_sat/iadd_sat

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12617>

ac/llvm: implement nir_op_pack_32_4x8

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12617>

ac/gpu_info: add has_accelerated_dot_product

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12617>

spirv: use sdot_2x16 and udot_2x16 opcodes

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12617>

nir: add sdot_2x16 and udot_2x16 opcodes

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12617>

nir: separate lower_add_sat

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12617>

aco/spill: add temporary operands of exec phis to next_use_distances_end

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: dfb10e4f4b7 ("aco/spill: don't count phis as variable access")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12702>

aco: don't coalesce constant copies into non-power-of-two sizes

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12702>

vulkan/wsi/x11: fix shm allocation control flow issue

shmget returns -1 on error. alloc_shm assigns it to an unsigned variable
and then checks whether it's < 0, which will never be true.

Found by Coverity.
CID: 1490891

Fixes: 1f55f9a97a5 ("vulkan/wsi/sw: add support for using host_ptr for shm pixmaps.")

Signed-off-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12696>

radv/llvm: rework VS input loads and implement the callback

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12693>

draw/tess: Fix unused-function warning with draw-use-llvm=disabled.

../src/gallium/auxiliary/draw/draw_tess.c:37:1: warning: unused function 'draw_tes_get_input_index' [-Wunused-function]
draw_tes_get_input_index(int semantic, int index,
^

Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12663>

v3d: Enable PIPE_CAP_PRIMITIVE_RESTART

We disabled PIPE_CAP_PRIMITIVE_RESTART on ffc4d82438d98 ("v3d: Disable
PIPE_CAP_PRIMITIVE_RESTART") because the HW can only support
PIPE_CAP_PRIMITIVE_RESTART_FIXED_INDEX subset.

But the introduction of PIPE_CAP_EMULATE_NONFIXED_PRIMITIVE_RESTART
allows us to support these nonfixed primitive restarts so we can now
re-enable NV_primitive_restart for OpenGL extensions.

Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12669>

vc4: remove primconvert

We are losing the optimization of converting a single quad to
a triangle fan reusing the same four vertex in the buffer.

Maybe this optimization can be ported to primconvert, QUADS are
always converted to TRIANGLES with primconvert. And i965, crocus
and svga have similar optimizations.

V3D doesn't implement this optimization and according to when it was
introduced on 230e646a4013 ("broadcom/vc4: Decompose single QUADs to
a TRIANGLE_FAN."):

 "No significant difference in the minetest replay, but it should reduce
  overhead by not requiring that we write quad indices to index buffers
  that we repeatedly re-upload (and making the draw packet smaller, as
  well)."

v2: Commit log includes more detail about the removed optimization.
    (Alejandro Piñeiro)

Reference: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5277
Suggested-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12669>

vc4: export supported prim types by vc4

This is now handled by gallium.

v2: Fix incorrect indentation (Alejandro Piñeiro)

Reference: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5277
Suggested-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12669>

v3d: remove primconvert

Reference: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5277
Suggested-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12669>

v3d: export supported prim types by v3d

This is now handled by gallium.

Reference: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5277
Suggested-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12669>

vulkan: Copy pNext structures when enqueuing commands

There is enough information in vk.xml to figure how to copy the chain of
extensions, so use it.

Otherwise, the caller might have released the structs after recording
the command but before it executes.

Closes #5314.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reported-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Acked-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12701>

Revert "lavapipe: unbreak imageless framebuffer"

This reverts commit c679dbe09cf104ac916fb604fe73528fc87d8e75.

No need to override the generated cmd enqueing function, we can use the
information in vk.xml to do the right thing with pNext arguments.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Acked-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12701>

glsl: use bool literals instead of integers

Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12674>

nouveau: use bool literals instead of integers

Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12678>

nouveau: fix forward declaration of struct

Mismatching class/struct is disliked by MSVC.

Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12698>

radv: optimize VRS when no depth stencil attachment is bound

This is allowed by the Vulkan spec and we have to handle this situation
internally. We used to create and bind a 4096x4096 image to copy the
VRS rates but this wasted too much VRAM (~33MiB). Now, the driver only
allocates a HTILE buffer (~1MiB) and bind it to the framebuffer.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12243>

radv: pass the HTILE buffer to radv_copy_vrs_htile()

Will be used to use a global HTILE buffer without an image.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12243>

radv: optimize copying VRS rates to the global HTILE buffer

By skipping the read operation which is unnecessary.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12243>

radv: allow to conditionally read HTILE value when copying VRS rates

When a subpass is bound without a VRS attachment, the driver has to
create one internally and the copy can be a write only operation.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12243>

aco: preserve subdword RC when lowering p_insert/p_extract

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12640>

aco/print_ir: always print SDWA dst & src selections

This way, it becomes more apparent how SDWA behaves.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12640>

aco: remove explicit dst_preserve flag

Instead, we can rely on the fact that subdword definitions
must preserve the unused bits while dword definitions either
pad or sign-extend.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12640>

aco: rewrite SDWA selector

This commit introduces a new struct SubdwordSel
in order to ease and clean up the usage of SDWA
selections. This includes removing the distinction
between register-allocated and fixed SDWA selections.
Instead, SDWA selections can now also access the high
bits of subdword variables. Alignment and sizes are
validated accordingly. Size, offset and sign_extend
can be evaluated via helper methods.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12640>

aco: fix p_insert lowering with 16bit sources

The previous lowering only wrote a single byte.

Fixes: 2f94353735b5ddfe2a72499e7bf6c7bbc80b9a00 ('aco: add p_extract/p_insert')
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12640>

venus: suggest the proper sampler ycbcr model conversion based on format

Cc: 21.2.3 mesa-stable
Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-by: Ryan Neph <ryanneph@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12694>

radv: remove NGG streamout support in LLVM

It has never really been used due to various issues with GDS in the
past and it will be lowered in NIR at some point.

The driver support is still there because it can likely be re-used.
This implementation can also be used as a reference point.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12695>

zink: use imageless framebuffers

this feature lets zink avoid the screen-based framebuffer cache with locks
in favor of a context-based one that doesn't need any complicated wizardry
to work since it doesn't need to track refcounts for attachments or work
across contexts since the surface info gets passed in when the renderpass
is begun

also expand the dummy surface to an array for use with multisampling and simplify
surface refs there for non-imageless case

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12661>

zink: store some surface metadata to struct during creation

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12661>

zink: store some image creation metadata to object struct

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12661>

zink: move get_framebuffer() to zink_framebuffer.c

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12661>

radv: fix determining the maximum number of waves that can use scratch

This estimation was incorrect, the number of waves doesn't only
depend of the number of VGPRs.
Though, {SPI,COMPUTE}_TMPRING_SIZE.WAVES should limit the number of
scratch waves in flight, not sure if limiting it really works.

This fixes a GPU hang with an upcoming game, and this might also
helps resolving some spurious random GPU hangs.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12700>

aco/lower_phis: optimize loop exit phis

This optimization works by ensuring that disabled lanes
are zero'd before any merge sequence.

Totals from 6075 (4.05% of 150170) affected shaders: (GFX10.3)
CodeSize: 57913908 -> 57913212 (-0.00%)
Instrs: 11055852 -> 11055678 (-0.00%)
Latency: 438705219 -> 438534652 (-0.04%); split: -0.04%, +0.00%
InvThroughput: 125284101 -> 125251397 (-0.03%); split: -0.03%, +0.00%
Copies: 807388 -> 821035 (+1.69%); split: -0.00%, +1.69%
Branches: 391827 -> 391782 (-0.01%)
PreSGPRs: 574841 -> 574838 (-0.00%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11659>

aco/lower_phis: propagate constants before emitting merge code

This generalizes a previous optimization.

Totals from 521 (0.35% of 150170) affected shaders: (GFX10.3)
CodeSize: 1680348 -> 1678884 (-0.09%)
Instrs: 307994 -> 307628 (-0.12%)
Latency: 5799845 -> 5792655 (-0.12%)
InvThroughput: 994859 -> 994030 (-0.08%)
Copies: 18992 -> 18767 (-1.18%)
Branches: 10143 -> 10037 (-1.05%)
PreSGPRs: 21904 -> 21853 (-0.23%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11659>

aco/lower_bool_phis: avoid creating trivial phis

For this purpose, get_ssa() is also refactored slightly.

Totals from 4 (0.00% of 150170) affected shaders: (GFX10.3)
CodeSize: 15504 -> 15376 (-0.83%)
Instrs: 2942 -> 2910 (-1.09%)
Latency: 292444 -> 291642 (-0.27%)
InvThroughput: 30842 -> 30770 (-0.23%)
Copies: 164 -> 150 (-8.54%)
Branches: 96 -> 82 (-14.58%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11659>

aco: refactor lower_phis()

No fossil-db changes.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11659>

aco: fix init_any_pred_defined() for loop header phis

This includes setting the correct end point of the propagation and
not propagating the incoming values after the loop header.
This patch also changes the propagation to a single iteration for
loop exit phis.

No fossil-db changes.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
aco: don't propagate incoming value in init_any_pred_defined()

No fossil-db changes.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11659>

panfrost: tidy up GPU naming to be in line with official names

Offical naming of Mali GPUs is like Mali-G52 or Mali-T860 wth a dash.
Also variants of the same GPU e.g. revision 1 are generally not part
of the name. This adds a dash instead of a space between Mali and the
model and also splits off the r1 revision with a space. This should
keep the naming in-line with offical names and with what the binary
DDK uses for naming.

Signed-off-by: Carsten Haitzler <carsten.haitzler@foss.arm.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12667>

radv: use common vkGet{Buffer,Image}MemoryRequirements()

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12416>

radv: use common vkBind{Buffer,Image}Memory()

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12416>

radv: use common vkGetDeviceQueue()

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12416>

radv: use common vkGetPhysicalDevice{Image}FormatProperties()

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12416>

radv: use common entrypoints for sparse image requirements/properties

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12416>

vulkan: add common entrypoints for sparse image requirements/properties

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12416>

radv: do not disable DCC for storage images if atomics aren't enabled

VK_FORMAT_R32_SFLOAT seems pretty common and it seems we can be a
little smarter when shader image 32-bit float atomics aren't enabled.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12406>

radv: track if shader image 32-bit float atomics are enabled

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12406>

radv: call nir_lower_int64() for LLVM

Seems to run fine now.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12665>

v3dv: make v3dv_image_view derive from vk_image_view

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12668>

v3dv: use subresource helpers in more places

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12668>

v3dv: make v3dv_image derive from vk_image

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12668>

vulkan: allow creating color views from depth/stencil images

Some drivers may want to implement certain depth/stencil operations by
reinterpreting a depth/stencil format as a color format. This is not
strictly allowed by the Vulkan spec, so handle it separately.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12668>

iris: Make sure a bound resource is flushed after iris_dirty_for_history.

This is the last step before we can start removing the history flush
mechanism: In cases where a dirtied buffer has the potential to be
concurrently bound to the pipeline (as indicated by the bind_history
mask), flag the "flush" dirty bits corresponding to its binding point.
This ensures that the buffer-local memory barriers introduced earlier
in this series are executed before the next draw call, which in turn
will emit any necessary PIPE_CONTROLs in cases where the buffer is
bound through a cache incoherent with the cache that performed the
write.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12691>

iris: Track dirty UBOs per-stage for more targeted flushing.

This allows us to skip over individual constant buffer bindings which
haven't been changed since the last flush, or which are set to a user
buffer, which means they don't require flushing.

Omitting this commit would lead to the following statistically
significant Piglit Draw Overhead regressions:

 107/DrawArrays (16 VBO| 8 UBO|  8 Tex) w/ 1 UBO change:   XXX ±2.31% x22 -> XXX ±2.55% x21  d=-3.49% ±2.38%  p=0.00%
 79/DrawArrays ( 1 VBO| 8 UBO|  8 Tex) w/ 8 UBOs change:   XXX ±1.90% x22 -> XXX ±2.25% x21  d=-3.20% ±2.04%  p=0.00%
 78/DrawArrays ( 1 VBO| 8 UBO|  8 Tex) w/ 1 UBO change:    XXX ±2.64% x22 -> XXX ±2.58% x21  d=-2.74% ±2.58%  p=0.12%
 45/DrawElements (16 VBO| 8 UBO|  8 Tex) w/ 1 UBO change:  XXX ±2.53% x22 -> XXX ±2.29% x21  d=-2.41% ±2.39%  p=0.20%
 108/DrawArrays (16 VBO| 8 UBO|  8 Tex) w/ 8 UBOs change:  XXX ±2.10% x22 -> XXX ±1.41% x21  d=-2.36% ±1.78%  p=0.01%
 16/DrawElements ( 1 VBO| 8 UBO|  8 Tex) w/ 1 UBO change:  XXX ±2.44% x22 -> XXX ±1.19% x21  d=-2.12% ±1.93%  p=0.09%
 46/DrawElements (16 VBO| 8 UBO|  8 Tex) w/ 8 UBOs change: XXX ±2.93% x22 -> XXX ±2.44% x21  d=-1.99% ±2.68%  p=1.93%

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12691>

iris: Use separate dirty bits for UBO and SSBO flushes.

This moves UBO+SSBO flushing into a dirty bit separate from the one
used for image and sampler views, which saves some CPU overhead in the
frequent case where buffers from only one or the other set are updated.

Omitting this commit would lead to the following statistically
significant Piglit Draw Overhead regressions:

 107/DrawArrays (16 VBO| 8 UBO|  8 Tex) w/ 1 UBO change:           XXX ±2.31% x22 -> XXX ±1.80% x21  d=-24.31% ±1.91%  p=0.00%
 78/DrawArrays ( 1 VBO| 8 UBO|  8 Tex) w/ 1 UBO change:            XXX ±2.64% x22 -> XXX ±2.21% x21  d=-24.13% ±2.22%  p=0.00%
 45/DrawElements (16 VBO| 8 UBO|  8 Tex) w/ 1 UBO change:          XXX ±2.53% x22 -> XXX ±1.90% x21  d=-23.63% ±2.07%  p=0.00%
 16/DrawElements ( 1 VBO| 8 UBO|  8 Tex) w/ 1 UBO change:          XXX ±2.44% x22 -> XXX ±1.97% x21  d=-23.23% ±2.04%  p=0.00%
 108/DrawArrays (16 VBO| 8 UBO|  8 Tex) w/ 8 UBOs change:          XXX ±2.10% x22 -> XXX ±1.50% x21  d=-22.15% ±1.71%  p=0.00%
 79/DrawArrays ( 1 VBO| 8 UBO|  8 Tex) w/ 8 UBOs change:           XXX ±1.90% x22 -> XXX ±1.70% x21  d=-22.12% ±1.64%  p=0.00%
 17/DrawElements ( 1 VBO| 8 UBO|  8 Tex) w/ 8 UBOs change:         XXX ±2.85% x22 -> XXX ±1.59% x21  d=-21.03% ±2.22%  p=0.00%
 46/DrawElements (16 VBO| 8 UBO|  8 Tex) w/ 8 UBOs change:         XXX ±2.93% x22 -> XXX ±1.09% x21  d=-20.62% ±2.18%  p=0.00%
 7/DrawElements ( 1 VBO| 8 UBO|  8 Tex) w/ vertex attrib change:   XXX ±9.30% x22 -> XXX ±7.02% x21   d=-6.49% ±8.08%  p=1.19%
 68/DrawArrays ( 1 VBO| 8 UBO|  8 Tex) w/ shader program change:   XXX ±1.60% x22 -> XXX ±1.93% x21   d=-2.23% ±1.75%  p=0.01%
 6/DrawElements ( 1 VBO| 8 UBO|  8 Tex) w/ shader program change:  XXX ±2.90% x22 -> XXX ±2.71% x21   d=-2.04% ±2.78%  p=2.08%

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12691>

iris: Insert buffer-local memory barriers for UBO reads.

Similar to what was previously done for other kinds of buffers --
Insert memory barriers at resolves-and-flushes time instead of relying
on the history flush mechanism.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12691>

iris: Insert buffer-local memory barriers for SSBO reads and writes.

Similar to what was previously done for vertex buffers, render
buffers, etc -- Insert memory barriers at resolves-and-flushes time
instead of relying on the history flush mechanism.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12691>

iris: Use DATA domain barrier for shader images instead of OTHER domain.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12691>

iris: Add read-write domain for data cache.

This will allow us to remove the history flushes performed for SSBOs
and instead take advantage of the same mechanism used for tracking
other memory accesses.

v2: Use C99 designated initializers (Ken).

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12691>

iris: Insert buffer-local memory barriers for indirect draw parameters.

This adds buffer-local barriers so any required synchronization
commands are emitted before a buffer object is used as source for
indirect draw parameters.  An unconditional PIPE_CONTROL meant to
flush the contents of the draw count buffer can now be removed, since
it's redundant with the more accurate buffer-local barrier introduced
here, which should avoid flushing in cases where the buffer wasn't
written by any incoherent cache since the last flush.

(Rebased by Kenneth Graunke.)

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12691>

iris: Add separate dirty bit for VBO flushes.

Instead of emitting barriers every time IRIS_DIRTY_VERTEX_BUFFERS is
flagged, use a separate dirty bit and optimize out the barriers in
cases where the same buffer object is re-bound as vertex buffer.

Omitting this commit would lead to the following statistically
significant Piglit Draw Overhead regressions:

 36/DrawElements (16 VBO| 8 UBO|  8 Tex) w/ vertex attrib change:      XXX ±7.22% x22 -> XXX±11.09% x21  d=-20.10% ±8.06%  p=0.00%
 98/DrawArrays (16 VBO| 8 UBO|  8 Tex) w/ vertex attrib change:        XXX ±7.27% x22 -> XXX ±7.70% x21  d=-17.76% ±6.83%  p=0.00%
 69/DrawArrays ( 1 VBO| 8 UBO|  8 Tex) w/ vertex attrib change:        XXX ±9.94% x22 -> XXX ±8.72% x21   d=-7.46% ±9.08%  p=1.02%
 53/DrawElements (16 VBO| 8 UBO|  8 Tex) w/ depth enable change:       XXX ±8.34% x22 -> XXX ±6.88% x21   d=-7.30% ±7.45%  p=0.26%
 61/DrawElements (16 VBO| 8 UBO|  8 Tex) w/ cull face enable change:   XXX±10.22% x22 -> XXX ±8.63% x21   d=-6.75% ±9.23%  p=2.11%
 55/DrawElements (16 VBO| 8 UBO|  8 Tex) w/ stencil enable change:     XXX ±9.30% x22 -> XXX ±7.25% x21   d=-6.60% ±8.16%  p=1.14%
 50/DrawElements (16 VBO| 8 UBO|  8 Tex) w/ viewport change:           XXX ±6.48% x22 -> XXX ±5.93% x21   d=-6.58% ±6.04%  p=0.09%
 54/DrawElements (16 VBO| 8 UBO|  8 Tex) w/ depth clamp enable change: XXX ±9.95% x22 -> XXX ±7.95% x21   d=-6.50% ±8.81%  p=2.02%
 35/DrawElements (16 VBO| 8 UBO|  8 Tex) w/ shader program change:     XXX ±7.27% x22 -> XXX ±7.25% x21   d=-5.77% ±7.06%  p=1.06%

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12691>

iris: Insert buffer-local memory barriers for VF reads.

This ensures that any required synchronization (i.e. PIPE_CONTROL)
commands are emitted whenever a vertex or index buffer is used in
cases where it had been previously accessed from an incoherent caching
domain.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12691>

iris: Annotate all BO uses through VF cache domain.

This promotes some of the previously introduced IRIS_DOMAIN_OTHER_READ
annotations used for vertex data to the more specific
IRIS_DOMAIN_VF_READ domain.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12691>

iris: Add read-only domain for VF cache.

This will allow us to order writes and reads of vertex and index
buffers by using the same cache tracking infrastructure introduced
previously for render and depth buffers.  The ultimate goal is to
remove the somewhat heavy-handed history flush mechanism currently
used for buffer objects, and use a single cache tracking mechanism
across the whole driver.

v2: Use C99 designated initializers (Ken).

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12691>

lavapipe: unbreak imageless framebuffer

vk cmd queue doesn't copy pNext pointers

Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12688>

venus: workaround a blob_mem mappable size check issue

For blob_mem allocated from virtgpu_virgl backend, the guest mappable
size queried can be smaller than the size returned from image memory
requirement query from the host side. Here we temporarily workaround
until we switch to use cross-domain backend in minigbm.

Cc: 21.2.3 mesa-stable
Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-by: Ryan Neph <ryanneph@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12687>

venus: renderer to check map size only when mappable

Cc: 21.2.3 mesa-stable
Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-by: Ryan Neph <ryanneph@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12687>

docs: update features for lavapipe

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12627>

lavapipe: support EXT_primitive_topology_list_restart

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12627>

iris: switch to explicit busy tracking

We're moving away from relying on the Kernel's implicit busy tracking
into our own tracking, except for shared buffers.

Not only this shouldn't hurt now (it doesn't, according to my
measurements), when we switch to vm_bind we will be able to cut some
significant overhead by simply omitting all the async buffers from the
execbuf ioctl.

v2:
- Change iris_bo_busy() to bool (Ken).
- Fix coding style issues (Ken).
- Rebase on not having the refcount _inc and _dec helpers anymore
  (Ken).

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4748
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12363>

iris: give each screen of a bufmgr a unique ID

We want to implement explicit BO dependency tracking and for that
we'll use arrays of dependencies (syncobjs) indexed by screen->id.
This is way more efficient than storing and checking screen pointers
everywhere.

v2: Properly use atomic operations in a non-racy way (Alyssa, Ken).

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12363>

iris: syncobjs are now owned by bufmgr instead of screen

The next patches will justify the new ownership. We want the BOs to
have references on the batches' syncobjs so we can implement implicit
tracking. In other words: BOs will be able to wait on syncobjs owned
by different screens. Since our syncobjs are actually just a Kernel
handle with a refcount, they can be used globally and it makes more
sense to map them to the bufmgr, just like the BOs.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12363>

radv: use radeon_set_sh_reg_seq() more for initial gfx/compute state

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12671>

venus: initialize physical devices once

Avoid re-enumeration when there is no device or no supported device.

Signed-off-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-by: Ryan Neph <ryanneph@google.com>
Reviewed-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12653>

venus: no supported device is not an error

Signed-off-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-by: Ryan Neph <ryanneph@google.com>
Reviewed-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12653>

venus: group physical device fields with a struct

This makes it clear what the mutex protects.

Signed-off-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-by: Ryan Neph <ryanneph@google.com>
Reviewed-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12653>

radv: add some asserts for descriptor updating

let's avoid any further issues with descriptor sizing

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12620>

vulkan: Remove dependency on Python 3.9+

Closes #5311.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Fixes: 997a6ca22654 ("vulkan: Generate entrypoints that enqueue commands")
Fixes: a7b0946ef046 ("vulkan: Generate code to place commands in a queue")
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12672>

freedreno/computerator: Add support for pvtmem

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11876>

ci/freedreno: Add a bunch of recent a530 and a630 flakes.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12683>

ci/freedreno: Annotate a bunch of piglit fails/crashes.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12683>

radv: just use UINT64_MAX when getting absolute timeout for that value

this would otherwise result in (UINT64_MAX - gettime()), which can effectively
be rounded to UINT64_MAX without a noticeable change

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12680>

mesa/st: create new surfaces before destroying old ones when updating attachments

try to avoid patterns that would destroy surfaces for drivers that cache them for
deduplication purposes

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12651>

util/format: NEON is not available with the soft-float ABI

Fixes: 80923e8d58cc ("util/format: Add some NEON intrinsics-based u_format_unpack.")
Reviewed-by: Emma Anholt <emma@anholt.net>
Tested-by: Ross Burton <ross.burton@arm.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12569>

ac/nir/nggc: Move gs_alloc_req up in NGG culling shaders.

This is the first part of a refactor to make vertex compaction optional.
Additionally, it may yield a very small benefit to allocate the PC
space sligtly sooner.

Fossil DB stats on Sienna Cichlid with NGGC on:

Totals from 58239 (45.27% of 128647) affected shaders:
CodeSize: 160502348 -> 160502340 (-0.00%)
Instrs: 30722664 -> 30722662 (-0.00%)
Latency: 137627419 -> 137782218 (+0.11%); split: -0.00%, +0.11%
InvThroughput: 21698587 -> 21699068 (+0.00%); split: -0.00%, +0.00%
Copies: 3288263 -> 3288261 (-0.00%)

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12246>

ac/nir/nggc: Only repack arguments that are needed.

Don't repack everything, only what is actually used.
The goal of this commit is primarily to remove unnecessary
LDS stores and loads. In addition to that, it also gets rid of
a few VALU instructions and reduces VGPR use.

Fossil DB stats on Sienna Cichlid with NGGC on:

Totals from 6951 (5.40% of 128647) affected shaders:
VGPRs: 206056 -> 205360 (-0.34%); split: -0.79%, +0.45%
CodeSize: 12344568 -> 12269312 (-0.61%); split: -0.62%, +0.01%
MaxWaves: 211206 -> 212196 (+0.47%)
Instrs: 2319459 -> 2308483 (-0.47%); split: -0.50%, +0.03%
Latency: 7220829 -> 7164721 (-0.78%); split: -1.21%, +0.43%
InvThroughput: 1051450 -> 1049191 (-0.21%); split: -0.36%, +0.15%
VClause: 25794 -> 25445 (-1.35%); split: -1.97%, +0.61%
SClause: 39192 -> 39277 (+0.22%); split: -0.21%, +0.43%
Copies: 315756 -> 313404 (-0.74%); split: -1.17%, +0.42%
Branches: 127878 -> 127879 (+0.00%); split: -0.00%, +0.00%
PreVGPRs: 168029 -> 160162 (-4.68%)

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12246>