radv: declare a new shader argument for loading the VRS rates

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14713>

nir: add nir_intrinsic_load_vrs_rates_amd

This intrinsic specific to RADV will be used to load VRS rates from
an user SGPR when RADV_FORCE_VRS is enabled by the application.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14713>

anv: use vk_image_view::format for creating dynamic renderpasses

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15007>

vulkan: Add back vk_image_view::format

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15007>

vulkan: Rename vk_image_view::format to view_format

When I originally added vk_image_view, I was overly clever when it came
to the format field.  I decided to make it only contain the bits of the
format contained in the selected aspects.  However, this is confusing
(not generally a good thing) and it's also not always what you want.
The Vulkan 1.3.204 spec says:

    "When using an image view of a depth/stencil image to populate a
    descriptor set (e.g. for sampling in the shader, or for use as an
    input attachment), the aspectMask must only include one bit, which
    selects whether the image view is used for depth reads (i.e. using a
    floating-point sampler or input attachment in the shader) or stencil
    reads (i.e. using an unsigned integer sampler or input attachment in
    the shader). When an image view of a depth/stencil image is used as
    a depth/stencil framebuffer attachment, the aspectMask is ignored
    and both depth and stencil image subresources are used."

So, while the restricted format makes sense for texturing, it doesn't
for when the image is being used as an attachment.  What we probably
actually want is both versions of the format.  We'll call the one given
by the VkImageViewCreateInfo vk_image_view::format and the restricted
one vk_image_view::view_format.

This is just the first commit which switches format to view_format so
the compiler will make sure we get them all.  The next commit will
re-add vk_image_view::format but this time unmodified.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15007>

venus: properly destroy deferred ahb image before real image creation

Fixes: 19b7b09885c ("venus: prepare image creation helpers for AHB")

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15037>

draw: Don't look at .nir if !IR_NIR.

I suspect this double-check and comment was due to originally using ir.nir
as the condition, which might be uninitialized if !IR_NIR.  You could only
take the branch if IR_NIR was set, and you should always not take if it
!IR_NIR, so it worked out in the end, but it would cause spurious valgrind
warnings if you hadn't zeroed out your TGSI shader's struct.

Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14896>

i915g: Initialize the rest of the "from_nir" temporary VS struct.

draw looked at the uninitialized XFB state, which should just be zeroed
out since i915 doesn't have XFB.

Fixes: 2b3fc26da8be ("i915g: Switch to using nir-to-tgsi.")
Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14896>

r300: Delete the loop unrolling.

There were two paths in this code: One was transform_loops, which would
try to detect loops with an iteration count it understood and unroll that
many times.  The other was emulate_loops, which would just figure out how
many instructions the program could have and still compile (hopefully),
and unroll this loop however many times would fit in that.

The transform_loops had no analysis as good as GLSL or NIR loop unrolling
have, so it shouldn't be missed -- any opportunity it found would only be
due to bugs in the unrolling code.

The emulate_loops path had an issue with computing the number of times it
should try to unroll -- if you had more instrs than ALUs available
already, you'd overflow and unroll approximately infinitely many times,
OOMing the system.  But, also, it's better to throw a compiler error about
unsupported loops than to run the loop an incorrect number of times and
call it a success.

Fixes: #5883, #6018
Reviewed-by: Filip Gawin <filip.gawin@zoho.com>
Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Reviewed-by: Pavel Ondračka <pavel.ondracka@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15004>

zink: radv ci updates

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15033>

vulkan/wsi/wayland: ensure added formats have flags

A format needs to be either alpha or opaque, but can't be neither.

Signed-off-by: Simon Ser <contact@emersion.fr>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Acked-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14874>

vulkan/wsi/wayland: de-duplicate wsi_wl_display_add_wl_shm_format

Re-use wsi_wl_display_add_drm_format_modifier from
wsi_wl_display_add_wl_shm_format instead of maintaining two
separate switches for DRM and shm formats.

Signed-off-by: Simon Ser <contact@emersion.fr>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Acked-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14874>

vulkan/wsi/wayland: introduce wsi_wl_display_add_vk_format_modifier

This is a helper to avoid repetitive code in
wsi_wl_display_add_drm_format_modifier.

No functional changes, just refactoring.

Signed-off-by: Simon Ser <contact@emersion.fr>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Acked-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14874>

vulkan/wsi/wayland: switch from alpha/opaque bools to bitfield

This makes the numerous wsi_wl_display_add_vk_format calls easier
to follow: "ALPHA" is easier to decode than "true, false".

No functional changes, just refactoring.

Signed-off-by: Simon Ser <contact@emersion.fr>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Acked-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14874>

vc4/ci: update failing piglit tests

See https://gitlab.freedesktop.org/mesa/mesa/-/issues/6038.

Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15012>

iris: fix a leak on surface states

Cc: mesa-stable
Closes:https://gitlab.freedesktop.org/mesa/mesa/-/issues/6013

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15010>

gallivm: fix missing cast in 4-bit blending paths.

This got noticed on an llvm debug build.

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14987>

zink: ci updates

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15020>

zink: always invalidate streamout counter buffer if not resuming

this otherwise treates begin/end/begin the same as begin/pause/resume

cc: mesa-stable

fixes:
KHR-GL46.texture_view.view_classes
KHR-GL46.transform_feedback.capture_geometry_separate_test
KHR-GL46.transform_feedback.capture_vertex_separate_test
KHR-GL46.transform_feedback.query_geometry_separate_test
KHR-GL46.transform_feedback.query_vertex_separate_test

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15020>

zink: export PIPE_SHADER_CAP_INDIRECT_TEMP_ADDR

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15020>

zink: map R8G8B8X8_SRGB -> R8G8B8A8_SRGB

this fixes a weird texstore bug that seems specific to this format

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15020>

zink: ci updates

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15018>

zink: activate conditional render for compute dispatch when necessary

fixes:
KHR-GL46.compute_shader.conditional-dispatching

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15018>

zink: restart conditional render when crossing batch boundary

this will only happen when conditional render was being used outside
of a renderpass

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15018>

zink: always terminate conditional render when flushing a batch

we might not know whether conditional render is active,
so forcibly disable when necessary

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15018>

zink: track internal conditional render state

this allows no-oping redundant calls

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15018>

intel/compiler: remove what looks like a bad rebase

This bit in the compiler looks like it was added by accident on one of
the latest versions of the original commit, but it clearly doesn't
belong there.

Fixes: 03e1e19246d ("anv: Refactor descriptor copy")

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15016>

gallivm/st/lvp: add flags arg to get_query_result_resource api.

Currently this just has wait, but in order to get the right answer
for vulkan partial, lavapipe/llvmpipe need to pass a partial flag
through here in the future.

This just changes the API so that's possible.

v2: use an enum (zmike)

Acked-by: Ilia Mirkin <imirkin@alum.mit.edu>
Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15009>

nir_to_tgsi: Add support for FBFETCH.

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15014>

nir_to_tgsi: Don't vectorize 64-bit instructions, to keep virgl happy.

virglrenderer makes invalid shaders when faced with vector 64-bit
instructions, which GLSL-to-TGSI never produced.  While this doesn't fix
everything, it does get more tests running, and virgl probably the primary
consumer of 64-bit TGSI.  virgl may be deprecating its host 64-bit
support, at which point we can drop this workaround.

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15014>

virgl: Move double operands to a temp to avoid double-swizzling bugs.

virglrenderer applies the swizzle when packing doubles.  Since we use scan
now, we can just use the file count for texture detection.

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15014>

virgl: Move tex immediate operands to a temp to avoid virglrenderer bug.

Prior to the noted MR, virglrenderer encoded the tex operands in a
limited-size temp buffer which we could easily overflow with TGSI
immediates.  GLSL-to-TGSI always emitted an extra MOV, so keep that
behavior when nir-to-tgsi feeds us more optimized TGSI.

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15014>

virgl: Add workarounds for virglrenderer input/sv signedness bugs.

GLSL-to-TGSI would emit some MOVs that made things OK, but NTT
successfully copy-propagates the inputs and sysvals more, triggering bugs
in virglrenderer when the signedness of the input is different from that
of the ALU op.

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15014>

virgl: Apply TGSI transforms to compute shaders, too.

We need to do this for the upcoming virglrenderer workarounds.

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15014>

virgl: Add a workaround for virglrenderer output writemask bugs.

Various workaround paths in virglrenderer assume that outputs are written
with a full writemask, which is not required by TGSI.  To work around
that, store affected outputs in a temp and do a full write after each
writemasked write.

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15014>

virgl: Work around old virglrenderer's BARRIER counting bug.

One less regression from doing nir-to-tgsi on CI's virglrenderer.

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15014>

tgsi_translate: Make the procType public when translating.

This means that tgsi_translate users can check the PIPE_SHADER stage
without having to separately tgsi_scan().

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15014>

venus: Exposes VK_EXT_4444_formats extension

Allows venus to passthrough the VK_EXT_4444_formats extension to
the vulkan client.

And add code to the device initialization and feature query
functions.

Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-by: Yiwei Zhang <zzyiwei@chromium.org>
Signed-off-by: Igor Torrente <igor.torrente@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14954>

turnip: advertise VK_EXT_queue_family_foreign

Both Venus and Android AHB requires this extension.

Turnip ignores VK_SHARING_MODE_EXCLUSIVE so this is a no-op.

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Acked-by: Rob Clark <robdclark@chromium.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14836>

ir3: Limit the maximum imm offset in nir_opt_offset for shared vars

STL/LDL have 13 bits to store imm offset.

Fixes crash in CS compilation in Monster Hunter World.

Fixes: b024102d7c2959451bfef323432beaa4dca4dd88
("freedreno/ir3: Use nir_opt_offset for removing constant adds for shared vars.")

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14968>

intel/compiler: fix array & struct IO lowering in mesh shaders

We really need offsets to be in dwords, not in vec4s.

The bug manifests as random failure of func.mesh.clipdistance.5 crucible
test, where stores to gl_MeshVerticesNV[x].gl_ClipDistance[4+n] actually write to
gl_MeshVerticesNV[x].gl_ClipDistance[1+n].

Fixes: 1f438eb0337 ("intel/compiler: Implement Mesh Output")
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14997>

spirv: Create PRIMITIVE_INDICES for NV_mesh_shader on-demand.

The shader can have SpvOpWritePackedPrimitiveIndices4x8NV while the
output variable may not exist. This seems to be a defect in the
NV_mesh_shader SPIR-V spec, let's work around it by creating the
variable on-demand.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15005>

compiler: Extract num_mesh_vertices_per_primitive function.

Prevent code duplication.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15005>

radv: remove set but unused radv_buffer::shareable

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14937>

radv: remove useless NULL checks in vkBind{Buffer,Image}Memory2()

The memory object must be a valid vkDeviceMemory handle.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14937>

radv: fix finding shaders by PC

Shaders are allocated contiguously in memory for a pipeline and
the freelist.next pointer is a pointer to the pipeline now.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14950>

radv: make the trap handler shader BO resident

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14950>

radv: fix allocating/uploading the trap handler shader

Since shaders are allocated per pipeline, the trap handler shader
was not uploaded at all.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14950>

docs/features: Mark VK_KHR_ray_query in progress

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15000>

freedreno/ir3: split up load/store/atomic by generation

Some bits are slightly different on a4xx. Use the encodings that work.
Perhaps these can be combined at some point if we get a proper
understanding of what they mean.

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14789>

isaspec: add gen-based leaf bitset separation

This is necessary for some ops which have slightly different encoding on
a4xx/a5xx, but are otherwise identical. This helps keeping the compiler
from having to worry about these details and creating separate ops.

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14789>

isaspec: fix gen_max to be 2^32-1

The minus sign has higher preference than shift:

>>> 1 << 32 - 1
2147483648
>>> (1 << 32) - 1
4294967295

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14789>

panfrost: Flesh out the Shader Program Descriptor

Only breaking change since Bifrost is that the shader contains barrier? flag is
now fragment-only, meaning it is just a spawn helper threads flag. This affects
compute shaders slightly.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15003>

panfrost: Simplify Valhall preload descriptor

Honestly, we could stand to do the same to Bifrost...

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15003>

panfrost: Clarify unknowns in z/stencil descriptor

Depth culling and clamping.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15003>

panfrost: Add more fields to Attribute Descriptor

More XML

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15003>

panfrost: Update primitive descriptor for Valhall

Contains stuff needed for layered rendering. Unfortunately, there's no more
provoking vertex per draw -- ugh! That's fine for Vulkan (just don't set
provokingVertexModePerPipeline), but requires inserting extra flushes on desktop
OpenGL.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15003>

radv: Only wait on CS/PS to finish if we wait on a semaphore.

I think plain submission doesn't need it.

Reviewed-By: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14574>

radv: Fix preamble argument order.

Used the wrong cmdbuffer in the wrong situation. Oops.

Fixes: 915e9178faf ("radv: Split out commandbuffer submission.")
Reviewed-By: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14574>

radv: Use larger arena sizes.

For some games that take like 400 MiB of shader binaries, the
number of shader arenas ends up going >1500. Cut that down a bit
by using larger arenas.

8 MiB should still be decent with small BAR and should still cut
things down from ~1500 to ~50 buffers.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14591>

lima/ppir: refactor bitcopy to use unsigned char

This code does not work as expected when built with clang and
-fstrict-aliasing.
Redefine it in unsigned char operations so that it does not
violate strict aliasing rules.

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Cc: 22.0 <mesa-stable>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14894>

lima/ppir: initialize slots array for dummy/undef

Some functions in ppir iterate the ppir_op_info slots arrays looking
for the PPIR_INSTR_SLOT_END token. The dummy/undef internal ops may
appear in the scheduling code and their slots arrays did not contain
that token, which could result in invalid array reads.
Reported by gcc -fsanitize=address.

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Cc: 22.0 <mesa-stable>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14894>

lima/gpir: avoid invalid write in regalloc

Reported by gcc -fsanitize=address, sometimes gpir regalloc attempts to
handle an uninitialized node->value_reg (containing the value -1), which
results in an invalid array access.
Avoid it for now to prevent crashes, but more investigation may be
required later on.

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Cc: 22.0 <mesa-stable>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14894>

lima: remove an unneeded lima_job_get assignment

Reported by scan-build.

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14894>

lima: add some checks for potential null pointer dereference

scan-build complains about a potential null pointer dereference in
some places around the lima code.
None of those seem to be a real issue as of now, but let's add some
asserts to cover for that and clean up the warning list.

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14894>

lima: fix warning of garbage value access

scan-build complains that an access of reg[j+1] in this code might
return garbage.
Let's take the chance to clean this open coded sorting code up and
just use qsort.

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14894>

lima/ppir: initialize spill_costs array in regalloc

Static analysis complains that spill_costs might be accessed in
non-initialized positions.
It does not seem to be an issue with the current code which initializes
it for every relevant register index, but we can also just initialize it
to not have to worry about that.

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14894>

lima/ppir: avoid ppir_codegen_outmod implicit conversion

Fix some clang -Wenum-conversion warnings like:

  warning: implicit conversion from enumeration type 'ppir_outmod' to
  different enumeration type 'ppir_codegen_outmod' [-Wenum-conversion]

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14894>

lima/ppir: clean up override-init warnings

Define ppir_op_unsupported as 0 so that we don't have to do the
initialization to -1.

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14894>

lima/gpir: clean up override-init warnings

Define gpir_op_unsupported as 0 so that we don't have to do the
initialization to -1.

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14894>

venus: use 64KB alignment for suballocations

TGL CCS surface addresses must be aligned to 64KB.

Reviewed-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15001>

venus: no roundtrip needed for shmem backed by BLOB_MEM_HOST3D

A successful DRM_IOCTL_VIRTGPU_MAP on BLOB_MEM_HOST3D implies a
roundtrip.

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-by: Ryan Neph <ryanneph@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14658>

venus: init renderer_info at renderer creation (part 2)

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-by: Ryan Neph <ryanneph@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14658>

venus: init renderer_info at renderer creation (part 1)

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-by: Ryan Neph <ryanneph@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14658>

aco/insert_exec_mask: refactor and remove some unnecessary WQM handling code

Some cases cannot happen and don't need to be handled anymore.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14951>

aco/insert_exec_mask: refactor and simplify get_block_needs()

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14951>

aco/insert_exec_mask: remove ever_again_needs and Exact_Branch

This information is not required anymore.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14951>

aco/insert_exec_mask: remove some unnecessary WQM loop handling code

These workarounds are were necessary to prevent infinite loops
with helper lane registers containing wrong data.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14951>

aco/insert_exec_mask: remove Preserve_WQM flag

If WQM is needed anywhere after discard_if(), it will also
be flagged as WQM. We can rely on that to preserve the WQM mask.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14951>

aco: don't emit WQM for bool_to_scalar_condition

This was only necessary to ensure that the source is computed in WQM
if the current exec mask is in WQM.

Totals from 23170 (17.17% of 134913) affected shaders: (GFX10.3)
VGPRs: 1384464 -> 1383400 (-0.08%); split: -0.08%, +0.01%
SpillSGPRs: 7575 -> 7574 (-0.01%)
CodeSize: 146444752 -> 146317104 (-0.09%); split: -0.13%, +0.04%
MaxWaves: 429870 -> 429868 (-0.00%)
Instrs: 27202586 -> 27170316 (-0.12%); split: -0.17%, +0.05%
Latency: 379488313 -> 379335412 (-0.04%); split: -0.07%, +0.03%
InvThroughput: 69500561 -> 69487704 (-0.02%); split: -0.03%, +0.01%
VClause: 473080 -> 473038 (-0.01%); split: -0.02%, +0.01%
SClause: 1080576 -> 1080571 (-0.00%); split: -0.06%, +0.06%
Copies: 1492865 -> 1504604 (+0.79%); split: -0.40%, +1.19%
Branches: 711295 -> 716849 (+0.78%); split: -0.01%, +0.79%
PreSGPRs: 1331362 -> 1328402 (-0.22%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14951>

aco: don't propagate WQM for p_as_uniform

This was needed, so that in case of active helper lanes,
these contain the correct value. It is now handled implicitly.

Totals from 1004 (0.74% of 134913) affected shaders: (GFX10.3)
CodeSize: 7581020 -> 7580892 (-0.00%); split: -0.00%, +0.00%
Instrs: 1454940 -> 1454908 (-0.00%); split: -0.00%, +0.00%
Latency: 12984953 -> 12984894 (-0.00%); split: -0.00%, +0.00%
InvThroughput: 3173037 -> 3173049 (+0.00%); split: -0.00%, +0.00%
PreSGPRs: 47498 -> 47273 (-0.47%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14951>

aco/insert_exec_mask: stay in WQM while helper lanes are still needed

This patch flags all instructions WQM which don't require
Exact mode, but depend on the exec mask as long as WQM
is needed on any control flow path afterwards.
This will mostly prevent accidental copies of WQM values
within Exact mode, and also makes a lot of other workarounds
unnecessary.

Totals from 17374 (12.88% of 134913) affected shaders: (GFX10.3)
VGPRs: 526952 -> 527384 (+0.08%); split: -0.01%, +0.09%
CodeSize: 33740512 -> 33766636 (+0.08%); split: -0.06%, +0.14%
MaxWaves: 488166 -> 488108 (-0.01%); split: +0.00%, -0.02%
Instrs: 6254240 -> 6260557 (+0.10%); split: -0.08%, +0.18%
Latency: 66497580 -> 66463472 (-0.05%); split: -0.15%, +0.10%
InvThroughput: 13265741 -> 13264036 (-0.01%); split: -0.03%, +0.01%
VClause: 122962 -> 122975 (+0.01%); split: -0.01%, +0.02%
SClause: 334805 -> 334405 (-0.12%); split: -0.51%, +0.39%
Copies: 275728 -> 282341 (+2.40%); split: -0.91%, +3.31%
Branches: 92546 -> 90990 (-1.68%); split: -1.68%, +0.00%
PreSGPRs: 504119 -> 504352 (+0.05%); split: -0.00%, +0.05%

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14951>

Re-indentation after the previous commit

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Eliminate ir_assignment::condition

Reformatting is left for the next commit.

v2: Remove assignments from the contructors. :face_palm:

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Make ir_assignment::condition private

And add get_condition().

This proof that nothing remains that could possibly set ::condition to
anything other than NULL.

v2: Fix bad rebase.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Don't visit rvalues in the condition of an assignment

At this point, this should always be NULL.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Don't lower vector indexing in the condition of an assignment

At this point, this should always be NULL.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Don't split structures in the condition of an assignment

At this point, this should always be NULL.

v2: Fix bad rebase.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Don't split arrays in the condition of an assignment

At this point, this should always be NULL.

v2: Fix bad rebase.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Don't tree graft in the condition of an assignment

At this point, this should always be NULL.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Don't dead-built-in varying eliminate in the condition of an assignment

At this point, this should always be NULL.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Remove unused condition parameter from ir_assignment constructor

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Don't constant-fold the condition of an assignment

At this point, this should always be NULL.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Don't clone assignment conditions

At this point, this should always be NULL.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Eliminate unused conditional assignment constructor

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Remove the ability to read text IR with conditional assignments

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Add ir_assignment constructor that takes just a write mask

The other constructor that takes a write mask and a condition will be
removed shortly.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Lower if to conditional select instead of conditional assignment

Platforms that don't have flow control also don't have anything that
could be written that has a side effect.  It should be safe to implement
these condition writes as

    foo = csel(condition, bar, foo);

This should eliminate the last thing in the GLSL compiler that can
create new conditions on assignments.  Everything else that can store
something in ir_assignment::condition derives it from a pre-existing
condition.

v2: Fix bad rebase.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl/ir_builder: Eliminate unused conditional assignment builders

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>

glsl: Don't try to emit the "linear sequence" in lower_variable_index_to_cond_assign

When there are four or fewer elements left in the array partition, the
strategy changes from a binary search of nested flow control to sequence
of conditional assignments like

    (assign, dest, src[constant_i+0], index == constant_i+0)
    (assign, dest, src[constant_i+1], index == constant_i+1)
    (assign, dest, src[constant_i+2], index == constant_i+2)
    (assign, dest, src[constant_i+3], index == constant_i+3)

or

    (assign, dest[constant_i+0], src, index == constant_i+0)
    (assign, dest[constant_i+1], src, index == constant_i+1)
    (assign, dest[constant_i+2], src, index == constant_i+2)
    (assign, dest[constant_i+3], src, index == constant_i+3)

Realistically, the first case should use ir_triop_csel instead.

The second case will either get turned back into flow control like

    if (index == constant_i+0)
        (assign, dest[constant_i+0], src)

    if (index == constant_i+1)
        (assign, dest[constant_i+1], src)

    if (index == constant_i+2)
        (assign, dest[constant_i+2], src)

    if (index == constant_i+3)
        (assign, dest[constant_i+3], src)

or a sequence of conditional selects like

    (assign, dest[constant_i+0], (csel, index == constant_i+0, src, dest[constant_i+0]))
    (assign, dest[constant_i+1], (csel, index == constant_i+1, src, dest[constant_i+1]))
    (assign, dest[constant_i+2], (csel, index == constant_i+2, src, dest[constant_i+2]))
    (assign, dest[constant_i+3], (csel, index == constant_i+3, src, dest[constant_i+3]))

The former case should continue to use the binary search.  The later
case could be generated from the binary search by other lowering passes.

At the end of the day, conditional assignments don't really help
anything here, so stop using them.

Radeon R430
total instructions in shared programs: 2398683 -> 2398419 (-0.01%)
instructions in affected programs: 5143 -> 4879 (-5.13%)
helped: 9
HURT: 8

total vinst in shared programs: 616292 -> 616010 (-0.05%)
vinst in affected programs: 4467 -> 4185 (-6.31%)
helped: 9
HURT: 8

total sinst in shared programs: 315417 -> 315667 (0.08%)
sinst in affected programs: 2568 -> 2818 (9.74%)
helped: 2
HURT: 15

total flowcontrol in shared programs: 1049 -> 1048 (-0.10%)
flowcontrol in affected programs: 7 -> 6 (-14.29%)
helped: 1
HURT: 0

total presub in shared programs: 47027 -> 47027 (0.00%)
presub in affected programs: 127 -> 127 (0.00%)
helped: 1
HURT: 1

total omod in shared programs: 3618 -> 3615 (-0.08%)
omod in affected programs: 8 -> 5 (-37.50%)
helped: 3
HURT: 0

total temps in shared programs: 450757 -> 451312 (0.12%)
temps in affected programs: 837 -> 1392 (66.31%)
helped: 8
HURT: 6

total consts in shared programs: 1031928 -> 1031920 (<.01%)
consts in affected programs: 1211 -> 1203 (-0.66%)
helped: 6
HURT: 7

The shaders that were hurt for temps... are all lies.  None of those
shaders should have compiled as all 6 had more than 32 temps to begin
with.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14573>