clover: implements clSetContextDestructorCallback

Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7468>

clover: add empty cl 3.0 dispatch entries.

This just fills out the dispatch table blanks.

Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7468>

clover/spirv: hook up spir-v environment for 3.0

For now just use the 1.2 env.

Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7468>

clover/llvm: add 3.0 versioning.

Just adds the 3.0 versioning to the compiler interface.

Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7468>

clover: access 3.0 and deprecated 2.2 API

Adds the api defines to open up deprecaated 2.2 and new 3.0 APIs.

Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7468>

vtn/opencl: add ctz support

ctz is a CL2.0 opcode but 3.0 requires it as well so just add support
for it.

Tested against CTS integer_ops integer_ctz test.

(long line broken up)

Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7468>

CL: update CL headers to 3.0

This just updates the headers from Khronos.

Change the cl_mem initialisers, not sure what totally correct answer is.

Acked-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7468>

docs: Add MESA_pack_invert and ANGLE_pack_reverse_row_order

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3156>

mesa: Implement GL_ANGLE_pack_reverse_row_order

Identical to GL_MESA_pack_invert in effect, just need to check for a
different enum value for GLES vs GL. The spec claims that "OpenGL 1.5 or
OpenGL ES 1.0 are required", but ReadPixels isn't a thing for ES1 so we
only enable it for ES2+.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3156>

mesa: Enable GL_MESA_pack_invert unconditionally

This newly enables the extension for r100 and vieux. As far as I can
tell, that's safe to do. vieux's ReadPixels is just _mesa_readpixels,
which clearly already handles it correctly because the extension is
enabled for classic swrast. r100 has some custom acceleration paths
before falling down to _mesa_readpixels, which winds its way through to
r100_blit, which already has code to handle pack->Invert being set.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3156>

v3dv: Remove unsigned comparison to zero.

subpass_idx is of type uint32_t.

Fix defects reported by Coverity Scan.

Macro compares unsigned to 0 (NO_EFFECT)
unsigned_compare: This greater-than-or-equal-to-zero comparison of
an unsigned value is always true. subpass_idx >= 0U.

Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7325>

nir: Handle ray-tracing intrinsics and storage classes in copy-prop etc.

We need to consider shader calls as potential writes to their payloads.
For other ray-tracing intrinsics, we may not have a shader payload
pointer and have to treat them more like a barrier.  We also need to
ensure that global and SSBO reads/writes aren't propagated across shader
call intrinsics.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6479>

spirv,nir: Add ray-tracing intrinsics

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6479>

nir,spirv: Add support for the ShaderCallKHR scope

It's currently entirely trivial.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6479>

spirv: Implement the new ray-tracing storage classes

The SPV_KHR_ray_tracing extension adds 6 new storage classes which is a
bit on the ridiculous side.  In order to avoid adding that many variable
modes to NIR, we make a few simplifying assumptions:

 1. CallableData and RayPayload data actually lives on the stack
    somewhere, presumably in the caller's stack.  We assume that these
    are no different from global variables and use nir_var_shader_temp
    for them.  We still need a separate storage class for the incoming
    variants but only so we can figure out which one the incoming one
    is and lower it to something useful.

 2. There's no difference between incoming CallableData and RayPaolad
    data.  We can use a single storage class for both.

 3. ShaderRecordBuffer data is just a global memory access.  This lets
    us avoid NIR variables entirely and just fetch the pointer via the
    shader_record_ptr system value and it's accessed using a 64-bit
    global memory pointer.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6479>

nir: Add new variable modes for ray-tracing

If we were desperate to reduce bits, we could probably also use
shader_in/out for hit attributes as they really are an output from
intersection shaders and read-only in any-hit and closest-hit shaders.
However, other passes such as nir_gether_info like to assume that
anything with nir_var_shader_in/out is indexed using vec4 locations for
interface matching.  It's easier to just add a new variable mode.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6479>

nir: Add intrinsics for object to/from world RT sysvals

These are a bit more tricky than most because they're matrix system
values.  We make the intentional choice here to not bother with allowing
indirect addressing of columns for these.  Since they're system values,
they may be magically constructed somehow or come from weird hardware so
it's easier on back-ends to just handle any indirects with bcsel.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6479>

nir/builder: Add a select_from_ssa_def_array helper

This is an operation we have to do already for nir_vector_extract and
I'm about to do something very similar for matrix columns.  Having a
more generic helper is useful.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6479>

spirv,nir: Add support for ray-tracing built-ins

Missing in this commit are NIR intrinsics for the ObjectToWorld and
WorldToObject built-ins.  Those are matrices and so they take a bit more
work and justify a separate commit.  For now, we add the enums and leave
the SYSTEM_VALUE <-> nir_intrinsic conversion commented out.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6479>

spirv: Add support for OpTypeAccelerationStructureKHR

For now, we assume its a 64-bit global pointer.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6479>

spirv: Pass the deref type to storage_class_to_mode for non-forward pointers

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6479>

spirv: Add a guard for OpTypeForwardPointer storage classes

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6479>

spirv: Remove a redundant vtn_fail_if

We already fail in these same cases in vk_desc_type_for_mode.  These
additional assertions are just extra code to update.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6479>

spirv: Add Ray Tracing execution models

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6479>

spirv: Add basic plumbing for ray-tracing capabilities

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6479>

compiler: Add new Vulkan shader stages

This particular ordering makes them conveniently match
VkShaderStageFlagBits, which is a property we already take advantage
of in the previous shader stages.

Abbreviations are based on the ones used in glslangValidator.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6479>

tu: Make sure spirv_to_nir knows we support imageStorageWithoutFormat.

You have to set these flags along with the extension, or you get a bunch
of warnings from spirv-to-nir.

Fixes: e781cc702557 ("tu: Expose shaderStorageImage*WithoutFormat")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7465>

nir/clip_disable: handle 2x vec4 case

some drivers may have pre-lowered gl_ClipDistance to 2x vec4 to match hw
usage, so for those cases we'll be getting deref_var here and then components
will be stored to the deref at some point

fixes mesa/mesa#3480

Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6563>

nir/clip_disable: try for better no-op

we can just check the bits using clip_distance_array_size here to simplify
everything and more easily determine if we need to be running this pass

Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6563>

nir/clip_disable: write 0s instead of undefs for disabled clip planes

this should yield more reliable and ideally even correct results

Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6563>

iris: Move blit scissoring earlier.

There's no need to e.g. prepare_access() if the blit is a noop.

Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7454>

anv: restrict number of subgroups per group

We are limited to 64 threads per dispatched group, regardless of what
num_cs_threads claims, so advertise that limit correctly.

Fixes (on TGL and up):
dEQP-VK.subgroups.size_control.compute.required_subgroup_size_min
and other *.required_subgroup_size_min tests.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7453>

turnip: enable VK_EXT_image_drm_format_modifier

Add missing GetPhysicalDeviceImageFormatProperties2 logic for the extension
and enable it.

Also stop exposing optimal tiling for formats which are linear only, to
simplify dealing with those.

Passes dEQP-VK.drm_format_modifiers.*

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6940>

turnip: don't always fallback to linear for mutable formats

Use VkImageFormatListCreateInfo, and enable VK_KHR_image_format_list to
expose it. (and reorganize linear fallback code a bit)

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6940>

turnip: remove unnecessary/redundant tu_image fields

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6940>

turnip: remove useless tu_image asserts

Validation layer already catches these errors, so don't bother.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6940>

turnip: LAYOUT_PREINITIALIZED is not different for optimal tiling

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6940>

turnip: don't implement CreateImage as two separate functions

Inline tu_image_create into tu_CreateImage.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6940>

aco: Fix format string used when raising validation errors

Validation errors mention the pretty-printed instruction including
operands with the reserved % character, which caused vasprintf to
expect more format arguments than aco provided.

Fixes: c2b1978aa47c ("aco: rework the way various compilation/validation errors are reported")
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7442>

nir/deref: Fix a typo

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3754
Fixes: df51518dc5b "nir/opt_deref: Add a deref mode specialization..."
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7459>

ci: Distribute ADMGPU driver to LAVA as a module

As it needs firmware to probe, and we cannot bundle it within the kernel
image because it is incompatible with the GPL.

Currently we rebind the driver after boot but that's slow and fragile,
as unloads of DRM drivers aren't generally tested.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7420>

ci: Update dEQP skips and fails for Bifrost on G52

Runs are much more stable now with the new kernel, and lots of tests
do pass now.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7420>

ci: Update kernel for LAVA to 5.10-rc2 plus patches

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7420>

util/threaded_context: use driver's buffer alignment for staging transfers

this coincidentally worked because radeonsi has a hardcoded value of 64, but
other drivers do not use this value and then things are subtly broken

Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7452>

v3d: Add GL_ARB_vertex_array_bgra support

This is done by adding support to PIPE_FORMAT_B8G8R8A8_UNORM, and
relying on the R/B swapping for vertex attributes implemented in the
compiler.

v2:
 - Simplify the loop (Iago)

v3:
 - Assert before derreferencing variable (Iago).

Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3078
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7196>

v3dv: mark the right bit to swap R/B vertex attributes

Now that the R/B swap mask for vertex attributes handles all the
attributes, ensure the right generic vertex attribute is marked.

Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7196>

v3d/compiler: extend swapping R/B support to all vertex attributes

So far the support for R/B swapping in vertex attributes were for the
generic attributes.

But there are cases like glSecondaryColorPointer() supporting BGRA
formats that require the R/B swapping to be also allowed in the
non-generic vertex attributes (in this case, in the COLOR1 attribute).

v2:
 - Don't split line (Iago)

Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7196>

intel/tools: add missing new lines to few remaining fail_if users

Signed-off-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7449>

intel/tools: refactor logging to be easier to follow by static analyzers

Refactor out the part of fail_if function that never returns into
NORETURN function and put the condition check outside.

Addresses many false positive warnings by Coverity.

Signed-off-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7449>

intel/tools: handle some failures

Addresses "Dereference null return value" issues reported by Coverity.

Signed-off-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7449>

anv: remove dead code from anv_create_cmd_buffer

pool can't be NULL at this point, because it was already
dereferenced earlier.

Addresses "Dereference before null check" issue reported by Coverity.

Signed-off-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7449>

intel/tools: allow --color option to be used without arg

There's already code handling that case and help text also says
it's possible.

Found, because Coverity complained about optarg NULL check,
suggesting optarg can be NULL for other options, where it's not
possible. IOW, false positive lead me to finding an unrelated issue.

Signed-off-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7449>

v3dv: expose more features

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7456>

broadcom/compiler: Handle non-SSA destinations for tex instructions

The NIR that is given to the VIR compiler is not in SSA form, and so
the v3d*_vir_emit_tex() functions must be able to handle both SSA and
register destinations.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7318>

vc4: use intmax_t for formatted output of timespec members

32bit architectures which have 64bit time_t does not fit the assumption
of time_t being same as system long int

Fixes
error: format specifies type 'long' but the argument has type 'time_t' (aka 'long long') [-Werror,-Wformat]
                        time.tv_sec);
                        ^~~~~~~~~~~

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2966>

amd/addrlib: Add missing va_end.

Fix defect reported by Coverity Scan.

Missing varargs init or cleanup (VARARGS)
missing_va_end: va_end was not called for debugPrintInput.ap.

Fixes: 69ea473eeb91 ("amd/addrlib: update to the latest version")
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7299>

loader: Print dlerror() output in the failure message

This just captures the last failure, but that's better than nothing.

Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7429>

intel/fs: Implement nir_intrinsic_{load,store}_shared_block_intel

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7448>

intel/fs: Implement nir_intrinsic_{load,store}_ssbo_block_intel

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7448>

intel/fs: Add surface OWORD BLOCK opcodes

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7448>

intel/fs: Implement nir_intrinsic_{load,store}_global_block_intel

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7448>

intel/fs: Add A64 OWORD BLOCK opcodes

Based on a patch for OWORD BLOCK READ from Jason Ekstrand.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7448>

spirv: Implement SpvCapabilitySubgroupBufferBlockIOINTEL

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7448>

nir: Add nir_intrinsic_{load,store}_deref_block_intel

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7448>

spirv: Implement SpvCapabilitySubgroupShuffleINTEL from SPV_INTEL_subgroups

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7448>

iris: Flush dmabufs during context flushes

Currently, every modifier that uses CCS also lacks support for
fast-clears. On gen9+, dmabufs may gain fast-cleared blocks through
clear calls. On gen12, fast-clearing can occur during any rendering
operation. Mark when dmabufs gain fast-cleared blocks and flush them
during a context flush operation.

Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3425
Tested-by: Simon Ser <contact@emersion.fr>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7384>

iris: Fix fast-clears of swizzled alpha formats

Resources with alpha formats that are mapped to R are fast-cleared with
the wrong clear color.

When such resources with are cleared via iris_clear_texture,
isl_color_value_unpack places channel data in the R channel.
convert_fast_clear_color then overwrites the channel with 0.

To avoid zeroing the clear color, move convert_fast_clear_color to the
other callers of clear_color: iris_clear and iris_clear_render_target.

Enables iris to pass the "A" case of the fcc-clear-tex piglit test.

v2. Rename convert_fast_clear_color to convert_clear_color. (Ken)

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3670
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7345>

iris: Fix SINT assert in convert_fast_clear_color

Don't assert that the size of every channel is greater than zero. This
assert doesn't work for integer formats with less than 4 channels.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7345>

iris: Fix fast-clears of swizzled LA formats

Resources with luminance alpha formats that are mapped to RG are
fast-cleared with the wrong clear color.

When such resources with are cleared via iris_clear_texture,
isl_color_value_unpack places channel data in the R and G channels.
convert_fast_clear_color then overwrites the G channel with R.

Delete the clear color override that's specific to luminance alpha
formats.

Enables iris to pass the "LA" case of the fcc-clear-tex piglit test.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7345>

iris: fix source/destination layers for 3D blits

See commit ea326912575fad09af59486ad62d126c4ea0ede7 for the
corresponding fix in anv.

Fixes Piglit's fbo-generatemipmap-3d.

Cc: mesa-stable
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7321>

st/nir: Drop setting interp mode on system values in builtins.

It's initialized to INTERP_MODE_NONE on creation, which makes more sense
for sysvals than FLAT, and is also the interp mode that GLSL IR sets up
for sysvals.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7320>

st/nir: Fix the st->pbo.use_gs case.

This case hadn't been ported to NIR before, and I missed that when
removing the TGSI path and replacing it with NIR -> NTT for TGSI drivers.
This caused breakage in nv50 on piglit's pbo-teximage.

In the process, the !use_gs gets its layer output fixed to be an int
instead of a vec4, which I suspect would fix validation in that path.

Fixes: 57effa342b75 ("st/mesa: Drop the TGSI paths for PBOs and use nir-to-tgsi if needed.")
Closes: #3680
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7320>

pan/bi: Correctly calculate render target index

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7446>

pan/bi: Lower depth/stencil stores

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7446>

pan/bi: Emit +ZS_EMIT as needed

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7446>

pan/bi: Stub handling for nir_intrinsic_store_combined_output_pan

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7446>

pan/bi: Factor out bi_emit_blend

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7446>

pan/bi: Factor out bi_emit_atest

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7446>

pan/bi: Infer z/stencil flags from sources passed

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7446>

pan/bi: Add +ZS_EMIT instruction to IR

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7446>

panfrost: Deduplicate shader properties

Between Midgard and Bifrost.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7446>

panfrost: Pass through src_type

Needed since Bifrost blends are typed well.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7446>

pan/mdg: Move writeout lowering to common panfrost

These will be used in the Bifrost compiler, albeit for a slightly
different purpose.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7446>

pan/mdg: Deduplicate nir_find_variable_with_driver_location

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7446>

nir: Add SRC_TYPE to store_combined_output_pan

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7446>

v3dv: add a v3dv_bo_init helper

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7440>

aco/ra: Fix counting of subdword variables in get_reg_create_vector

The loop variable "k" shadowed another variable in the outer scope, so
this loop had no actual effect.

Fixes: 52cc1f8237d ("aco: improve p_create_vector RA for sub-dword operands")
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7427>

aco: implement 8/16-bit instructions which can be trivially widened

When nir_lower_bit_size becomes more capable, we might want to revert some
of this.

fossil-db (parallel-rdp, Navi):
Totals from 217 (31.77% of 683) affected shaders:
SGPRs: 11320 -> 10200 (-9.89%)
VGPRs: 7156 -> 7364 (+2.91%)
CodeSize: 1453948 -> 1430136 (-1.64%); split: -1.66%, +0.02%
Instrs: 258530 -> 254840 (-1.43%); split: -1.44%, +0.01%
Cycles: 37334360 -> 37247936 (-0.23%); split: -0.26%, +0.03%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4791>

aco: implement some 16-bit arithmetic instead of lowering

fossil-db (parallel-rdp, Navi):
Totals from 210 (30.75% of 683) affected shaders:
SGPRs: 9704 -> 10248 (+5.61%)
VGPRs: 5884 -> 5368 (-8.77%)
CodeSize: 1155564 -> 1098752 (-4.92%)
Instrs: 199927 -> 189940 (-5.00%)
Cycles: 20438392 -> 19860124 (-2.83%)

v2: use divergence analysis to determine which instructions to lower.

Co-Authored-by: Daniel Schürmann <daniel@schuermann.dev>
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4791>

radv: rework nir_lower_bit_size callback and run DA on GFX8+

No fossil-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4791>

radv: do nir_lower_bit_size after algebraic optimizations

There are too many algebraic optimizations to be certain that one of them
couldn't create instructions which need lowering. It also creates better
code for some reason.

fossil-db (parallel-rdp, Navi):
Totals from 217 (31.77% of 683) affected shaders:
VGPRs: 7716 -> 7672 (-0.57%)
CodeSize: 1516152 -> 1510688 (-0.36%); split: -0.38%, +0.02%
MaxWaves: 3964 -> 3982 (+0.45%)
Instrs: 269445 -> 268508 (-0.35%); split: -0.36%, +0.02%
Cycles: 37963416 -> 37912592 (-0.13%); split: -0.15%, +0.01%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4791>

radv: move a few passes to after load/store vectorization

load/store vectorization can create 8/16-bit alu to do packing/unpacking,
which would make shader_info::bit_sizes_used out of date.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4791>

nir/lower_bit_size: optimize upcast of b2i8/b2i16

This also seems to be done by nir_opt_algebraic, but RADV will be moving
nir_lower_bit_size() to after that (so it doesn't create unsupported
8/16-bit instructions) and it doesn't seem worth creating a new pass just
for this simple optimization.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4791>

nir: add shader_info::bit_sizes_used

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4791>

va: support VA_RT_FORMAT_PROTECTED

Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7006>

va/picture: make sure destination buffer is protected if needed

Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7006>

frontends/va: Added protected playback support for VP9

Add VP9 header handling in slice data buffer.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7006>

radeon/vcn: program drm message buffer

Add a function to handle drm message buffer using input decryption parameters.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7006>

radeon/vcn: delay dec->ctx and dec->dpb allocation

This will allow to allocate them as encrypted if needed.

Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7006>

radeon: add si_vid_create_tmz_buffer helper

Same code as si_vid_create_buffer except that the buffer is using TMZ.

Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7006>