frontends/omx/av1: enable AV1 OMX Bellagio support

By adding name, role and profile to OMX decoder

Signed-off-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7596>

frontends/omx/av1: add AV1 tasks management

This handles different task queues based on OMX bellagio

Signed-off-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7596>

frontends/omx/av1: add AV1 OBU header parsers

Based on AV1 spec: https://aomediacodec.github.io/av1-spec

Signed-off-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7596>

frontends/omx/bellagio: add AV1 initial support to omx dec

Also add bitstream base parser based on AV1 spec

Signed-off-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7596>

radeonsi: cap AV1 support to SIENNA CICHLID

since AV1 is avaiable from VCN3

Signed-off-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7596>

radeonsi: cap AV1 codec configuration

This defines the support size to 8K and no interlaced buffer

Signed-off-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7596>

radeon/vcn: fill up the probs buffer

with the segmetation data and mask

Signed-off-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7596>

radeon/vcn: get AV1 message buffer

This passes parameters to firmware based on interface by frames.

Signed-off-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7596>

radeon/vcn: fill up the context buffer

with the mode tables, mv probs tables, and coef probs tables

Signed-off-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7596>

radeon/vcn: add AV1 context buffer

The context buffer will be filled up with the tables, and used
by firmware for decode

Signed-off-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7596>

radeon/vcn: add AV1 default tables for the context

These tables will be used by firmware

Signed-off-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7596>

radeon/vcn: add AV1 dpb buffer size

Just use the default size for now

Signed-off-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7596>

radeon/vcn: add AV1 support to the decoder

This adds AV1 stream type to the AV1 decoder

Signed-off-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7596>

radeon/vcn: add AV1 codec driver firmware interfaces

This collects parameters that firmware and hardware require for
the AV1 codec.

Signed-off-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7596>

vl: add AV1 codec picture support

This adds AV1 video profile, format and picture description to vl.

Signed-off-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7596>

anv/image: Define anv_image_get_aux_addr (v3)

Simple refactor. No intended change in behavior.

Replace each derivation of aux address with anv_image_get_aux_addr().

The function will soon do more in support of
VK_EXT_image_drm_format_modifier, where the image bo and aux bo may be
disjoint.

v2:
  - Replace param 'aspect' with 'plane'.
v3:
  - Workaround for stencil ccs. If no aux surface, then return
    ANV_NULL_ADDRESS.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> (v2)
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> (v3)

anv/image: Check surface offsets after adding each surface

Pre-patch, we checked the offsets once per aspect after adding all
surfaces for the aspect. The additional checks will make it easier to
diagnose layout bugs.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

anv/image: Rewrite check_surfaces() [v2]

Pure refactor. No intended change in behavior.

This makes the code infinitely easier to understand. And it uncovers
a potential bug (marked with XXX comment).

v2: Fix narrowing conversions on 32-bit arch. s/size_t/uintmax_t/.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> (v1)
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> (v2)

anv/image: Further split add_*_surface funcs (v2)

Months ago, make_surface() added *all* surfaces required for the given
aspect. It was a monster monolithic function, and difficult to reason
about its correctness. In commit c652ff8c (2020-03-06), I split the code
for aux surfaces into its own function, add_aux_surface_if_supported().

This patch continues the splitting, therefore making bugs easier to
identify.

Code changes:
   - Move the code that adds the shadow surface from make_surface() to
     a new function add_shadow_surface(), called from
     add_all_surfaces().
   - Move the call to add_aux_surface_if_supported() from make_surface()
     to add_all_surfaces().
   - To preserve correctness of the assertions on image layout in
     make_surface(), move them to the loop in add_all_surfaces() after
     all the aspect's surfaces have been added.
   - Rename make_surface() to add_primary_surface() because now that's
     what it does.

Pure refactor, no intended change in behavior.

v2:
  - Rebase onto "anv: Fix isl_surf_usage_flags for stencil images".
  - Sanitize the image's extent earlier.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> (v2)
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> (v2)

anv/image: Define add_all_surfaces()

This deduplicates the loops in anv_image_create() and
resolve_ahw_image().

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

anv/image: Add more asserts to choose_isl_tiling_flags

Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

anv/image: Move some DRM code in anv_get_image_format_properties()

In anv_get_image_format_properties(), the special-case code for
VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT is tiny. It is mostly a detached
'case' in the 'switch' block for VkImageType. So move the special-case
code to immediately follow the 'switch' block.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

anv/image: Emit error message for non-2D DRM images

In vkGetPhysicalDeviceImageFormatProperties.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

anv/image: Drop redundant rejection of YCbCr formats with modifiers

The check in anv_get_image_format_properties() is already handled in
anv_get_image_format_features().

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

anv/image: Respect VkImageFormatListCreateInfo for VkImageFormatProperties (v2)

When filling VkImageFormatProperties, anv_get_image_format_properties()
checks the requested VkImageUsageFlags and VkImageCreateFlags against
the VkFormatFeatureFlags available to the queried VkFormat. However, we
neglected to consider if any formats given in
VkImageFormatListCreateInfo
further restricted the available VkFormatFeatureFlags.

The image view formats are more likely to introduce additional
restrictions when DRM format modifiers are present.

v2:
  - Do not drop anv_formats_ccs_e_compatible().

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> (v2)
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> (v2)

anv/image: Fail earlier in anv_get_image_format_properties

If anv_get_image_format_features reports that the inputs are
unsupported, fail immediately.

Without the early fail, I have less confidence in the function's
correctness when a DRM format modifier is present.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

anv/image: Minor refactor of VkImageFormatProperties::sampleCounts

The code in anv_get_image_format_properties() that set sampleCounts
appears correct, but weirdly inconsistent. Clean the code to
consistently set sampleCounts in the same location as
maxExtent/maxMipLevels/maxArraySize.

Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

anv/image: Rename get_wsi_format_modifier_properties_list()

Rename it to get_drm_format_modifier_properties_list() because it is now
independent of WSI.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

anv/image: Fix VkExternalMemoryProperties for images (v5)

In vkGetPhysicalDeviceImageFormatProperties2, we advertised support for
VK_IMAGE_TILING_LINEAR and VK_IMAGE_TILING_OPTIMAL for all memory
handles.

However, when importing or exporting an image, there must exist a method
that enables the app and driver to agree on the image's memory layout.
If no method exists, then we should reject image creation.

v2:
  - Reduce copy-paste for Lionel.
v3:
  - Treat tiling LINEAR and DRM_FORMAT_MODIFIER as identical when
    determing compatible memory handles.
  - Improve comments.
v4:
  - Remove DMA_BUF from opaque_fd_only_props.
v5:
  - Minor changes to code style for `if`. (for jekstrand)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> (v4)
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> (v4)

anv/image: Delete the list of modifier-compatible formats

The code asserted that we supported no more than 4 formats with
modifiers: /VK_FORMAT_B8G8R8(A8)?_(SRGB|UNORM)/.
Strangely, 2 of the 4 were non-power-of-two formats, which were rejected
elsewhere.

The assertion's comment suggested that we use a hard-coded list of
formats because the driver was not yet able to determine if a given
format was compatible with a given modifier.  Therefore, the list only
contained formats that were compatible with *all* modifiers. That code
deficiency no longer exists: anv_get_image_format_features() can check
format/modifier compatibility.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

anv/image: Refactor iteration over modifiers

Refactor in get_wsi_format_modifier_properties_list().

Instead of iterating over a function-local hard-coded list, iterate over
all modifiers in isl_drm.c.

This will improve agreement in behavior between
VkDrmFormatModifierPropertiesListEXT
VkPhysicalDeviceImageDrmFormatModifierInfoEXT.

The future disagreement this patch attempts to prevent is the
combination of:
    a. VkDrmFormatModifierPropertiesListEXT neglects to return a valid
       modifier because its hard-coded list of modifiers drifts
       out-of-sync with hard-coded lists elsewhere in the code. (Already
       today, the list in get_wsi_format_modifier_properties_list() does
       not match the list in isl_drm.c; though, this has produced no bug
       yet).
    b. vkGetPhysicalDeviceImageFormatProperties2 accepts, via
       VkPhysicalDeviceImageDrmFormatModifierInfoEXT, the modifier
       overlooked in (a), because it does not use the same hard-coded
       list in get_wsi_format_modifier_properties_list(). (Recall that
       the spec requires vkGetPhysicalDeviceImageFormatProperties2 to
       correctly accept/reject any int that the app provides, even when
       the int is an invalid modifier).
    c. The Bug. The driver told the app in (b) that it can legally
       create an image with format+modifier, but the app cannot query
       the VkFormatFeatureFlags of the format+modifier due to (a).

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

isl: Make public the list of modifiers

This allows Vulkan and GL to iterate over the full list of modifiers
instead of hard-coding in various places the "same" list as isl.

(Anvil's list has already diverged from isl's list. It omits Gen12
modifiers).

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

anv/image: Fill drmFormatModifierTilingFeatures (v2)

Fill VkDrmFormatModifierPropertiesEXT::drmFormatModifierTilingFeatures
with anv_get_image_format_features().

anv_formats.c:get_wsi_format_modifier_properties_list() incorrectly left
it uninitialized.

v2: Increment drmFormatModifierPlaneCount if modifier support aux.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> (v2)
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> (v2)

anv/image: Teach anv_get_image_format_features() about modifiers (v3)

Because anv_get_image_format_features() now understands modifiers, also
relocate most of the modifier compatibility checks from
anv_get_format_plane() into anv_get_image_format_features() in order to
avoid duplication.

The new signature forces some code movement in
anv_get_image_format_properties().

v2:
  - Reject VK_FORMAT_B4G4R4A4_UNORM_PACK16 with modifiers on HSW.
v3:
  - Revert the v2 change.
  - Query isl_format_layout instead of pipe_format. (for jekstrand)
  - Drop misguided comments. (for jekstrand)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> (v2)
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> (v3)

isl: Add isl_format_layout::uniform_channel_type

If each format channel has the same base type (such unorm), then that
is the format's "uniform channel type".

Calculating the field at buildtime is probably better than looping over
all channels at runtime each time we wish to query it.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

anv/image: Use isl_drm_modifier_get_score()

It replaces anv_drm_format_mod_score().

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

isl: Define isl_drm_modifier_get_score() [v3]

Return the modifier's score, which indicates the driver's preference for the
modifier relative to others. A higher score is better. Zero means
unsupported.

Intended to assist selection of a modifier from an externally provided list,
such as VkImageDrmFormatModifierListCreateInfoEXT.

v2:
  - Rename anv_drm_format_mod_score to isl_drm_modifier_get_score.
  - Squash all incremental changes to anv_drm_format_mod_score.
v3:
  - Drop redundant 'unlikely'. (for nchery)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> (v2)
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> (v3)

anv/image: Fix isl_surf_usage_flags for stencil images

Respect VkImageStencilUsageCreateInfoEXT.

CC: mesa-stable@lists.freedesktop.org
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

anv/image: Check DISJOINT in vkGetPhysicalDeviceImageFormatProperties2 (v2)

The code did not return error when VK_IMAGE_CREATE_DISJOINT_BIT was
incompatible with the other input params.

If the Vulkan spec forbids a set of input params for vkCreateImage,
but permits them for vkGetPhysicalDeviceImageFormatProperties2,
then vkGetPhysicalDeviceImageFormatProperties2 must reject those input
params with failure.

- v2: Clearer commit message.

CC: <mesa-stable@lists.freedesktop.org>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> (v2)
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>

zink: more accurately track supported blits

We don't care if blits need to respect render-conditions if there's no
active one. So let's hit the potentially faster native blit-paths
instead.

Fixes: 5743fa6e709 ("zink: enable conditional rendering if available")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3792
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7606>

zink: always insert barriers for general-layout

We need to always have barriers between individual uses of resources
in the general-layout, because otherwise a write-cache might not be
flushed before the resource is used.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7652>

zink: mark general layout as transfer-read/write

The general layout can be used for transfers, so we need to make sure
the vulkan driver knows. This will help the driver know when it needs to
flush caches.

While we're at it, also add shader-read, which is another access we use.
We should stop using that one ASAP, but for now this seems like the
right thing to do.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7652>

v3dv: rename playout and dslayout fields to use underscores.

Following a suggestion from Alejandro, since playout is a word on its own
and can be confusing. It also makes it more consistent with other
variable names that use an underscore.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7651>

v3dv: blit shader clean-ups

This avoids redundant per-layer operations that are the same across
layers or that only need to do once. Namely:

- The sampler for the blit source is the same for all layers.
- The decision about whether we need to load TLB contents or not only
  needs to be done once.
- Some command buffer state such as the pipeline, the viewport and the
  scissor is the same for all layers and should only be bound once.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7651>

v3dv: initialize pipeline layouts for meta operations at driver initialization

This removes the need to lock just to check if we have created them
due to the lazy allocation strategy we had in place.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7651>

v3dv: add a buffer to image copy path using a texel buffer

This is much faster than the blit fallback (which requires to upload
the linear buffer to a tiled image) and the CPU path.

A simple stress test involving 100 buffer to image copies of a
single layer image with 10 mipmap levels provides the following
results:

Path           | Recording Time | Execution Time |
-------------------------------------------------|
Texel Buffer   |     2.954s      |     0.137s    |
-------------------------------------------------|
Blit           |    10.732s      |     0.148s    |
-------------------------------------------------|
CPU            |     0.002s      |     1.453s    |
-------------------------------------------------|

So generally speaking, this texel buffer copy path is the fastest
of the paths that can do partial copies, however, the CPU path might
provide better results in cases where command buffer recording is
important to overall performance. This is probably the reason why
the CPU path seems to provide slightly better results for vkQuake2.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7651>

v3dv: fix width for buffer view texture state

This is in units of texels, not bytes.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7651>

radv: don't subtract max_verts_per_prim from hw_max_esverts on gfx10.3

Ported from RadeonSI.

GFX10.3 does it properly.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7566>

radv: don't count unusable vertices to the NGG LDS size

Ported from RadeonSI.

To get optimal LDS usage since the previous change.

Cc: 20.2
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7566>

radv: fix applying the NGG minimum vertex count requirement

Ported from RadeonSI.

The restriction was applied too late.

Cc: 20.2
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7566>

radv: do VGT_FLUSH when switching NGG -> legacy on Sienna Cichlid

Ported from RadeonSI.

Cc: 20.2
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7566>

mesa/gallium: add MESA_MAP_ONCE / PIPE_MAP_ONCE

If set, this bit tells the driver that the buffer will only be
mapped once.

radeonsi uses it to disable its "never unmap buffers" optimisations.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3660
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7428>

radv: Fix radv_queue_init failure handling

Do not destroy pending_mutex or thread_mutex if uninitialized.

Do not use or destroy thread_cond if uninitialized.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7599>

panfrost: Don't expose fp16 support on Bifrost unless explicitly requested

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7636>

pan/bi: Stop extracting the immediate attribute index from src0

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7636>

pan/bi: Add a varying_index field to bi_texture

So we can get rid of the offset adjusment done in pack_variant()

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7636>

pan/bi: Fix LD_VAR with non-constant index

src0 and src1 were mixed leading to invalid varying indices. In order to
fix that properly, we first extend load_vary to pass the immediate index
through a dedicated field and add a special boolean. This way, we don't
have to make sure src0 always contains the index, and can instead match
the src numbering defined in ISA.xml.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7636>

pan/bi: Stop passing special varying names through src0

It's just clearer to have dedicated fields encoding the fact that the
LD_VAR should be SPECIAL, and another field storing the special var id.

With this change, the source index know matches the ISA.xml definition.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7636>

pan/bi: Pass LD_VAR update mode explicitly

Let the compiler pass the update mode instead of inferring from the
constant value.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7636>

pan/bi: Move LD_VAR packing out of bi_pack_add()

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7636>

pan/bi: Set roundmode to RTZ for f2u operations

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7636>

pan/bi: Let the GPU pick the right format based on the varying descriptor

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7636>

pan/bi: Support automatic register format

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7636>

pan/bi: Support txs operations

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7636>

pan/bi: Don't use TEXS for tex operations with a src that's not lod or coord

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7636>

pan/mdg: Try demoting uniforms instead of spilling to TLS

mir_estimate_pressure often underestimates the register pressure,
letting too many registers be used for uniforms, causing RA to fail.

Mitigate this by demoting some uniforms back to explicit loads to free
up work registers if register allocation fails.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7616>

turnip: Close sync_fd only if it is a valid file descriptor.

Fix defects reported by Coverity Scan.

Argument cannot be negative (NEGATIVE_RETURNS)
negative_returns: sync_fd is passed to a parameter that cannot be negative.

Fixes: cec0bc73e55 ("turnip: rework fences to use syncobjs")
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7647>

clover/spirv: Add missing break for SpvOpExecutionMode case.

Fix defect reported by Coverity Scan.

Missing break in switch (MISSING_BREAK)
unterminated_case: The case for value SpvOpExecutionMode is not
terminated by a 'break' statement.

Fixes: ee5b46fcfdb ("clover/spirv: support CL_KERNEL_COMPILE_WORK_GROUP_SIZE")
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7519>

frontends/va: Fix *num_entrypoints check.

Fix defect reported by Coverity Scan.

Dereference before null check (REVERSE_INULL)
check_after_deref: Null-checking num_entrypoints suggests that it
may be null, but it has already been dereferenced on all paths
leading to the check.

Fixes: 5bcaa1b9e970 ("st/va: add encode entrypoint v2")
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7360>

freedreno: Disable PIPE_CAP_PREFER_IMM_ARRAYS_AS_CONSTBUF.

We now have NIR opt_large_constants support in place, so we can flip the
switch and get better optimization before lowering to a constant buffer,
but also avoid having constant data mixed in with the shader's uniforms,
which should lower CPU overhead on affected shaders.

Only a few shaders are affected (<.01% impact across shader-db), but for
those the impact is pretty big:

instructions in affected programs: 748 -> 639 (-14.57%)
nops in affected programs: 364 -> 284 (-21.98%)
non-nops in affected programs: 384 -> 355 (-7.55%)
mov in affected programs: 47 -> 27 (-42.55%)
cov in affected programs: 9 -> 6 (-33.33%)
dwords in affected programs: 932 -> 836 (-10.30%)
full in affected programs: 13 -> 14 (7.69%)
constlen in affected programs: 140 -> 64 (-54.29%)
(ss) in affected programs: 14 -> 15 (7.14%)

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5810>

freedreno+turnip: Upload large shader constants as a UBO.

Right now if the shader indirects on some large constant array, we see NIR
load_consts (usually from the const file) of its contents into general
registers, then indirection on the GPRs.  This often results in register
allocation failures, as it's easy to go beyond the ~256 dwords of
registers per invocation.

By moving the large constants to a UBO, we can load an arbitrary number of
them.  They also can be theoretically moved to the constant reg file (~2k
dwords), though you're unlikely to hit this path without an indirect load
on your large constant, and we don't yet let UBO indirect loads get moved
to constant regs.

This possibly won't work out right if we have 16-bit load_constants, but
without other MRs in flight we won't see 16-bit temps to be lowered to
this.

This allows 2 kerbal-space-program shaders to compile that previously
would fail, and fixes the new dEQP-VK and -GLES2 tests I wrote that
dynamically index a 40-element temporary array of float/vec2/vec3/vec4
with constant element initializers.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2789
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5810>

freedreno/ir3: Fix incorrect optimization of usage of 16-bit constbuf vals.

If you're loading a 32b word from the const file and doing a cov.u32u16
split to two 16bit values, we can't turn that into a reference of a 16-bit
float value directly from the constbuf, because the
CONSTANT_DEMOTION_ENABLE results in a f2f16 operation on the 32-bit value
that we didn't want.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5810>

freedreno/ir3: Switch emit_const_ptrs() to take BOs instead of prscs.

Just indirect in the caller, which means that I'll be able to pass a
non-resource BO in the large-constants case.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5810>

freedreno/ir3: Include at least 4 NOPs so that cffdump doesn't disasm junk.

cffdump looks at the following 4 instructions to decide if the shader has
*really* ended, so if we pack data after that (such as turnip's next
stage's shader), it might decode instructions that aren't really part of
the shader.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5810>

nir: Add a size_align helper function for aligning elements to 16 bytes.

This is useful for freedreno's intrinsic opt_large_constant lowering,
where we want arrays and struct elements aligned to 16 to avoid generating
lots of extra instructions to extract from the right component.

Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5810>

freedreno: Fix leak of shader binary on disk cache hits.

It's supposed to be ralloced -- there's not even a shader variant destroy
function for freeing, just ralloc_free() on the ir3_shader_variant or the
parent ir3_shader when you're done!

Fixes: f97acb4bb4b1 ("freedreno/ir3: disk-cache support")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5810>

intel/fs: Add assert on the brw_STAGE_prog_data downcasts

Motivation is to detect earlier certain bugs that can occur when
missing a check for the stage before using the downcast.

Reviewed-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7540>

spirv/cl: add enqueued workgroup size.

Unless the non uniform work group extension is supported, this
just aliases workgroupsize, so just do that for now.

Fixes:
CL CTS basic enqueued_local_size

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7642>

clover/image: handle MEM_KERNEL_READ_AND_WRITE flag.

fixes CTS 3.0 test_computeinfo

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7642>

clover: add CL 3.0 event/queue queries

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7642>

clover: add 3.0 program properties

the real IL code will rewrite this

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7642>

clover: add device/platform info for CL 3.0

This just adds all the dummy 2.x/3.0 device and platform
info queries that return fixed not supported values.

As these are supported they will have to be migrated into the core.

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7642>

clover: add cl 3.0 SVM invalid support

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7642>

clover: add all CL 3.0 API with invalid functions

These CL 2.x APIs are all part of CL3.0 but have to return
specific values to show they aren't supported.

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7642>

clover: add support command queue properties

Fixes api queue_properties_queries

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7642>

clover: handle memory object properties properly.

This adds proper support for the buffer/image property APIs.

Fixes:
CL CTS api buffer_properties_queries

v1.1: use a helper for properties parsing.

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7642>

etnaviv: drop nir_print_shader(..) call

It makes no sense to print the shader in the middle of some NIR passes.
Instead someone could use NIR_PRINT=1 and call it a day. Also there
is a nir_print_shader(..) before calling emit_shader(..).

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7625>

etnaviv: fix disabling of INT filter for real

Missing a copy of the pipe_sampler_state into the etna_sampler_state object
lead to the texture_use_int_filter() to always see a max_anisotropy of 0, so
the INT filter wasn't disabled when necessary. Also state emission should
never change the state objects, as this might also lead to stale information
being kept around the in the state object.

Fixes: 89a41dae7702 (etnaviv: do not use int filter when
                     anisotropic filtering is used)

Cc: <mesa-stable@lists.freedesktop.org>
Signed-off-by: Lucas Stach <l.stach@pengutronix.de>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7638>

docs/features: document d3d12 features

Now that the d3d12 driver has landed, we should document it's features,
like we do with other drivers.

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7583>

aco: don't use SMEM for SSBO stores

fossil-db (Navi):
Totals from 70 (0.05% of 138791) affected shaders:
SGPRs: 2324 -> 2097 (-9.77%)
VGPRs: 1344 -> 1480 (+10.12%)
CodeSize: 157872 -> 154836 (-1.92%); split: -1.93%, +0.01%
MaxWaves: 1288 -> 1260 (-2.17%)
Instrs: 29730 -> 29108 (-2.09%); split: -2.13%, +0.04%
Cycles: 394944 -> 391280 (-0.93%); split: -0.94%, +0.01%
VMEM: 5288 -> 5695 (+7.70%); split: +11.97%, -4.27%
SMEM: 2680 -> 2444 (-8.81%); split: +1.34%, -10.15%
VClause: 291 -> 502 (+72.51%)
SClause: 1176 -> 918 (-21.94%)
Copies: 3549 -> 3517 (-0.90%); split: -1.80%, +0.90%
Branches: 1230 -> 1228 (-0.16%)
PreSGPRs: 1675 -> 1491 (-10.99%)
PreVGPRs: 1101 -> 1223 (+11.08%)

Totals from 70 (0.05% of 139517) affected shaders (RAVEN):
SGPRs: 2368 -> 2121 (-10.43%)
VGPRs: 1344 -> 1480 (+10.12%)
CodeSize: 156664 -> 153252 (-2.18%)
MaxWaves: 636 -> 622 (-2.20%)
Instrs: 29968 -> 29226 (-2.48%)
Cycles: 398284 -> 393492 (-1.20%)
VMEM: 5544 -> 5930 (+6.96%); split: +11.72%, -4.76%
SMEM: 2752 -> 2502 (-9.08%); split: +1.20%, -10.28%
VClause: 292 -> 504 (+72.60%)
SClause: 1236 -> 940 (-23.95%)
Copies: 3907 -> 3852 (-1.41%); split: -2.20%, +0.79%
Branches: 1230 -> 1228 (-0.16%)
PreSGPRs: 1671 -> 1487 (-11.01%)
PreVGPRs: 1102 -> 1225 (+11.16%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6143>

mesa/main: add missing include in glformats.h

This header uses uint32_t without including stdint.h. This worked fine
by accident until a new c-source started including it.

Fixes: 1bf539b3a21 ("mesa: Clamp some depth values in glClearBufferfv")
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7637>

aco/tests: add output modifier tests

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7605>

aco: fix fp16 *0.5 omod

We were testing for -0.5 instead.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Fixes: 1210e0bd620 ("aco: create 16-bit input and output modifiers")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7605>

aco: disable omod if the sign of zeros should be preserved

The RDNA ISA doc says that omod doesn't preserve -0.0 in 6.2.2. LLVM
appears to always disable omod in this situation, but clamp is unaffected.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Fixes: df645fa369d ("aco: implement VK_KHR_shader_float_controls")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7605>

Revert "ci: Temporarily disable jobs on the Collabora lab"

Lab is back online now.

This reverts commit 902ac3d7c5ab75f2ff31bf23d1d5add7b5c377ed.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7630>

nir: don't consider txf_ms_mcs a query instruction

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6172>

util: fix unknown pragma warning on msvc

MSVC has no idea about these pragmas, and spews warnings about them,
making it hard to spot real problems. So let's only use these macros on
GCC.

Fixes: 2ec290cd92a ("util: Fix/silence variable shadowing warnings")
Reviewed-by: Tony Wasserka <tony.wasserka@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7633>

aco: remove v_{add,sub,subrev}_u32 on GFX8

These opcodes are never used and they always write the carry-out
according to the GCN3 ISA documentation.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7569>

microsoft/compiler: Fix reference to renamed intrinsic getter

Fixes: b9c61379 ("microsoft/compiler: translate nir to dxil")
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7609>

util: Fix/silence variable shadowing warnings

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7552>