anv: fix vma heap memory leak

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: a5f9e59ce3 ("anv: Use vma_heap for descriptor pool host allocation")
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21385>

ci: bump tags of deqp images

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21392>

ci: fix grouping of image tags

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21392>

ci: remove no-op sed

This is a duplicate from the first patch applied above.

Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21392>

ci: simplify adding & removing deqp patches

Instead of everyone having to copy the curl command from somewhere else
when a new deqp version needs new patches; now all they need to do is
paste the commit hash in the array.

Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21392>

venus: temporarily redirect VkDrmFormatModifierPropertiesListEXT to "2" variant

Temporarily remove driver-side uses of
VkDrmFormatModifierPropertiesListEXT so the encode/decode procedures can
be fixed asynchronously in a follow-up.

Signed-off-by: Ryan Neph <ryanneph@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21367>

panfrost: Fix prim restart XML on Valhall

Harmless in practice (so no need to backport) but still very wrong. Noticed
looking at traces of Dolphin trying to debug acute misrendering.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20780>

radv: add a size check in radv_create_buffer for Android

This is to make dEQP-VK.api.buffer.basic.size_max_uint64 pass on
android.

The test creates a buffer of size UINT64_MAX and makes sure the memory
requirement for the buffer is sane.  It fails because our memory
requirement is "align64(UINT64_MAX, 16)" which is 0 after overflow.

The test checks maintenance4's maxBufferSize and is skipped normally.
But the extension can be disabled on an android build.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21346>

radv: Call nir_lower_array_deref_of_vec in radv_lower_io_to_scalar_early.

This fixes an issue when a vector component of an arrayed output has a deref.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8197
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21028>

radv: Advertise ray query support with LLVM

What could go wrong?

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21268>

radv: Pre-compile BVH build shaders if there is a cache

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21268>

radv: Force ACO for BVH build shaders

They hang with LLVM.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21268>

radv: Make accel struct meta state initialization thread safe

Fixes: 0d5570b ("radv: Always compile accel structure shaders on demand.")
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21268>

ac/llvm: Implement bvh64_intersect_ray_amd

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21268>

zink: handle semi-matching i/o for separate shaders

while separate shaders requires i/o blocks to match between stages,
there are two tricky cases:
* sparse location specification
* variables are required to match in type by location

the first item means user locations must increment if a slot is not used

the second item means that e.g., a mat3x2 can match three vec2 variables
in matching slots

fix both of these cases now

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21383>

panfrost: Disable CRC by default

Known unsound code.

So far I'm not convinced transaction elimination is doing us much good. Even in
synthetic glmark style benchmarks this seems to be a few % hit at most. Given
that transaction elimination is unsound by design, and that panfrost's
implementation is buggy in several places and getting it right (up to the
unsoundness of the hardware feature itself) would take actual engineering
effort, and the priority is making glamor work... disabling is the obvious
choice here.

For now, we leave the code but gate it behind a env var
flag (PAN_MESA_DEBUG=crc) rather than defaulting to enabled unless
PAN_MESA_DEBUG=nocrc is set. This way, we can still experiment with it if we
need that data ("what performance could we gain if we had this feature,
unsoundness be damned?"). That said, I'm not really ok with having unsoundness
on my devices, y'know? Back of the napkin math suggests that it's not unlikely
that somebody has hit a transaction elimination collision in the wild with the
DDK.

Boils down to values.

Closes: #8113
Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21258>

anv: track vram only BOs to print things out on ENOMEM execbuf

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21380>

anv: move debug submit to helper and call it on execbuf failure

Helps telling when you've run out of local memory.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21380>

radv: stop using a PS epilog when the FS doesn't write any color outputs

This is a small optimization for fragment shaders that only write
depth/stencil/sample mask without any color outputs.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21341>

radv: only skip emitting the pipeline blend state if the FS uses an epilog

The blend state is emitted from the command buffer when the FS uses
an epilog (either compiled from a lib with GPL or compiled on-demand).

This shouldn't change anything but it will allow to disable using a
PS epilog when the fragment shader doesn't write any color outputs.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21341>

pvr: Handle VK_QUERY_RESULT_WAIT_BIT.

Not handling device loss currently. That needs to be done
throughout the code base so out of scope for this.

Signed-off-by: Karmjit Mahil <Karmjit.Mahil@imgtec.com>
Reviewed-by: Frank Binns <frank.binns@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20091>

anv: Wa_14016407139, add required pc when SBA programmed

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21374>

intel/perf: also add the oa timestamp shift on MTL

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 90c86fe63e94 ("intel: add MTL performance metrics")
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21378>

radv/amdgpu: only set a new pstate if the current one is different

AMDGPU pstate is per context but if there is multiple AMDGPU contexts
in flight, the kernel can return -EBUSY.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21222>

Revert "radv: acquire pstate on-demand when capturing with RGP"

This change is wrong for two reasons:
- it hangs most of the time maybe, because changing PSTATE when the
  application is running is broken somehow
- it increases the time between triggering and generating the capture
  considerably, because there is a delay for changing PSTATE

This restores previous logic where PSTATE is set to profile_peak at
logical device creation. Though, it also re-introduces an issue with
multiple logical devices (kernel returns -EBUSY) but this will be
fixed in the next commit.

This fixes GPU hangs when trying to record RGP captures on my NAVI21.
Note that profile_peak is only required for some RDNA2 chips (including
VanGogh).

Cc: mesa-stable
This reverts commit 923a864d94517462698c529bdc0e5c056d37b4e1.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21222>

lima: don't use resource_from_handle while creating scanout

resource_from_handle implementations create an additional reference to
the scanout resource, which caused lima to leak those resources after
commit ad4d7ca8332488be8a75aff001f00306a9f6402e.

Do as the other drivers do and import the bo directly while creating
the scanount resource.

Cc: 22.3 mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8198
Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21330>

pvr: Add support to copy descriptors on vkUpdateDescriptorSets()

Signed-off-by: Karmjit Mahil <Karmjit.Mahil@imgtec.com>
Reviewed-by: Frank Binns <frank.binns@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21332>

pvr: Move descriptor write into pvr_write_descriptor_set()

Moving descriptor write functionality from
pvr_UpdateDescriptorSets() into pvr_write_descriptor_set().

This is in preparation for adding descriptor copy support.

Signed-off-by: Karmjit Mahil <Karmjit.Mahil@imgtec.com>
Reviewed-by: Frank Binns <frank.binns@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21332>

virgl: add more formats to conv table

Adding UYVY, YUYV, P010 to formats_conv_table.

Signed-off-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21250>

r600/sfn: Fix Cayman trans from string and add test for copy prop

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21357>

r600/sfn: Fix alu trans op flag setup

Fixes: commit 2df023a1f1990aad6c20eca85af19c7d21a43203
   r600/sfn: pre-evaluate allowed dest mask in Alu instructions

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21357>

r600/sfn: Fix handling of fetch through texture clause

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21357>

r600: Don't start new CF for every fetch through tex clause

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21357>

r600/sfn: Forward setting the block ID and index

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21357>

r600/sfn: address use in group only if instr can be added

Otherwise the group will signal an address use that may not
be relevant.

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21357>

r600/sfn: rename texture coordinate offset for clarity

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21357>

r600/sfn: Stop try scheduling in t-slot with empty related v-slot

This requires adding a nop in the relates v-slot, and the readport
valiation seems to be broken for this case, so drop this for now.

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21357>

r600/sfn: Don't copy propagate indirect loads to more than one dest

Propagating the indirect load to more instructions would result
in more address load instructions. This would (a) remove the advantage
of eliminating one move, and (b) introduce more latency, because between
address load and use two cycles must pass.

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21357>

r600/sfn: Silence warnings about unused parameters

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21357>

r600/sfn: Fix a typo

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21357>

r600/sfn: drop useless instr use count

This is handled with the dest registers

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21357>

r600/sfn: Work around dependency issue when splitting op to group

The instruction that is split may still be referenced as extra
dependency in other instructions, so add a handle to the instruction
that it can be set to be scheduled.

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21357>

r600/sfn: Use range_base for atomics and images

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21357>

mesa/st: support compute shader decoding of ASTC

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19886>

mesa: add astc decoder shader template (glsl es version)

This shader originates from Granite 3D engine and has been adapted
to be used with Open GL and some GLSL ES specifics.

GLSL ES adaptation:

- remove Vulkan specifics: EXT_samplerless_texture_functions usage,
  specialization constants, push constant usage
- inline bitextract.h
- always DECODE_8BIT and hardcode error color (for now)
- port to GLSL ES, required some type changes, explicit type
  conversions and setting up precisions for types

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19886>

mesa/st: initialize resources for ASTC decoding

Generates required resources for ASTC texture decoding pass.

Partition table resources will be cached in to hash during runtime
as one is required for each block size.

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19886>

mesa/st: add astc decoder lookup tables

Commit introduces ASTC decoding lookup tables from Granite 3D engine.

These lookup tables will be used during transcoding by a compute
shader in later commits when decoding ASTC textures.

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19886>

radv: add support for rectangularLines

dEQP-VK.*rectangular_line* pass on NAVI21.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21287>

radv: reduce maximum line width to 8.0

Using 8191.875 seems to big for the hardware to correctly render wide
rectangular lines. This can also be reproduced with AMDVLK by forcing
rectangularLines = True, and fixed by reducing the maximum size as well.

Other drivers seem to expose that value.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21287>

zink: more accurately handle i/o for separate shaders

this can be simplified since i/o is required to match exactly between
stages, meaning that assigning in increasing order should always be correct

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21336>

zink: delete some now-broken ntv dref sampling code

depth splatting should be handled now by the match_tex_dests() pass

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21336>

vulkan: Update the XML and headers to 1.3.241

Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21225>

vulkan/device-select-layer: Include vulkan.h

In the upcoming header update, vk_layer.h starts including vulkan_core.h
instead of vulkan.h.  This will break this layer as it needs a couple of
window-system extension #defines.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21225>

vulkan/layers: Use PUBLIC instead of VK_LAYER_EXPORT

VK_LAYER_EXPORT is going away in the next Vulkan header update.  We
already have a PUBLIC macro in util/macros.h which does the same thing.
Unlike VK_LAYER_EXPORT, it should work in Windows too.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21225>

vulkan: Properly filter structs in vk_physical_device_features

This uses get_all_required to filter structs and also filters struct
members based on API.

Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21225>

vulkan: Move the features generator to vulkan/util

This makes it easier to start depending on vk_extensions.py

Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21225>

vulkan: Filter out provisional extensions

Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21225>

Vulkan: Properly filter structs in vk_cmd_queue_gen

Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21225>

vulkan: Properly filter by api in enum_to_str

This switches us to using get_all_required() for figuring out which
enum types we care about and then carefully filtering every value as
needed.  We also add a number field to Extension so we keep all the
extension XML parsing in one place.

Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21225>

vulkan: Properly filter entrypoints

We now use get_all_required() to get all required commands and use that
to filter instead of doing it manually.  Also, we can pull entrypoint
extension etc. information from the requirements struct.  Finally, we
also have to filter the actual commands themselves as well as arguments
per-API because there may be multiple versions or variants depending on
the API being used.

Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21225>

vulkan: Add a get_all_required() helper

This searches for the names of everything of a particular type: command,
enum, etc. and returns a Requirements struct with any core version and
extensions that require it.

Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21225>

vulkan: Parse the platform in Extensions.from_xml()

This makes handling guards on entrypoints a bit easier.

Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21225>

vulkan: Improve extension parsing

This adds an Extension.from_xml() helper for doing the parsing so we can
re-use it in other code.  We also improve filtering of extensions.  The
Vulkan XML schema is changing to make the supported attribute a comma-
separated list.  This is to allow for vulkansc to also exist in the XML
schema.

Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21225>

vulkan: Remove unused fields from Extension and ApiVersion

These are a left-over from when these classes were used by ANV to define
extension enables in python.  They haven't been used since we added
extension table structs and move extension enables to C.

Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21225>

Revert "vk/util: keep track of extension requirements"

This reverts commit ca98e4446b690709ce517b33d17cb3e2af3f5084.  The way
extension requirements are specified is about to change significantly.
Since this is so new, it's easier to just revert for now.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21225>

Revert "vk/runtime: turn vk.xml extension requirements into asserts"

This reverts commit 6ac830ccb1a54a821c8d035675425f0d97434faa.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21225>

glsl: copy prop vars before scalarizing alus

This generally gives us better results and doing it here in nir will
also allow us to remove more glsl optimisation calls that do a similiar
thing for us.

(Updated shader-db results by idr.)

Tiger Lake, Ice Lake, and Skylake had similar results. (Ice Lake shown)
total instructions in shared programs: 20246333 -> 20240715 (-0.03%)
instructions in affected programs: 235253 -> 229635 (-2.39%)
helped: 425 / HURT: 114

total cycles in shared programs: 891730115 -> 891631113 (-0.01%)
cycles in affected programs: 37347925 -> 37248923 (-0.27%)
helped: 952 / HURT: 692

total spills in shared programs: 7072 -> 6716 (-5.03%)
spills in affected programs: 505 -> 149 (-70.50%)
helped: 7 / HURT: 0

total fills in shared programs: 9897 -> 8511 (-14.00%)
fills in affected programs: 1674 -> 288 (-82.80%)
helped: 7 / HURT: 0

total sends in shared programs: 1053685 -> 1053411 (-0.03%)
sends in affected programs: 2821 -> 2547 (-9.71%)
helped: 30
HURT: 2

LOST:   13
GAINED: 13

Broadwell and Haswell had similar results. (Broadwell shown)
total instructions in shared programs: 18149157 -> 18147271 (-0.01%)
instructions in affected programs: 204630 -> 202744 (-0.92%)
helped: 294 / HURT: 121

total cycles in shared programs: 939488196 -> 939508444 (<.01%)
cycles in affected programs: 36394777 -> 36415025 (0.06%)
helped: 718 / HURT: 620

total sends in shared programs: 1005426 -> 1005152 (-0.03%)
sends in affected programs: 2821 -> 2547 (-9.71%)
helped: 30 / HURT: 2

LOST:   2
GAINED: 2

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Tested-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19715>

panfrost: fix tiny sample_positions BO memory leak

Fixes a 4KB memory leak that happens once per-device creation.

Cc: mesa-stable
Signed-off-by: Italo Nicola <italonicola@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Chris Healy healych@amazon.com
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21372>

intel/nir: Use nir_lower_mem_access_bit_sizes()

This drops the Intel-specific pass in favor of the new generic one.

No shader-db changes on Skylake or DG2.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21232>

nir: Add a load/store bit size lowering pass

This is based on brw_nir_lower_mem_access_bit_sizes() but ended up being
substantially different.  While the core concepts are all the same, the
brw_* version made a lot of Intel-specific assumptions.  The new version
takes a callback which takes a number of bytes of data and an alignment
pair and returns a bit size and number of components to load/store.

Reviewed-by: M Henning <drawoc@darkrefraction.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21232>

ci: enable dEQP-VK.ubo.random.all_shared_buffer.48

The previous commits fix the slow compile time, allowing us to
enable this test.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5152

Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20381>

nir/nir_opt_copy_prop_vars: don't call memset when cloning

This makes the pass significantly faster cutting execution time
by around 30% in the cts test
dEQP-GLES31.functional.ubo.random.all_per_block_buffers.20

This 30% improvement is in addition to all the improvements from
the proceeding patches.

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20381>

nir/nir_opt_copy_prop_vars: reorder clone calls

This helps with the reuse of dynamic arrays.

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20381>

nir/nir_opt_copy_prop_vars: reuse dynamic arrays

As per the previous commit if we don't reuse these dynamic arrays
we end up needlessly thrashing the memory handling functions.

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20381>

nir/nir_opt_copy_prop_vars: reuse hash tables

Due to how this pass works we can end up thrashing memory if we
do not reuse these hash tables rather than reusing them.

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20381>

nir/nir_opt_copy_prop_vars: avoid comparison explosion

Previously the pass was comparing every deref to every load/store
causing the pass to slow down more the larger the shader is.

Here we use a hash table so we can simple store everything needed
for comparision of a var separately.

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20381>

nir/nir_opt_copy_prop_vars: remove extra loop

The fix in 947f7b452a55 introduced an extra loop over the copies
array to find the correct entry in the case it had been moved.

The problem is these loops can be iterated over millions of times
so lets simply update the entry pointer in the case we change its
location in the array.

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20381>

nir/from_ssa: Move the loop bounds check in resolve_parallel_copy

We loop, effectively, over two stacks: ready and to_do and finish only
when both are empty.  In the case where ready is empty, we pull one off
of to_do, add a copy to a temporary, and push it onto the ready stack.
Previously, we assumed that we would never get to the temporary copy
case if to_do has exactly one entry because that would imply that there
was only one copy left which means there can't possibly be a cycle to
break.  This was true until c7fc44f9ebbe ("nir/from_ssa: Respect and
populate divergence information") which changed things such that
temporary copies sometimes get added in the case where a convergent
value is copied both to convergent and divergent destinations.

This patch adjusts our loop iteration to always attempt to clear the
ready stack before checking if there's anything left on the to_do stack.
I also added an assert to make the exit condition more clear.

Fixes: c7fc44f9ebbe ("nir/from_ssa: Respect and populate divergence information")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8037
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21315>

nir/from_ssa: Only re-locate values that are destinations

There is an optimization in the parallel copy algorithm where, after a
copy has been performed, we can treat the destination as the new source
for future copies of the same source.  In particular, consider the
following parallel copy: A -> B, C -> A, A -> C.  In this case, after we
have done the A -> B copy, we can make note that the value in A is now
in B and emit the sequence: A -> B, C -> A, B -> C.  This allows us to
resolve the swap cycle between A anc C without allocating a temporary
register because we know B is also a copy of A.

When one of the registers involved is convergent and the other is
divergent, this optimization is problematic because, while convergent to
divergent copies are fine, we can't re-use the divergent copy in later
copies if any of those copies are to a convergent variable.  We could,
but it would require a read_first_invocation which would get messy.  In
In c7fc44f9ebbe ("nir/from_ssa: Respect and populate divergence
information"), we attempted to deal with this by limiting the rename
optimization to the case where the divergence matched.

The problem is that we did the re-name part whenever the divergence
matched but only marked it as ready if the thing being copied was a
destination.  (We actually left two instances of loc[a] = b, one which
always happened and one which only happened if we also wanted to flag
the source as being ready to use as a destination.)  While this
technically doesn't cause any problems, it may result in more inter-mov
dependencies which hurts instruction scheduling.  For example, if we had
the parallel copy A -> B, A -> C, A -> D, we now end up emitting the
sequence A -> B, B -> C, C -> D which has many more data hazards between
instructions caused by the constant shuffling.

This commit restores the original logic in which we only perform the
rename optimization if the rename would free up a register we will later
use as a destination.  This isn't entirely optimal as it still doesn't
prove that there is a cycle involved first, but it should lead to a
reduction in unnecessary dependencies.

No shader-db changes on SKL or DG2

Fixes: c7fc44f9ebbe ("nir/from_ssa: Respect and populate divergence information")
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21315>

freedreno/drm: Optimize stateobj re-emit

For long-lived stateobjs, it is common to re-emit to the same submit
multiple times.  By giving each submit a unique sequence # we can detect
this case and skip the extra append_bo().

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21274>

freedreno: Add seqno helper

It is a pretty common pattern to allocate a non-zero sequence # for
lightweight checking if an object is the same, changed, for use in cache
keys, etc.  (And also pretty common to forget to handle the rollover
zero case.)  Add a helper for this.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21274>

freedreno: Drop batch lock

Now that we are not tracking cross-context batch dependencies, there is
no scenario where one context could trigger flushing another context's
batch.  So we can drop the batch lock intended to protect against this.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21274>

freedreno/batch: Stop tracking cross-context deps

The app is expected to provide suitable cross-context synchronization
(fences, etc), so don't try to do it's job for them.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21274>

freedreno: Avoid taking screen lock

Avoid taking screen unlock for batch unref.  Instead just split the
destroy fxn into locked and unlocked variants.  That way we only end
up taking the screen lock on final unref but avoid it in the common
case.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21274>

freedreno/a6xx: Pre-compute PROG related LRZ state

PROG state mostly just disables various LRZ related flags, which can
be handled as a simple mask.  The exception is ztest mode, which is
either overriden by PROG state, or we use the all 1's value (which
isn't valid from hw standpoint) to signal that it needs to be computed
at draw time, which fortunately fits in with the bitmask approach.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21274>

freedreno: Move FD_MESA_DEBUG cases out of draw_vbo

If the debug options are enabled, just plug in a debug version of
draw_vbo with the additional checks.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21274>

freedreno: Move blend out of dirty-rsc tracking

This was not doing any actual resource tracking, just updating
gmem_reason.  And furthermore, a6xx+ doesn't care about the bits
it was setting.  So move this to per-gen backend for the gens that
need it, and avoid setting FD_DIRTY_RESOURCE when FD_DIRTY_BLEND
is set.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21274>

freedreno/a6xx: Remove tex-state refcnting

Now that we use a flag to trigger the tex state invalidation coming from
other contexts, we can drop the refcnt'ing.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21274>

freedreno/drm: Make rb refcnt non-atomic

Now that the one special case where multiple threads could race to
ref/unref, we can go back to using non-atomic refcnts.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21274>

freedreno/a6xx: Do tex-state invalidates in same ctx

If a resource invalidate is triggered by a different ctx (potentially on
a different thread) simply flag that the tex state needs invalidation,
but defer handling it to the ctx that owns the tex state.

This will let us remove atomic refcnt'ing on the tex state, and more
importantly atomic refcnt'ing on the fd_ringbuffer (as this was the one
special case where rb's could be accessed from multiple threads).

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21274>

freedreno/a6xx: Multi-draw support

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21274>

freedreno/a6xx: Split out flush_streamout() helper

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21274>

freedreno/a6xx: Drop unused return

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21274>

freedreno: Push num_draws down to backend

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21274>

freedreno: Account for multi-draw in num_draws

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21274>

radv/rt: don't hash maxPipelineRayRecursionDepth

The stack size has no effect on the generated shader anymore.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21159>

radv/rt: use dynamic_callable_stack_base also for static stack_sizes

This patch also removes rt_pipeline->dynamic_stack_size and replaces
it by checking for rt_pipeline->stack_size == -1u.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21159>

radv/rt: introduce and set rt_pipeline->stack_size

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21159>

radv: unconditionally enable scratch for RT shaders

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21159>

radv: rename shader_info->cs.uses_sbt -> shader_info->cs.is_rt_shader

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21159>