review.tizen.org Git - platform/upstream/mesa.git/log

panvk: Implement descriptor copies properly

All we were doing was copying panvk_descriptor structs around which
don't actually contain data that's used by anything interesting.  We
need to copy the actual data arround.  Annoyingly, that means we need a
descriptor copy function per descriptor type.  Woo!

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16276>

panvk: Set immutable samplers properly up-front

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16276>

panvk: Rewrite the write portion of vkUpdateDescriptorSets

The new design is based on the ANV code which I massively cleaned up
some time ago.  Each descriptor type has a write function and they have
consistent prototypes.  This makes it all much easier to read and figure
out what's going on.  It also makes it easier to make changes going
forward because you aren't re-plumbing function arguments if you ever
change the type of data in any given descriptor type.  You just change
the write function.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16276>

panvk: Re-arrange descriptor set functions

Put them in the order we call them which is also roughly descriptor type
enum order.

Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16276>

FIXUP: Use 16-bit things for texture sizes

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16276>

panvk: Implement texture/image queries

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16276>

pavnk: Pass bind layouts to texture and image descriptor helpers

Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16276>

panvk: Add an elems field to panvk_buffer_view

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16276>

panvk: Advertise VK_KHR_variable_pointers

Now that our SSBO descriptor handling code no longer craws deref chains
back to the variable, we should be handling variable pointers properly.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16276>

panvk: Enable robustBufferAccess

It should already work for UBOs. This should do everything we need for
SSBOs. Not sure about vertex and index buffers but we can deal with
those later.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16276>

vulkan: Start collecting enabled features in vk_device

We're not trying to make this 100% complete. For now, all we care about
is robustBufferAccess.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16276>

panvk: Put SSBO addresses in the descriptor buffer

Instead of storing SSBO pointers in the very limited sysval space, store
them in the UBO we've attached to the descriptor set.  This gives us a
virtually unlimited number of SSBOs.  Dynamic SSBOs still live in the
sysval space so we can update them as part of vkCmdBindDescriptorSets().
Also, the new code (based on the code in ANV) loads those SSBO addresses
in a way that never chases the deref chain back to the variable so we
should now be able to handle all of variable pointers.  The code as
written in this patch is a bit overly generic because it switches on
address modes a bit more than panvk needs but we ended up needing all
that flexibility in ANV so we may as well leave hooks for it in panvk.

Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16276>

panvk: Interleave UBOs with multiple descriptor sets

The original intention was to put all the non-dynamic UBOs first
followed by all the dynamic ones.  However, we got the calculations
wrong and, once you went above one descriptor set, things start stomping
each other.

Also, the whole strategy is a bit busted.  Vulkan pipeline layout
compatability rules say that it's ok to create a pipeline with one
layout and then bind with another so long as the bottom N descriptor set
layouts match and the pipeline uses at most N descriptors.  This means
that, while it's safe to have each subsequent set add onto a given pool
of descriptors, if you're going to combine two of those pools, you need
to be careful that the position of descriptors in set N only depends on
the layouts of sets M <= N.  The easy way to do this is to interleve
where we do the UBOs for set 0 then dynamic for set 0 then UBOs for set
1 then dynamic for set 1, etc.

Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16276>

panvk: Put the sysval and push const UBOs at fixed indices

In theory, this may cost us a tiny bit of descriptor space but in
practice, given that the viewport transform is a sysval, we'll always
need it for 3D and given that SSBO pointers live there, we'll basically
always need it for compute. It also makes a lot of things simpler.
We're about to start using the sysval UBO directly in our descriptor set
code and knowing the index up-front is really nice.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16276>

panvk: Stop calling lower_uniforms_to_ubo

We don't need it because Vulkan doesn't have GL-style uniforms. It
*shouldn't* be doing anything but sometimes it inserts an extra UBO
binding and adds 1 to all our UBO indices for no good reason.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16276>

panvk: Use a flat sysvals struct

PanVK uses fewer sysvals than the GLES driver, as some data that would
be a data in GLES is instead part of the descriptor set or the pipeline
state in Vulkan. Therefore, it is simpler and more efficient to use a
flat, fixed layout provided by the driver for our sysvals, rather than
the compiler choosing a layout.

This commit switches to a flat sysval layout.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16276>

panvk: Get rid of the per-pipeline sysvals BO

This is a micro-optimization and probably not a correct one at that.
The cost involved in re-uploading the viewport is tiny compared to the
mental overhead from trying to do this juggle.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16276>

bifrost,midgard: Allow providing a fixed sysval layout

Vulkan doesn't need nearly as many system values and would like to bake
its layout up-front instead of having it provided by the back-end
compiler.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16276>

panfrost: Add some sanity checking for sysvals

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16276>

panfrost,panvk: Make fixed_sysval_ubo < 0 mean compiler-assigned

In 3559efb9bf5c ("panfrost: Allow passing an explicit UBO index for the
sysval UBO"), an explicit UBO index was added and it was implicitly
assumed that it would be > num_ubos.  This was convenient because it
meant 0, the default for designated initializers, implicitly meant
compiler-assigned.  However, we're about to move the sysval UBO to 0
which breaks this assumption.   Also, we don't want the back-end
compiler to even look at num_ubos since it's meaningless in Vulkan.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16276>

panvk: Add a buffer to each descriptor set

Later in the series, we will map descriptor sets to driver-internal
buffers bound as UBOs. These buffers will contain various internal data,
like buffer and texture sizes. Resource access will be lowered to pull
from this UBO in the shader. To prepare, create a backing buffer when
creating descriptor set and emit a UBO record so we can bind it.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16276>

panvk: Break descriptor lowering into its own file

It's about to get a lot more complicated so let's split it out.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16276>

panvk: Move CreateDescriptorSetLayout to per-arch

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16276>

intel/dev: Enable first set of DG2 PCI IDs

Mostly Matt Roper's kernel patch commit message:

The IDs added here are the subset reserved for 'motherboard down'
designs of DG2. We have all the necessary support upstream to enable
these now.

The remaining DG2 IDs for add-in cards will be enabled in a future
patch once some additional required functionality has fully landed.

Ref: https://patchwork.freedesktop.org/patch/msgid/20220425211251.77154-3-matthew.d.roper@intel.com
Cc: 22.1 <mesa-stable>
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16449>

intel/dev: Add DG2 G12 PCI IDs

Ref: https://patchwork.freedesktop.org/patch/483381/?series=103098&rev=1
Ref: https://patchwork.freedesktop.org/patch/msgid/20220425211251.77154-3-matthew.d.roper@intel.com
Cc: 22.1 <mesa-stable>
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16449>

intel/dev: Add INTEL_PLATFORM_DG2_G12

Cc: 22.1 <mesa-stable>
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16449>

radv/winsys: allow to use RADV_FORCE_FAMILY=gfx1100

The bare minimum to play with fossils.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16446>

r600: Elide downloads for discarded and immutable compute memories.

Compute memory item demotion invokes a device to host transfer unconditionally,
but there are at least two cases where this is not necessary:

1. The item is mapped for discarding with PIPE_MAP_DISCARD_RANGE (e.g.
CL_MAP_WRITE_INVALIDATE_REGION).

2. The item cannot be written to by the device.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16116>

r600: Improve compute memory pool performance; reduce fragmentation.

The compute memory pool forced a defragmentation (a left-packing relocation)
of items prior to promoting (adding) items to the tail end of the pool.

This patch instead makes an initial pass over the fragmented pool intent on
promoting items back to where they may have been recently demoted, filling
in the gaps first before conducting the defragmentation (if at all).

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16117>

r600: Fix userspace pointer support for evergreen compute.

Resources returned by r600_buffer_from_user_memory() are not compatible
with the evergreen compute memory pool, though they're added to it anyway.

This results in a segfault reproducible from Clover when the user passes
CL_MEM_USE_HOST_PTR.

This patch allows user_ptr resources to participate in the compute global
memory pool as intended. The result appears to finally allow for zero-copy
DMA out of userspace for anonymous pages.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16114>

amd: fix ac_build_mbcnt_add in wave32 mode

add_src was ignored and i32_0 was always used.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Fixes: 1e49018cedf ("amd: Add extra source to the mbcnt_amd NIR intrinsic.")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16427>

radeonsi: don't use wave32 for GE on gfx10 if culling is used

This is a workaround for issue #6457: it fixes the rendering and
prevent the hang but I can't explain why.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6457
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16427>

radeonsi: fix gs_invocation query with NGG

When NGG is active, the GS invocation counter is always incremented, even
if there's no explicit GS.

Implementing the counter manually fixes it:
* in emit_gs_epilogue for the legacy path
* in gfx10_ngg_gs_emit_prologue for the ngg path

This fixes piglit's arb_query_buffer_object-qbo test.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15861>

radeonsi/ngg: reuse the pipeline stats buffer when using atomics

To support PIPE_STAT_QUERY_GS_INVOCATIONS and PIPE_STAT_QUERY_GS_PRIMITIVES
being used at the same time we have to reuse the same buffer.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15861>

radeonsi/query: use the qbo correct size

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15861>

radeonsi/test: update sienna_cichlid results

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15861>

radeonsi: implement GL_GEOMETRY_SHADER_PRIMITIVES_EMITTED_ARB in shaders

Statistics only work in non-NGG mode. If screen->use_ngg is true, we can't
know if the draw will actually use NGG or not, so this commit switch
to a shader based implementation of this counter.

To avoid modifying si_query, the shader implementation behaves like the hw
one: it uses the same buffer size and offset.

The emulation path activation in the shader is controlled by vs_state_bit[31].

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15861>

radeonsi: store the pipeline stats index

Will be used in later commits.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15861>

radeonsi: deduplicate query offsets

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15861>

gallium: deduplicate target -> stats index code

Now that target_to_index returns the correct index for
PIPE_QUERY_PIPELINE_STATISTICS we can use it in store_query_result instead
of duplicating the switch statement.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15861>

gallium: always return the correct pipeline stats index

Drivers with PIPE_CAP_QUERY_PIPELINE_STATISTICS_SINGLE = 0 shouldn't care
about the value of index, but radeonsi actually needs it in some situations.

So return the correct index instead of 0.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15861>

gallium: add a union to access queries counters

This allows to loop over counters easily.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15861>

radeonsi: implement pipeline stats workaround

DISABLE_INSTANCE_PACKING needs to be enabled when stats queries are
active to fix incorrect results.

We need to emit this for indexed and non-indexed draws.

Based on PAL's waDisableInstancePacking.

This fixes:
KHR-GL46.pipeline_statistics_query_tests_ARB.functional_primitives_vertices_submitted_and_clipping_input_output_primitives

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15861>

radeonsi: flush VGT streamout like PAL

Ported from 165b016bbeedd4e8d1517a5e95949871cc55a123.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15861>

zink: fix sparse texture depth calcs for arrayed textures

use the array, not the depth

cc: mesa-stable

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16456>

zink: remove misleading sparse comment

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16456>

zink: set sparse flag in cubemap lowering

Fixes: 2d745904ca7 ("zink: add a gently mangled version of the d3d12 cubemap -> array compiler pass")

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16456>

zink: fix sparse binding for arrayed textures

this needs to set the arrayLayer, not the z offset

big thanks to Piers Daniell for catching this

cc: mesa-stable

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16456>

zink: clamp out min_lod operands for explicit lod ops

this is illegal (and nonsensical)

cc: mesa-stable

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16456>

zink: fix up sparse texture sampling for shadow samplers

the problem here is that this returns a vec2 instead of a vec5, which
throws all the existing calculations off

given that the shader is (still) expecting a vec2 return from this,
and there's no way to sanely rewrite with nir to be valid for both
sampler types as well as spirv translation, just pad out to a vec2
here and be done with it

Fixes: 73ef54e3424 ("zink: handle residency return value from sparse texture instructions")

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16456>

nir: remove unreachable loop terminators

Remove the conditional break statements associated with all
terminators that are associated with a fixed iteration count,
except for the one associated with the limiting terminator.

This logic matches similiar functionality that exists in the
old GLSL IR unrolling code.

This change helps a piglit test pass on the r300 driver once
we switch off the old GLSL IR unrolling code.

Shader-db results IRIS (BDW):

total instructions in shared programs: 17538619 -> 17538595 (<.01%)
instructions in affected programs: 216 -> 192 (-11.11%)
helped: 3
HURT: 0
helped stats (abs) min: 7 max: 10 x̄: 8.00 x̃: 7
helped stats (rel) min: 10.00% max: 12.07% x̄: 11.38% x̃: 12.07%

total cycles in shared programs: 858674910 -> 858672810 (<.01%)
cycles in affected programs: 79540 -> 77440 (-2.64%)
helped: 3
HURT: 0
helped stats (abs) min: 620 max: 800 x̄: 700.00 x̃: 680
helped stats (rel) min: 2.45% max: 2.83% x̄: 2.63% x̃: 2.62%

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16399>

nir: always set the exact_trip_count_unknown loop terminator property

Previously we only cared if this was set for the limiting
terminator. However in the following patch we will make use of this
information on other terminators to decide if we can eliminate them.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16399>

ntt: make use of new samplers_used field

Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16435>

ntt: Don't gather samplers_declared twice

Prior to scanning instrucitons, we initialize it based on variables. If
this is adding anything, we have a bug somewhere.

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16435>

lavapipe: Set images_used in lvp_lower_pipeline_layout

This mirrors what we do for textures. Also, the current code is busted
because it sets it based on var->data.binding which
lvp_lower_pipeline_layout will change so it can get out-of-sync.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16435>

llvmpipe: Fill out samplers even if nr_samplers = 0

Coming in from Vulkan or OpenCL, it's possible for nr_samplers to be
zero if all we ever use is texelFetch(). Annoyingly, samplers and
sampler views are handled by the same function. Fortunately, it will
work if some of the samplers or sampler views are missing so we can just
pass the maximum.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16435>

gallium/draw: Properly handle nr_samplers != nr_sampler_views in keys

First, make all key_size functions take nr_samplers and nr_sampler_views
separately so we ensure both get passed in. Second, rework the offset
helpers to take MAX(nr_samplers, nr_sampler_views) so we get the image
param offset correct if nr_samplers < nr_sampler_views. While we're
here, also re-order the size calculations to be in the same order as the
things land in memory.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16435>

mesa/st: Set samplers_used in lower_tex_src_plane

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16435>

ttn: Set shader_info::samplers_used

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16435>

gallium: Set shader_info::samplers_used in pstipple_fs

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16435>

lavapipe: Set shader_info::samplers_used

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16435>

clover: Set images/samplers_used when lowering images

Also, stop using BITSET_SET_RANGE_INSIDE_WORD for textures so we can
handle more than 32 of them.

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16435>

turnip: Add a TU_DEBUG=perf debug option.

For doing performance investigation, I often find it useful to have a "are
we tripping over any of our performance TODOs?" flag, so add it and use it
in a few of the TODOs.

This also greatly cleans up the deqp-vk logs.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16316>

radv: Print task shader stage name before disasm.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14929>

radv: Lower shared and task_payload variables in task/mesh shaders.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14929>

radv: Allow linking task shaders.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14929>

radv: Postprocess task shader configuration.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14929>

radv: Use I/O lowering for task and mesh shaders.

We set the number of task shader ring entries in radv_device
based on the generous assumption that each CU can run task/mesh
shaders with maximum occupancy.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14929>

radv: Enable nir_opt_offsets for task shaders.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14929>

radv: Implement task shader intrinsics in the ABI.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14929>

radv: Add task ring entry argument for mesh shaders.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14929>

radv: Add task shader arguments.

Mostly the same as for compute shaders, but with a few extras:

task_ring_offsets:
Same as what ring_offsets is to graphics shaders.
Contains an address that points to a buffer that contains
the ring buffer descriptors.

task_ring_entry:
Index that can be used to address the draw and payload rings.

draw_id:
Same meaning as in graphics shaders.

task_ib_addr/task_ib_stride:
Indirect buffer address and stride from the draw calls.
These are used to emulate the firstTask feature of NV_mesh_shader.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14929>

radv: Fill task shader info.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14929>

radv: Set wave size for task shaders.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14929>

radv: Set user data register for task shaders.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14929>

radv: Add radv_pipeline_has_task helper.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14929>

ac: Add task ring entry shader argument.

This is going to be used by both task and mesh shaders for
accessing the draw and payload ring buffers.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14929>

ac/nir: Add I/O lowering for task and mesh shaders.

Task shaders store their output payload to VRAM where mesh
shaders read from. There are two ring buffers:

1. Draw ring: this is where mesh dispatch sizes and
the ready bit are stored.

2. Payload ring: this is where the optional payload
is stored (up to 16K per task workgroup).

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14929>

nir: Mark negative re-distribution on fadd as imprecise

Otherwise, it would mutate `fneg(fadd(-0, 0))` into `fadd(0, -0)` which
isn't correct since -0 + (+0) = +0 + (-0) = +0.

This fixes the OpenCL contraction tests on Iris.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16041>

iris: Set BindingTableEntryCount for compute shaders

This may slightly increase perf somewhere because the hardware can now
pre-cache binding tables. The real feature is that INTEL_DEBUG=bat now
dumps out surface states for compute.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15759>

svga: workaround for unexpected double swizzle

SM5 requires swizzles for 64 bits alu source to be either .xyzw,
.xyxy, .zwxy, or .zwzw. If the swizzles are not in the valid pattern,
move the source according to the specified swizzle to a temporary register
first.

Reviewed-by: Neha Bhende <bhenden@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16464>

pan/bi: Ensure the end NOP isn't eliminated

Otherwise the lowering doesn't work.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16410>

pan/bi: Support atomics on Valhall

Atomics on Valhall work basically the same as on Bifrost, however the
instruction selection is simplified as there are no clauses. Support the
simplified set of atomic instructions.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16410>

pan/bi: Handle shared/scratch on Valhall

There's no .seg modifier, so we have some easy lowering to do ourselves.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16410>

pan/bi: Handle shared atomic exchange on Valhall

Need to lower the WLS into a segment addition, since the .seg modifier was
dropped on Valhall.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16410>

pan/bi: Preserve AXCHG.i32 destination

dEQP-GLES31.functional.image_load_store.2d.atomic.exchange_r32f_result

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16410>

pan/bi: Support image loads on Valhall

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16410>

pan/bi: Emit LEA_TEX on Valhall

As opposed to LEA_ATTR_TEX. In principle we could do this for Bifrost too, but
let's keep the Midgard compatible path for now.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16410>

pan/bi: Don't analyze td on Valhall

The implementation is based on clauses, so it won't work on Valhall.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16410>

lavapipe: enable subgroup quad operations

Signed-off-by: Autumn on Tape <autumn@cyfox.net>
Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16340>

mesa: Make sure to fallback to handling the original choose texture format

It is possible for st_ChooseTextureFormat to return MESA_FORMAT_NONE when
samples is forced to 2. Always allow to fallback to the original case.

Fixes: 89c94502b6650fed222abd3588e9c927811580aa
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6441
Signed-off-by: Corentin Noël <corentin.noel@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16432>

aco: drop unused radv include

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16342>

aco: remove radv vs prolog key from aco internals.

This creates an aco specific key, and converts radv to it.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16342>

aco/radv: convert to aco shader info at the radv level.

This removes the radv shader info type from aco completely.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16342>

aco/info: add some more fields.

These fields are also used in aco.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16342>

aco: convert vs and so info over to aco structs.

This renames the vs to vp (vertex pipeline) on the way past.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16342>

aco: remove radv specific streamout info

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16342>

aco/info: reduce the gs ring info to what is needed.

Only one member was being used.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16342>

aco: move to a minimal aco shader info struct.

This should be kept to only things aco uses, and expanded when
radeonsi support is added. Things should be removed if lowered in NIR.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16342>