d3d12: Fix offset for buf/image copies with suballocated buffers

Reviewed-by: Sil Vilerino <sivileri@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14933>

d3d12: Don't suballocate TBO buffers

Reviewed-by: Sil Vilerino <sivileri@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14933>

d3d12: Fix TBOs from suballocated buffers

Reviewed-by: Sil Vilerino <sivileri@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14933>

d3d12: Delete make_resource_writeable

This never did anything useful AFAICT since we didn't actually
suballocate buffers, and when this ended up being invoked it breaks
the ability to read back XFB data.

Reviewed-by: Sil Vilerino <sivileri@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14933>

d3d12: Always respect offsets when mapping a bo, not just when there's a range

Reviewed-by: Sil Vilerino <sivileri@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14933>

d3d12: Fix range calculation for suballocated buffers in d3d12_bo_unmap

Reviewed-by: Sil Vilerino <sivileri@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14933>

d3d12: Fix set constant buffers

Reviewed-by: Sil Vilerino <sivileri@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14933>

tc: CPU storage needs to be freed with align_free

Cc: mesa-stable
Acked-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Sil Vilerino <sivileri@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14933>

panfrost: Fix Depth Source enum

As I suspected... sigh.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14935>

panfrost: Remove unused layout enums

Folded into Valhall-specific plane descriptor enums.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14935>

panfrost: Remove some indexed formats on Valhall

Block compressed formats like ETC2 are now indicated in the plane descriptor,
rather than the pixel format descriptor. Various other minor formats were
removed in Valhall; remove them from the XML so we don't accidentally try to use
them.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14935>

panfrost: Update supported job types

Remove a few that no longer exist, and rename IDVS helper to Malloc Vertex. The
distinction between Malloc Vertex jobs and regular Indexed Vertex jobs is that
the hardware allocates varying buffers dynamically for Malloc Vertex jobs.
Regular IDVS and even legacy tiler jobs are also supported where desired.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14935>

panfrost: Flesh out tiler heap descriptor

Merged with the Buffer descriptor, hence why it shares a type nibble. However,
Bifrost uses a dedicated tiler heap descriptor, and I see no benefit to merging.
So pretending it's a dedicated descriptor on Valhall too allows us to reuse the
Bifrost code with no modifications.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14935>

panfrost: Strip % in GenXML names

A new Valhall enum will represent percentages, so allow that.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14935>

panfrost: Flesh out Buffer descriptor

Add fields required for structured buffers.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14935>

vulkan,lavapipe: Simplify command recording code-gen

The Entrypoint class already has utilities for gettingt he parameter
list as either declarations or as comma-separated argument names for a
call.  Use that instead of hand-rolling it.  The only modification we
need to make is to add the ability to start the list somewhere other
than at the beginning.

Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Acked-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14919>

lavapipe: ci updates

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14911>

zink: ci updates

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14911>

lavapipe: use util_pack_color_union() for generating clear colors

this enables clamping for packed formats (e.g., RGB10_A2UI) where color
values may exceed the width of the component

cc: mesa-stable

fixes (zink):
KHR-GL45.direct_state_access.renderbuffers_storage*

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14911>

ci/freedreno: Add another unsizedArrayLength flake.

Started appearing on Feb 1, but given that the rest of this test group
flakes, I assume it's similar.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14934>

venus: Allow usage of virtio-mmio based device

Libdrm reports bustype as DRM_BUS_PLATFORM for virtio-mmio
based device. DRM_BUS_PCI is reported only for virtio-pci based
devices. Add possibility to use devices with DRM_BUS_PLATFORM.

Signed-off-by: Andrii Pauk <Andrii.Pauk@opensynergy.com>
Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14531>

aco: optimize discard_if when WQM is not needed afterwards

Totals from 11560 (8.57% of 134913) affected shaders: (GFX10.3)
CodeSize: 12092560 -> 11997652 (-0.78%)
Instrs: 2205325 -> 2181598 (-1.08%)
Latency: 15376048 -> 15356958 (-0.12%); split: -0.12%, +0.00%
InvThroughput: 3526105 -> 3525120 (-0.03%); split: -0.03%, +0.00%
Copies: 98543 -> 87601 (-11.10%)
Branches: 16919 -> 16873 (-0.27%)
PreSGPRs: 291584 -> 291532 (-0.02%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14805>

aco: merge block_kind_uses_[demote|discard_if]

These serve the same purpose. The new name is
block_kind_uses_discard.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14805>

aco: make Preserve_WQM independent from block_kind_uses_discard_if

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14805>

aco: remove block_kind_discard

This case doesn't seem to happen in practice.
No need to micro-optimize it.

This patch merges instruction selection for discard/discard_if.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14805>

aco: emit nir_intrinsic_discard() as p_discard_if()

This simplifies the code and emits a slightly better
sequence in some cases.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14805>

mesa: align constant/uniform uploads to driver expected alignment

This fixed a problem for Zink where uniform buffer alignment varies by
GPU, e.g. 64 bytes for an RTX 2070 SUPER but 256 bytes for a GTX 1070
Ti.

Tested running Superposition on Windows 10 with Nvidia 1070 Ti with
496.13 driver.  Without the fix Superposition soft locks on its splash
screen.  With the fix Superposition runs through its benchmark.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14674>

zink: Fix MSVC RTC in zink_get_framebuffer_imageless()

The bit fields in zink_framebuffer_state cause a false positive with
MSVC's run-time checks enabled.  setting state.num_attachments in
zink_get_framebuffer_imageless().  Writing some bits of num_attachments
involves reading bits from layers and samples that haven't been
initialized.

Fixed by assigning to num_attachments earlier in the function.  Not
quite sure why that makes a difference but at a guess there's a
heuristic that considers assignment close to declaration as
initialization.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14566>

zink: export PIPE_CAP_CULL_DISTANCE_NOCOMBINE

fixes:
KHR-GL46.cull_distance.functional

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14878>

gallium: add PIPE_CAP_CULL_DISTANCE_NOCOMBINE

for drivers where separate cull distance variables are required, this
lets them avoid having to write yet another pass to undo gallium's mangling
of shader info

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14878>

anv: fix conditional render for vkCmdDrawIndirectByteCountEXT

We just forgot about conditional render for this entry point.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 2be89cbd826f9a ("anv: Implement vkCmdDrawIndirectByteCountEXT")
Tested-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14891>

anv: enable ray queries

Only on platforms that support it.

v3: Split out code setting up ray query shadow buffer (Caio)
    Don't forget to setup ray query globals even when no shadow buffer
    is used (Lionel)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13719>

intel/fs: lower ray query intrinsics

v2: Add helper for acceleration->root_node computation (Caio)

v3: Update comment on "done" bit (Caio)
    Remove progress bool value for impl function (Caio)
    Don't use nir_shader_instructions_pass to search the shader (Caio)

v4: Rename variable for if/else block (Caio)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13719>

intel/nir: document RT builder

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13719>

nir/lower_shader_calls: consider relocated constants as rematerializable

After all they're constants.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13719>

intel/nir/rt: add more helpers for ray queries

v2: Split stack_id helper in sync/async version (Caio)
    Fixup a few bit field mistake (Caio)
    Simplify some bitfield manipulations (Caio)

v3: Remove duplicated helper (Caio)
    Simplify brw_nir_rt_set_dword_bit_at (Caio)
    Comment brw_nir_rt_query_mark_init (Lionel)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13719>

intel/nir/rt: make RT manipulation helpers helper invocations ready

Since we need to be able to perform ray queries in helper invocations,
we need to have all the helpers properly tag their load/store
operations so that they operate in helper lanes.

v2: Switch from macros to inline functions (Caio)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13719>

intel/nir: document committed argument

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13719>

intel/fs: make trivial shader complete tracing operations with missing shaders

v2: Apply workaround only on < DG2-512-C0 & < DG2-128-B0

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13719>

intel/nir/rt: load bvh_level value off mem_hit structure

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13719>

intel: add a new intrinsic to get the shader stage from bindless shaders

We'll use this to apply ray tracing operations in our trivial return
shader based on the stage we're in.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13719>

nir/builder: add nir_ior_imm() helper

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13719>

intel/nir: use a single intel intrinsic to deal with ray traversal

In the future we'll want to reuse this intrinsic to deal with ray
queries. Ray queries will use a different global pointer and
programmatically change the control/level arguments of the trace send
instruction.

v2: Comment on barrier after sync trace instruction (Caio)
    Generalize lsc helper (Caio)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13719>

intel/nir: fix shader call lowering

We're replacing a generic instruction by an intel specific one, we
need to remove the previous instruction.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: c5a42e401036 ("intel/fs: fix shader call lowering pass")
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13719>

intel/fs: limit FS dispatch to SIMD16 when using ray queries

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13719>

intel/compiler: tracker number of ray queries in prog_data

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13719>

intel/fs: load more fields from BVH instance leafs

v2: Fixup mask (Caio)
    Drop old comment (Caio)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13719>

intel/fs: don't set allow_sample_mask for CS intrinsics

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 77486db867bd ("intel/fs: Disable sample mask predication for scratch stores")
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13719>

intel/fs: add support for ACCESS_ENABLE_HELPER

v2: Factor out fragment shader masking on send messages (Caio)
    Update comments (Caio)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13719>

intel/fs: name sources for A64 opcodes

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13719>

intel/nir/rt: add a new number of SIMD lanes per DSS helper

v2: Add prefix brw_nir_rt (Caio)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13719>

intel/fs: add support for Eu/Thread/Lane id

This index will be used for accessing ray query data in memory.

v2: Drop a MOV (Caio)

v3: Rework back code emission (Caio)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13719>

intel/fs: rework dss_id opcode into generic opcode

We'll want different types of IDs based on topology. Let's make this
more flexible and also move the bit shifting code a layer above where
it's easier to do bitshifting operations, especially if you need to
stash things into temporary registers.

v2: Keep previous comment.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13719>

nir: change intel dss_id intrinsic to topology_id

This will allow to reuse the same intrinsic for various topology based
ID.

v2: fix intrinsic comment (Caio)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13719>

intel/ds: fix compilation with perfetto

Forgot to test with perfetto...

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 9da3d714b87c ("anv: add dynamic rendering traces")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5992
Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Tested-by: Rohan Garg <rohan.garg@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14914>

meson: add radv to meson devenv

I either rebased this out of the original PR, just failed to commit it
and then reset it.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14879>

vk/sync: add asserts for timeline semaphore count matching

spec requires that the number of timeline waits/signals matches the
base number of waits/signals if there are any timeline semaphores
being processed by the submit, so asserting here is in line with what
validation will yield

failure to match these will also hang every driver I've tested, so asserting
here potentially saves some people their desktop session

Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14741>

zink: min/max blit region in coverage functions

these regions might not have the coords in the correct order, which will
cause them to fail intersection tests, resulting in clears that are never
applied

cc: mesa-stable

fixes:
GTF-GL46.gtf30.GL3Tests.framebuffer_blit.framebuffer_blit_functionality_all_buffer_blit
GTF-GL46.gtf30.GL3Tests.framebuffer_blit.framebuffer_blit_functionality_color_and_depth_blit
GTF-GL46.gtf30.GL3Tests.framebuffer_blit.framebuffer_blit_functionality_color_and_stencil_blit
GTF-GL46.gtf30.GL3Tests.framebuffer_blit.framebuffer_blit_functionality_linear_filter_color_blit
GTF-GL46.gtf30.GL3Tests.framebuffer_blit.framebuffer_blit_functionality_magnifying_blit
GTF-GL46.gtf30.GL3Tests.framebuffer_blit.framebuffer_blit_functionality_minifying_blit
GTF-GL46.gtf30.GL3Tests.framebuffer_blit.framebuffer_blit_functionality_missing_buffers_blit
GTF-GL46.gtf30.GL3Tests.framebuffer_blit.framebuffer_blit_functionality_nearest_filter_color_blit
GTF-GL46.gtf30.GL3Tests.framebuffer_blit.framebuffer_blit_functionality_negative_dimensions_blit
GTF-GL46.gtf30.GL3Tests.framebuffer_blit.framebuffer_blit_functionality_negative_height_blit
GTF-GL46.gtf30.GL3Tests.framebuffer_blit.framebuffer_blit_functionality_negative_width_blit
GTF-GL46.gtf30.GL3Tests.framebuffer_blit.framebuffer_blit_functionality_scissor_blit

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14867>

zink: reject invalid draws

cc: mesa-stable

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14859>

zink: fix PIPE_CAP_TGSI_BALLOT export conditional

this requires VK_EXT_shader_subgroup_ballot

cc: mesa-stable

fixes (lavapipe):
KHR-GL46.shader_ballot_tests.ShaderBallotAvailability
KHR-GL46.shader_ballot_tests.ShaderBallotFunctionRead

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14858>

zink: export PIPE_SHADER_CAP_TGSI_CONT_SUPPORTED

this is supported and has been for a while

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14858>

radeonsi/blit: relax conditions to use sdma copy for prime buffers

We don't need to check if it's imported: PIPE_BIND_DRI_PRIME is enough.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14615>

radeonsi: create prime buffers as uncached

8791e831b11 marked imported prime buffers as uncached (useful when prime
buffer is allocated by the display GPU), but they should also be created
as uncached (useful when allocated by the render GPU).

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14615>

gallium: rename PIPE_BIND_DRI_PRIME

The new name PIPE_BIND_PRIME_BLIT_DST is more precise.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14615>

gallium/dri: add missing PIPE_BIND_DRI_PRIME handling

e9c3dbd0461 added PIPE_BIND_DRI_PRIME but it was only set when
importing a prime buffer.

This commit adds handling of this flag in the other codepath = the
one where the prime buffer is allocated by the render GPU.

With this change PIPE_BIND_DRI_PRIME is still only set for the
render GPU - the display GPU will never see this flag; a future
commit will rename it.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14615>

ci/iris: Mark qbo tests as flakes

These appear to have some kind of race condition and usually fail,
but sometimes pass.  We had already attempted to mark them as flakes
on amly, but need to mark them as flakes on KBL+ too.

See https://gitlab.freedesktop.org/mesa/mesa/-/jobs/18522605 and
https://gitlab.freedesktop.org/mesa/mesa/-/jobs/18527737 where these
unexpectedly passed on KBL, and also where the top-level test not
being caught by the regex led to failures.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14916>

crocus: Enable compat profile the same way as core profile

Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11756>

iris: Add support for flushing the blitter (hackily)

To flush the blitter, we need to use MI_FLUSH_DW rather than the usual
PIPE_CONTROL we use on the 3D engine.  Most of our code is set up to
suggest flushes via PIPE_CONTROL commands, however, so we hackily just
emit MI_FLUSH_DW when they ask for any kind of PIPE_CONTROL flush.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14912>

blorp: Assert that blorp_copy() on the blitter can handle it

Safeguards against callers that don't guarantee the necessary things.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14912>

intel/genxml: Add missing MI_FLUSH_DW::Flush CCS field

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14912>

radv: fix R_02881C_PA_CL_VS_OUT_CNTL with mixed cull/clip distances

Matches radeonsi.

Seems Vulkan CTS doesn't really test cull distances. Removing
VARYING_SLOT_CULL_DIST0/VARYING_SLOT_CULL_DIST1 variables doesn't break
any of dEQP-VK.clipping.*, except for tests which read the variables in
the fragment shader.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5984
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14882>

tu: Implement VK_AMD_buffer_marker to support Graphics Flight Recorder

Graphics Flight Recorder is:
 "The Graphics Flight Recorder (GFR) is a Vulkan layer to help
  trackdown and identify the cause of GPU hangs and crashes.
  It works by instrumenting command buffers with completion tags."

This is a nice little tool which could help quickly identify the call
which hanged. Or if command buffer is executed for too long.

The tiling nature of our GPU shouldn't be a big issue aside from
lower performance.

For non-segfault case, if:
- Hang happens at the same place in cmdbuf and draw/dispatch is not
  finished at that point - it is likely that there is an infinite
  loop in some of the shaders in this draw.
- Hang happens always in different place - likely there is nothing
  wrong and command buffer just takes too long to execute and you
  should try increasing hangcheck_period_ms. If it doesn't help
  it is likely a synchronization issue.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13553>

egl/wayland: Don't replace existing backbuffer in get_buffers

If the surface already has a current backbuffer - say through a
buffer_age query - we do not want to replace it in get_buffers, because
it means the result we'd previously returned them is stale.

If we already have a backbuffer set on the surface, keep it locked in no
matter what until we hit SwapBuffers.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14873>

egl/wayland: Reset buffer age when destroying buffers

A buffer age of 0 means that the buffer is uninitialised or has unknown
content. We rely on the buffer age initially being 0 through zalloc when
the surface is first created; when they are first used for a swap, we
set their age to 1, and then we increment the age of every buffer in the
chain with a non-zero age when we swap.

Now that we can release buffers, both through dmabuf-feedback as well as
detecting when we're using a deeper swapchain than the compositor needs,
make sure to reset their age as they are released. Without doing this,
the age will stay as it was before it was released and be incremented,
returning the wrong age to the user the first time a previously-released
buffer slot has been reused.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5977
Fixes: 22d796feb828 ("egl/wayland: break double/tripple buffering feedback loops")
Fixes: b5848b2dac14 ("egl/wayland: use surface dma-buf feedback to allocate surface buffers")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14873>

ci/iris: Add skips and flakes notes for recent #intel-ci logs.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14904>

ci/crocus: Add recent flakes from #intel-ci

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14904>

ci/softpipe,llvmpipe: Disable Xvfb server reset on piglit runs.

The resets take time that we don't need to spend.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14904>

Revert "radv: re-apply "Do not access set layout during vkCmdBindDescriptorSets.""

The most famous RADV revert over the past months. This was an issue
in RADV and not an use-after-free (descriptor set layouts can be
destroyed almost at any time).

This reverts commit b775aaff1ec86f4ebd50867a045695da1fbeb2e1.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14621>

radv: add reference counting for descriptor set layouts

The spec states that descriptor set layouts can be destroyed almost
at any time:

   "VkDescriptorSetLayout objects may be accessed by commands that
    operate on descriptor sets allocated using that layout, and those
    descriptor sets must not be updated with vkUpdateDescriptorSets
    after the descriptor set layout has been destroyed. Otherwise,
    descriptor set layouts can be destroyed any time they are not in
    use by an API command."

Based on ANV.

Gitlab: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5893
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14621>

crocus: find correct relocation target for the bo.

If we have batch a + b, and writing to batch b, causes batch a
to flush, all the bo->index get reset, and we try to submit a -1
to the kernel.

Look the bo index up when creating relocations.

Fixes crash seen in KHR-GL46.compute_shader.pipeline-post-fs
and a trace from Wasteland 3

Fixes: f3630548f1da ("crocus: initial gallium driver for Intel gfx 4-7")

Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14905>

crocus: enable GL46 tests for HSW in ci

Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14889>

asahi: Fix memory unsafety in delete_sampler_state

The type is wrong, masked by a void*, meaning the free is completely
wrong. ASan is rightfully unhappy. Fixes crashes destroying the context.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14901>

agx: Don't kill helper threads in ld_var

Apparently this is yet another .kill bit. Fixes:

dEQP-GLES3.functional.shaders.derivate.dfdx.linear.*
dEQP-GLES3.functional.shaders.derivate.dfdy.linear.*

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14899>

agx: Handle texture array indices

These need to be converted to integers.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14899>

agx: Implement nir_op_txb

Like explicit LODs, biases must be 16-bit, so add a lowering rule for
this. With the LOD mode selection updated for txb, we can then ingest
biases like explicit LODs and allowlist txb. Passes:

dEQP-GLES2.functional.shaders.texture_functions.fragment.texture2d_bias
dEQP-GLES2.functional.texture.mipmap.2d.bias.*

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14899>

agx: Translate LOD modes more generically

Now includes support for auto_load_bias mode.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14899>

agx: Add AUTO_LOD_BIAS mode

Automatic load with a bias.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14899>

asahi: Correctly set IOGPU_ATTACHMENT::size

Not sure what this is used for, but let's not lie to the kernel.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14898>

asahi: Identify IOGPU_ATTACHMENT::size

Oops.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14898>

mesa: fix misaligned pointer returned by dlist_alloc

In cases where the to-be-allocated node size with padding exceeds BLOCK_SIZE
but without padding doesn't, a new block is not created and no padding is done
to the previous instruction, causing a misaligned pointer to be returned.

v2: Per Ilia Mirkin's suggestion, remove the extra condition in the first
    if statement, let it unconditionally pad the last instruction if needed.
    The updated currentPos will then be taken into account in the
    block size checking.

This fixes crash seen with lightsmark and Optuma apitraces

Fixes:  05605d7f537c4 (' mesa: remove display list OPCODE_NOP')

Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>
Reviewed-by: Neha Bhende <bhenden@vmware.com>
Tested-by: Neha Bhende <bhenden@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14871>

svga: store shared_mem_size in svga_compute_shader instead of svga_context

When new context was created, shared_mem_size was getting overwritten.
This fixes glretrace failure seen with manhattan, aztec and BASS2_intro
apitraces

Fixes: 247c61f2d0454a ('svga: Add support for compute shader, shader buffers and image views')

Tested with glretrace, piglit

Reviewed-by: Charmaine Lee <charmainel@vmware.com>
(cherry picked from commit dd6793ec9218782b1b716a87582d7219bae4e75f)

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14870>

anv: Increase maxUniformBufferRange to 2^30 when not using the sampler

The limit here is from the RENDER_SURFACE_STATE height/width/depth
fields - it's 2^30 for ISL_FORMAT_RAW buffers, and 2^27 otherwise.

anv_isl_format_for_descriptor_type() uses ISL_FORMAT_R32G32B32A32_FLOAT
for uniform buffers when compiler->indirect_ubos_use_sampler is set
(Icelake and earlier), but ISL_FORMAT_RAW when it isn't (Tigerlake+).

So we can increase the limit on Tigerlake and later.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14892>

r300: fix deadcode elimination in loops with breaks

We are updating the deadcode state while walking the program backwards.
When encountering ENDLOOP, we scan the loop, mark everything in the loop
as used and than continue as usuall. We were previously trying to be
smart with the breaks. This was however not working as expected.

Instead, save the most pesimistic deadcode state from the ENDLOOP and
just restore it anytime we see a break.

This keeps the code simple and more importantly does not touch the flat
and IF(-ELSE)-ENDIF paths at all so reduces the chances of regression.
No changes with my shader-db.

Fixes piglits on RV530:
    shaders/ssa/fs-if-def-else-break.shader_test
    spec/glsl-1.10/execution/vs-loop-array-index-unroll.shader_test

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5832
Signed-off-by: Pavel Ondračka <pavel.ondracka@gmail.com>
Acked-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14661>

anv: add dynamic rendering traces

v2: Get rid of subpass_count

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14798>

anv: flush utrace before at device destroy

Ensuring any remaining traces are displayed.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14798>

zink: use scanout obj when returning resource param info

embarrassing typo since the base obj has no modifier data available

cc: mesa-stable

fixes #5980

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14875>

pan/midg: Support 8/16 bit load/store

Needed for panvk copy shaders to support 8 or 16bit formats.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14885>

pan/midg: Add a pass to lower non-logbase2 global/shared loads

Compute shaders might do vec3(Xbits) loads which are translated
to LD.<next_pow2(3 * X)> by the midgard compiler. This might cause
out-of-bound accesses potentially leading to pagefaults if the
access is at the end of a BO. One solution to avoid that (maybe not
the best) is to split non-log2 loads to make sure we only read what's
requested.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14885>

pan/midg: Fix swizzle packing on 64bit instructions with src-expansion + dst-shrinking

In that case, the mask is specified on 32bit lanes, so we need to shift
it if it's > 0x3. The expand modifier will take care of selecting the
right side of the 32bit vector.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14885>

pan/midg: Fix the upper/lower limit on 8bit vectors

If I'm correct, the lower/upper split on 8bit vectors is 8, not 4.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14885>