broadcom/compiler: log spilling shaders to perf output

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8825>

broadcom/compiler: disallow spilling if TMU pipelining was enabled

TMU pipelining makes TMU spilling difficult and can easily lead to
doing large amounts of spills to compile a shader. It is best to
only use pipelining if we can compile without spilling.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8825>

broadcom/compiler: disable TMU pipelining if we fail to register allocate

TMU pipelining can severely reduce our capacity to emit TMU spills,
causing us to fail to compile a shader we may otherwise be able to
compile. This is because pipelining extends the liveness of TMU
sequences by posponing the thread switch and LDTMU until a result
is needed, and we can't emit TMU spills while in the middle of a
TMU sequence.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8825>

broadcom/compiler: support pipelining of image load/store instructions

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8825>

broadcom/compiler: refactor image load/store TMU emission code

This mostly moves code around to group together the code involved with
actually emitting a TMU sequence. This will make it a bit easier to
then implement pipelining while reusing this code, similar to how we
handled other cases of TMU pipelining.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8825>

broadcom/compiler: support pipelining of tex instructions

This follows the same idea as for TMU general instructions of reusing
the existing infrastructure to first count required register writes and
flush outstanding TMU dependencies, and then emit the actual writes, which
requires that we split the code that decides about register writes to
a helper.

We also need to start using a component mask instead of the number
of components that we need to read with a particular TMU operation.

v2: update tmu_writes for V3D_QPU_WADDR_TMUOFF

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8825>

broadcom/compiler: implement pipelining for general TMU operations

This creates the basic infrastructure to implement TMU pipelining and
applies it to general TMU. Follow-up patches will expand this
to texture and image/load store operations.

TMU pipelining means that we don't immediately end TMU sequences,
and instead, we postpone the thread switch and LDTMU (for loads)
or TMUWT (for stores) until we really need to do them.

For loads, we may need to flush them if another instruction reads
the result of a load operation. We can detect this because in that
case ntq_get_src() will not find the definition for that ssa/reg
(since we have not emitted the LDTMU instructions for it yet), so
when that happens, we flush all pending TMU operations and then
try again to find the definition for the source.

We also need to flush pending TMU operations when we reach the end
of a control flow block, to prevent the case where we emit a TMU
operation in a block, but then we read the result in another block
possibly under control flow.

It is also required to flush across barriers and discards to honor
their semantics.

Since this change doesn't implement pipelining for texture and
image load/store, we also need to flush outstanding TMU operations
if we ever have to emit one of these. This will be corrected with
follow-up patches.

Finally, the TMU has 3 fifos where it can queue TMU operations.
These fifos have limited capacity, depending on the number of threads
used to compile the shader, so we also need to ensure that we
don't have too many outstanding TMU requests and flush pending
TMU operations if a new TMU operation would overflow any of these
fifos. While overflowing the Input and Config fifos only leads
to stalls (which we want to avoid anyway), overflowing the Output
fifo is incorrect and would end up with a broken shader. This means
that we need to know how many TMU register writes are required
to emit a TMU operation and use that information to decide if we need
to flush pending TMU operations before we emit any register
writes for the new TMU operation.

v2: fix TMU flushing for NIR registers reads (jasuarez)

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8825>

broadcom/compiler: prepare TMU spilling code to account for TMU pipelining

Follow-up patches will implement support for TMU pipelining in the
compiler, which basically means that we will be able to have more
than one outstanding TMU operation.

Our spilling code currently relies on properly identifying the end
of a TMU sequence (since we can't emit a new TMU sequence for a spill
in the middle of an existing TMU sequence), however, that code expects
that only one TMU sequence may be outstanding, which won't be true
once we implement pipelining.

This change fixes the 'end of TMU sequence' checks to account for this
in preparation for upcoming patches.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8825>

broadcom/compiler: fix indentation with TABs

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8825>

ci: Run 'time' in the background and propagate signals to test process

Simply exec'ing time didn't produce any output from it when a test
timed out.

Fixes: 35f59e14f833 "ci: Use GNU time as meson test wrapper"
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8830>

nvc0/ir: Initialize SchedDataCalculator members in constructor.

Fix defects reported by Coverity Scan.

uninit_member: Non-static class member score is not initialized in this constructor nor in any functions that it calls.
uninit_member: Non-static class member prevData is not initialized in this constructor nor in any functions that it calls.
uninit_member: Non-static class member prevOp is not initialized in this constructor nor in any functions that it calls.

Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8604>

nvc0/ir: Initialize NVC0LoweringPass member gpEmitAddress.

Fix defect reported by Coverity Scan.

Uninitialized pointer field (UNINIT_CTOR)
uninit_member: Non-static class member gpEmitAddress is not initialized
in this constructor nor in any functions that it calls.

Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8605>

amd: Disable surface modifier test on Windows

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8855>

radv: Implement VK_KHR_zero_initialize_workgroup_memory.

Reuses the pass that was implemented for ANV.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8751>

spirv: Fail when parsing invalid Initializers

Fail when parsing Initializers used in Variables with Storage Classes
that doesn't support it.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8820>

zink: don't pick a cpu device ever.

This goes down the list and picks the first non-cpu device, when
we merge the CI patch we should add a forcing env var in here.

Fixes: 8d46e35d1 ("zink: introduce opengl over vulkan")
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8845>

winsys/amdgpu: expand the slab allocation range to [256 B, 1 MB]

This increases the wasted memory to 140 MB for DeusExMD, still below
the original number. The advantage is that we now get 2 MB pages for more
buffers and fewer total buffers allocated by the kernel, enabling faster
GPU page translation and slightly lower kernel overhead.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8683>

winsys/amdgpu,pb_slab: add slabs with 3/4 of power of two sizes to save memory

Instead of aligning slab allocations to powers of two (e.g. 129K -> 256K),
implement slab allocations with 3/4 of power of two sizes to reduce
overallocation. (e.g. 129K -> 192K)

The limitation is that the alignment must be 1/3rd of the allocation size.

DeusExMD allocates 2.1 GB of VRAM. Without this, slabs waste 194 MB due
to alignment, i.e. 9.2%. This commit reduces the waste to 102 MB, i.e. 4.9%.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8683>

winsys/amdgpu: clean up slab alignment code, handle small buffers better

The next commit will build upon this.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8683>

winsys/amdgpu,radeonsi: add HUD counters for how much memory is wasted by slabs

Slabs always allocate the next power of two size from their pools. This
wastes memory if the size is not a power of two.

bo->base.size is overwritten because the default is the allocated power of
two size, but we need the real size to compute the wasted size in
amdgpu_bo_slab_destroy. entry_size is added to the hole in pb_slab_entry
to hold the real entry size.

Like other memory stats, no atomics are used.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8683>

radv: fix waiting on the last enabled RB for occlusion queries

Wait on the last enabled RB, not the last RB. This fixes GPU hangs
because the GPU was waiting forever.

Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4212
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8836>

freedreno: Put an upper limit on VSC size

Left unchecked, an app that just did an endless series of draws could
result in VSC buffer sizes >4GB, which doesn't work out well.

This limit is semi-arbitrary (ie. it is lower than hw limit, but 32*8MB
seems a bit excessive and not a limit that you'd hit in the real world).

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8842>

freedreno/decode: Fix overflow

CP_SET_DRAW_STATE state-groups count as a 4th level of IB.  Fixes a
crash seen on 32b/arm builds of crashdec.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8842>

turnip: fix resolve MSAA D32_SFLOAT_S8_UINT image to S8_UINT

According to VK_KHR_depth_stencil_resolve spec (see
VUID-VkSubpassDescriptionDepthStencilResolve-pDepthStencilResolveAttachment-03182):

"If the VkFormat of pDepthStencilResolveAttachment has a stencil
component, then the VkFormat of pDepthStencilAttachment must have a
stencil component with the same number of bits and numerical
type"

The issue with D32_SFLOAT_S8_UINT format is that it is implemented as
two planes, so we need to execute the separate_stencil path in
tu_emit_blit() to resolve its stencil component into S8_UINT image.

Fixes the following tests:

dEQP-VK.renderpass2.depth_stencil_resolve.image_2d_32_32.samples_2.d32_sfloat_s8_uint.compatibility_depth_zero_stencil_zero_testing_stencil
dEQP-VK.renderpass2.depth_stencil_resolve.image_2d_32_32.samples_2.d32_sfloat_s8_uint_separate_layouts.compatibility_depth_zero_stencil_zero_testing_stencil

Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Reviewed-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8527>

turnip: fix resolve MSAA D24_UNORM_S8_UINT image to S8_UINT

According to VK_KHR_depth_stencil_resolve spec (see
VUID-VkSubpassDescriptionDepthStencilResolve-pDepthStencilResolveAttachment-03182)

"If the VkFormat of pDepthStencilResolveAttachment has a stencil
component, then the VkFormat of pDepthStencilAttachment must have a
stencil component with the same number of bits and numerical type"

That means that we can resolve MSAA depth/stencil to a stencil only
image only if the stencil component matches with same number of bits
and type.

Although the driver only supports VK_RESOLVE_MODE_SAMPLE_ZERO_BIT
resolve mode, it was doing a sample average when resolving a MSAA
D24_UNORM_S8_UINT image to S8_UINT.

Fixes the following tests:

dEQP-VK.renderpass2.depth_stencil_resolve.image_2d_32_32.samples_2.d24_unorm_s8_uint.compatibility_depth_zero_stencil_zero_testing_s
tencil
dEQP-VK.renderpass2.depth_stencil_resolve.image_2d_32_32.samples_2.d24_unorm_s8_uint_separate_layouts.compatibility_depth_zero_stenc
il_zero_testing_stencil

Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Reviewed-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8527>

i965: use aligned malloc for context instead of ralloc

Fixes: 3175b63a ("mesa: don't allocate matrices with malloc")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4118
Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8805>

intel/perf: introduce additional ralloc context parameter

This makes it possible to use a separate ralloc context, not gl context
itself which might not be allocated with ralloc.

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8805>

intel/perf: cleanup, remove duplicate function declaration

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8805>

radv: Fix assert.

Fixes: 7f7da82dbb7 ("radv: Add image layout with drm format modifiers.")
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8828>

radv: Add modifier fails for CTS bug.

Fixes: 58e52326254 ("radv: Enable DRM format modifiers on GFX9+.")
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8828>

radv: memset the alignment hole in cache_entry to 0

Detected using valgrind. Otherwise these bytes at the end
will be touched by zstd compression, spamming valgrind output.

Other option is to do full memset(entry, 0, size),
but that is somehow unnecessary and suboptimal.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8229>

turnip: fix UINT64_MAX size wrapping in tu_GetBufferMemoryRequirements()

tu_GetBufferMemoryRequirements() ends up wrapping the UINT64_MAX size
to 0 when aligning.

Fixes:

   dEQP-VK.api.buffer.basic.size_max_uint64

Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Reviewed-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4493>

turnip: set sparseAddressSpaceSize to zero

According to Vulkan spec, "Table 46. Required Limits", as sparse
binding is unsupported, we need to return unsupported limit for
sparseAddressSpaceSize, which is zero.

Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Reviewed-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4493>

pan/decode: Prefer sizeof to ARRAY_SIZE for char

One less macro.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8829>

pan/decode: Remove tile range validation

Fault pointer works as you expect so it's not terribly useful.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8829>

pan/mdg: Drop unused stage parameter to disassembler

No longer used but was adding a dependency on compiler/shader_enums.h

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8829>

pan/decode: Remove mesa header dependencies

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8829>

pan/decode: Remove unused disasm stats

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8829>

pan/decode: Remove pandecode_prop

For pre-GenXML printing.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8829>

pan/decode: Simplify tiler printing

Again, most of this is either wrong or doesn't really matter.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8829>

pan/decode: Remove unused MEMORY_PROP macro

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8829>

pan/decode: Deduplicate shader property printing

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8829>

pan/decode: Deduplicate SFBD blend printing

Annoying verbosity of traces on midgard.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8829>

pan/decode: Remove dependency of decoder on the encoder

Obstacle to decouple panwrap from the rest of mesa.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8829>

pan/decode: Remove tiler size checks

Bad dependency and also mostly speculation at the time of writing.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8829>

pan/decode: Be explicit when printing invocations

Our "canonical" forms weren't really... good.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8829>

turnip: add missing register write to disable dithering

This was causing rendering issues with low precision formats because GL
driver can enable it.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8707>

turnip: don't always use 3d ops for blit_image

Revert this accidentally committed testing change.

Fixes: 872c4bcd27db ("turnip: implement z-scaling and z-mirroring BlitImage")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8707>

turnip: IMAGE_FILTER_{LINEAR,CUBIC}_BIT only for non-integer formats

Avoid CTS trying to use linear filtering for integer formats.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8707>

turnip: use vk_format_is_int to disable COLOR_ATTACHMENT_BLEND_BIT

This is simpler and easier to understand.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8707>

turnip: delete unused vk_format_parse.py file

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8707>

turnip: fix logicOp

Don't ignore logic op for integer formats.

Blend also doesn't need this path, because it isn't valid for blendEnable
to be true for integer formats.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8707>

radv: correctly enable WGP_MODE for tessellation control

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8811>

radv: correctly enable WGP_MODE for NGG and GS

Previously, we would set WGP_MODE on GFX10+ and then only on GFX10.
Because we used bitwise or, the result was WGP_MODE being set on GFX10+.

We also set the wrong bit, S_00B848_WGP_MODE instead of S_00B228_WGP_MODE.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8811>

radv: round-up num_records division in radv_flush_vertex_descriptors

Vertex attribute bounds checking is supposed to be done per-attribute:
   is_oob = index * stride + attrib_offset + attrib_size > buffer_size
but we were obtaining num_records by dividing the buffer size by the
stride, making it per-vertex:
   is_oob = index * stride + (stride - 1) >= buffer_size

An example from Dead Cells (Wine) is:
attribute bindings: 0, 1, 2
attribute formats: r32g32, r32g32, r32g32b32a32
attribute offsets: 0, 0, 0
binding buffers: all the same buffer
binding offsets: 0, 8, 16
binding sizes: 128, 120, 112
binding strides: 32, 32, 32

Workaround this issue without switching to per-attribute descriptors by
rounding up the division. This is still incorrect, but it should now no
longer consider in-bounds attributes out-of-bounds.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3796
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4199
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8835>

radv: Use typed outarray API

MSVC cannot perform GCC __typeof__ for C code. (C++ has decltype.)

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8818>

ac: Remove unnecessary header

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8819>

aco/ra: Add helper to get a PhysRegInterval for the register demand

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8737>

gallium/u_vbuf: skip non-indirect draws with 0 vertices

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8770>

Revert "gallium/u_vbuf: skip draws with 0 vertices"

This reverts commit be8d811e57973e9d3632f90e47fda1f5c24ca379.

Fixes: be8d811e57973e9d3632f90e47fda1f5c24ca379
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4184

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8770>

anv: Implement VK_KHR_zero_initialize_workgroup_memory

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8708>

nir: Add nir_zero_initialize_shared_memory

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8708>

spirv: Recognize zero initializers in Workgroup variables

This will be used to implement
VK_KHR_zero_initialize_workgroup_memory.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8708>

spirv: Refactor variable initializer code

Pass the vtn_value and let vtn_create_variable do the validation.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8708>

lima: always set stride in texture descriptor

We can just always specify the stride parameter regardless of whether
an alignment was forced or not. This fixes some issues where it is not
straightforward to detect the need to specify stride by checking the
buffer width (e.g. imported dmabuf to be used as texture).

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8549>

lima: enable r and rg pixel formats again

Enable r and rg targets to allow r and rg so that lima exposes
GL_EXT_texture_rg.
This is notably required by programs working with textures for
video playback.

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8549>

lima: set yuv formats as external_only

lima is not able to use yuv textures directly.
Set them as external_only so that drivers don't attempt to send yuv
planes directly as dma bufs.

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8549>

nir: Drop the lower_mem_constant_vars declaration

The function was removed in c730ace12b51d46585fac6710fbe57dccd67071c.

Fixes: c730ace12b5 "nir,clover: Drop nir_lower_mem_constant_vars"
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8834>

clover/api: make use of validate_mem_migration_flags in clEnqueueMigrateMemObjects

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6401>

nouveau: add support for SVM migrate

v2 (Ralph): don't allign address as the kernel handles that already
            support migration from GPU to system RAM
v3 (Karol): use DIV_ROUND_UP for sizes not being page aligned

Signed-off-by: Jérôme Glisse <jglisse@redhat.com>
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6401>

clover: implement clEnqueueSVMMigrateMem

Memory migration for SVM (share virtual memory). This allow to migrate
a range of virtual address of the process to device memory to speed up
device processing when that memory is in use by the device.

v2 (Karol): use tracked SVM allocation in order to support cases where
            the size of the migration is not specified

Signed-off-by: Jérôme Glisse <jglisse@redhat.com>
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6401>

clover: track allocated svm pointers

We need those to proper validate the SVM API.

v2: use std::map instead of std::unordered_map
v3: guard against segfaults on std::prev with empty containers

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6401>

gallium: add support for SVM (Share Virtual Memory) migrate

v2 (Karol): Fix declaration of pointers argument
v3 (Karol): Move flags into function interface as bools

Signed-off-by: Jérôme Glisse <jglisse@redhat.com>
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Acked-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6401>

zink: wrap some long lines

These lines are very long, let's wrap them a bit.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8832>

zink: refactor vertex-order emitting

Using a ternary expression inside the argument list avoids some
repetition, showing that the rest of the call is the same. This increase
readability a tad.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8832>

zink: add a get_spacing-helper

Similarly to the previous commit, this makes the code a bit easier to
read.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8832>

zink: add a get_primitive_mode-helper

This just makes the code a bit easier to read, where the details are
hidden slightly.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8832>

zink: do not insist shaders come from glsl

We don't know what shading-language our shaders come from. This
information is lost before we get here. So let's not declare that these
come from GLSL shaders, even though that's likely to be the case.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8832>

zik: correct spir-v caps for textures and images

There's a couple of minor issues with these the way they where:

1. All shader stages can perform texturing. So let's check for textures
   instead of fragment-shader
2. Not all implementations support StorageImageExtendedFormats. We don't
   expose image support for those implementations, but we shouldn't
   enable the cap just because we're texturing.

Both of these issues can be tackled by splitting these into texturing
and images conditionals.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8832>

zink: only emit cap when needed

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8832>

zink: only emit SpvCapabilityDerivativeControl when needed

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8832>

zink: correct return-type for function

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8832>

zink: be more careful about limits when unsupported

These limits are dependent on feature caps, so let's try to thread
a bit more carefully when the cap is unsupported.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8832>

zink: remove stale TODO

By now, I think we're fairly sure about this one ;)

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8832>

nir: Add some ssa-only fast-paths for nir_src rewrite

Basically every pass in NIR uses nir_ssa_def_rewrite_uses which calls
nir_instr_rewrite_src which is fairly complex because it handles all
sorts of non-SSA cases.  Since we already know a priori that every
source written by nir_ssa_def_rewrite_uses is SSA, we can check new_src
once at the top of the function and cut out all that complexity.

While we're at it, we expose a new SSA-only nir_ssa_def_rewrite_uses_ssa
helper which takes an SSA def which avoids the one SSA check.  It's also
more convenient 90% of the time.

Compile time as tested by Rhys Perry <pendingchaos02@gmail.com>

    Difference at 95.0% confidence
        -797.166 +/- 418.649
        -0.566174% +/- 0.296441%
        (Student's t, pooled s = 325.459)

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8790>

va/picture : Added failure check for stability

This patch created to exit from for loop incase handle function
returns error vaStatus. This will help to capture the correct
failure return to application.

Signed-off-by: SureshGuttula <sguttula@amd.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8679>

anv: switch intel perf queries to query layout

Apart from the single additional marker field, these queries will now
use the same layout as all other drivers.

This should allow us to modify a single component to add an additional
register for new metrics.

v2: Capture the query beging registers in reverse order to ensure
    timestamp is as close as possible from measured draw call.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6518>

anv: switch khr perf query code to use query layout

This unifies performance data gathering between the GL & Vulkan
drivers.

v2: Also move all NOOPs to before the query, leaving none inside

v3: Capture the query beging registers in reverse order to ensure
    timestamp is as close as possible from measured draw call.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6518>

anv: compute commands required to implement perf queries

We'll use this later to try to limit the number of NOOPs emitted for
self modifying batches.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6518>

intel/perf: drop the special READ_REG operator

Makes things a bit more uniform.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6518>

intel/perf: add DG1 support

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Acked-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6518>

intel/perf: add RKL support

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Acked-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6518>

intel/dev: identify rocketlake

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6518>

intel/perf: break TGL perf configs in GT1/2

Programming and equations are different enough that we really need 2
files.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Acked-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6518>

intel/dev: identify tigerlake

We'll need that to pick the right query sets between TGL/RKL/DG1.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6518>

intel/perf: add async compute metrics

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Acked-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6518>

intel/perf: small ICL equation refactor

No functional changes.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6518>

intel/perf: update files from IGT

IGT has received a bunch of updates, this is resyncing the files with
it.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Acked-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6518>

intel/perf: remove reordering script

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6518>

intel/perf: reorder xml files

Make the file match the order of the ones from IGT (which have changed
because of python2->3 transition).

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Acked-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6518>