nir: Check against combined alignment in nir_lower_mem_access_bit_sizes

Checking against align_mul is insufficient if align_offset > 0.  We need
to check against the combined alignment instead.

Fixes: 2e2d7803c7f1 ("nir: Add a load/store bit size lowering pass")
Reviewed-by: M Henning <drawoc@darkrefraction.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21524>

mesa/framebuffer: fix gl_framebuffer.resolve refcnt imbalance

Indeed, "resolve" is not freed at the gl_framebuffer destroy
stage.

For instance, this issue is triggered and detected with
"piglit/bin/fbo-depthstencil clear default_fb -samples=2 -auto"
while setting GALLIUM_REFCNT_LOG=refcnt.log.

Fixes: f5bde99cbdd2 ("gallium: plumb resolve attachments through from frontends -> pipe_framebuffer_state")
Signed-off-by: Patrick Lerda <patrick9876@free.fr>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21663>

radeonsi: assume shader is never NULL in si_emit_shader_*

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21641>

radeonsi: simplify encoding VGPRS and SGPRS

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21641>

radeonsi: check the pm4.reg_va_low_idx assertion unconditionally

This is not a hot path. We can always do this.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21641>

radeonsi: add si_pm4_set_reg_va to simplify setting reg_va_low_idx for RGP

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21641>

radeonsi: reindent code in si_state_binning.c

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21641>

radeonsi: don't merge SET_* packets that have a different index in si_pm4_state

Oops.

Fixes: c8e2c6faf64 ("radeonsi: use SET_SH_REG_INDEX with index=3 for registers containing CU_EN")

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21641>

radeonsi: reorganize si_emit_framebuffer_state for better readability

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21641>

radeonsi: remove Smart Access Memory because CPU access has large overhead

Related: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8176

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21641>

radeonsi/ci: update flakes and gfx8-polaris11 results

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21641>

amd/rtld: allow 64K LDS for all shader stages except for gfx6

Gfx6 can only use 32K LDS per workgroup.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21641>

amd: massively simplify how info->spi_cu_en is applied

Instead of having ac_set_reg_cu_en that sets the register, replace it with
ac_apply_cu_en that only returns the modified register value,
which allows a large simplification in both drivers because a lot of code
becomes duplicated after it's switched to ac_apply_cu_en.

RADV also didn't apply it to a few registers. Fixed.

This removes 82 lines of code in total.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21641>

ac/nir: don't use load_esgs_vertex_stride_amd on gfx6-8

An improvement for 9f1e6d8f70a8fa2c174e0070c4331f5f178e6f1.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21641>

amd: query cache sizes from the kernel

Also rename l1_cache_size -> tcp_cache_size. L1 means shader array cache.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21641>

amd,radeonsi: change enabled_rb_mask to 64 bits

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21641>

amd: update amdgpu_drm.h

From kernel commit 817714d9665e.

Acked-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21641>

asahi: Pull device name from device struct

This isn't filled in yet, but will be once the UAPI init function is
added.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21662>

asahi: Add agx_bo_mmap() calls to transfer path

We have the prototype for this already.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21662>

asahi: Add result buffer to context/batches

The result buffer is where the kernel places statistics and fault
information after the GPU executes a command. Dummy structure pending
UAPI.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21662>

asahi: Add agx_debug_fault() helper

We expect to forward GPU fault information to userspace. Since Mesa can
get that information, we can look up the fault address to log what was
the containing or nearest BO. Add a helper for that, so it can be called
from the driver.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21662>

asahi: Add APIs for DMA-BUF sync file import/export

These are generic ioctls, so it is safe to add them now.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21662>

asahi: Implement Linux driver scaffolding, sans UAPI

With macOS support out of the way, we can start implementing a lot of
the Linux driver interface and bookkeeping without actually adding the
UAPI proper. Let's do that to reduce the size of the UAPI patchset.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21662>

asahi: Align device submission API with upcoming UAPI

Nothing implemented, but this lets us get the batch tracking bits in,
including explicit sync/DMA-BUF integration which uses generic ioctls.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21662>

asahi: Add nocluster,sync,stats debug flags

These are only useful with the upcoming Linux UAPI, but there's no harm
in getting the debug scaffolding in now.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21662>

asahi: Drop macOS backend

This might be useful in the future, but it is best reimplemented in
terms of the upcoming Linux UAPI instead of having parallel codepaths.
Let's drop it.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21662>

venus: ensure invariance of buffer memory requirement size

Need to align the size for the initial cache miss.

Fixes: ef255444c19 ("venus: switch to lazy VkBuffer cache")

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21644>

mesa: initialize VertexProgram._VaryingInputs before the first use

Noticed by code inspection.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21438>

mesa: remove a redundant call to _mesa_update_edgeflag_state_vao

It's called again a few lines later.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21438>

mesa: fix glPopClientAttrib with fixed-func VP and zero-stride varyings

This was missed.

Fixes: 3a294ff01fb9d1d8b - mesa: move the _mesa_set_varying_vp_inputs call to where the state changes
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8246

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21438>

vulkan/wsi/wayland: Fix destruction of event queue with proxies still attached.

Destroy the surface dmabuf feedback proxy before destroying the event
queue that the proxy is attached to.

This silences a warning that libwayland 1.22 emits for programs that use
Vulkan/Wayland:

warning: queue 0x557a4efbcf70 destroyed while proxies still attached:
  zwp_linux_dmabuf_feedback_v1@18 still attached

Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21647>

egl/wayland: Fix destruction of event queue with proxies still attached.

Destroy the display wrapper proxy before destroying the event queue that
the proxy is attached to.

This silences a warning that libwayland 1.22 emits for programs that use
EGL/Wayland:

warning: queue 0x562a5ed2cd20 destroyed while proxies still attached:
  wl_display@1 still attached

Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21646>

anv: fix scratch buffer reloc in 3DSTATE_HS

We need to have the scratch buffer added to the pipeline BO tracking
list, so it's added to the batch buffer and finally to the execbuffer
list. Otherwise we pagefault (or read the default scratch page on
i915).

Fixes
dEQP-VK.subgroups.ballot_broadcast.graphics.subgroupbroadcast_u16vec4
on CI (and probably other tests).

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 2028f1caa385 ("anv: emit 3DSTATE_HS in cmd_buffer_flush_gfx_state")
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21653>

hasvk: Disable non-zero fast clears for 8xMSAA images

Using texelFetch to read samples from an 8xMSAA fast cleared image on
Haswell can read transparent black pixels around triangles from where
there should be none. This issue isn't present when using sample
shading, resolving the image using vkCmdResolveImage or in a copy the
image. The easiest way to fix this is by just disabling non-zero fast
clears for 8xMSAA images.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7587

Cc: mesa-stable
Reviewed-by: Filip Gawin <filip@gawin.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21444>

radv: remove is_internal pipeline creation parameter

Instead, check if the cache is the meta shader cache. This catches the
shaders created by radv_create_radix_sort_u64().

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21606>

pvr: Advertise STORAGE_IMAGE_BIT for B10G11R11_UFLOAT_PACK32

For optimalTilingFeatures we should be advertising
STORAGE_IMAGE_BIT for B10G11R11_UFLOAT_PACK32 too.

Signed-off-by: Karmjit Mahil <Karmjit.Mahil@imgtec.com>
Reviewed-by: Frank Binns <frank.binns@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21649>

pvr: Don't advertise currently unsupported features

This commit removes the advertising of features that are currently
unsupported by the driver and aren't strictly necessary for
Vulkan 1.0.

Signed-off-by: Karmjit Mahil <Karmjit.Mahil@imgtec.com>
Reviewed-by: Frank Binns <frank.binns@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21649>

radv: fix DCC decompress on GFX11

The hardware requires one color output to be set by CB registers,
otherwise the DCC decompression does nothing.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8127
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8175
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8370
Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21629>

radv: Use common helpers to translate format in SDMA copy.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21585>

radv: Remove SDMA padding from copy helpers.

These are handled in winsys already; no need to duplicate and complicate
the code paths.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21585>

radv: SDMA v4 size field is size - 1

After cross-checking with kernel and the old buffer copy code, it seems
that the size field should be size - 1 instead.

Fixes: 7b5ab48c40b ("radv: partial sdma support")

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21585>

radeonsi: SDMA v4 size field is size - 1

After cross-checking with kernel and the old buffer copy code, it seems
that the size field should be size - 1 instead.

Fixes: 46c95047bd7 ("radeonsi: implement si_sdma_copy_image for gfx7+")

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21585>

anv/hasvk: speed up null image/view descriptor writes

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reported-by: Chuansheng Liu <chuansheng.liu@intel.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21642>

Revert "iris: Avoid abort() if kernel can't allocate memory"

This reverts commit f9d8d9acbb6a620684fb4dac4affe25816587d92.

Now ENOMEM is handled in submit_batch(), we don't need to check it for
resetting anymore.

Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20449>

iris: Retry DRM_IOCTL_I915_GEM_EXECBUFFER2 on ENOMEM

We are seeing endless DRM_IOCTL_SYNCOBJ_WAIT ioctl when system memory is
under pressured.

Commit f9d8d9acbb6a ("iris: Avoid abort() if kernel can't allocate
memory") avoids the abort() on ENOMEM by resetting the batch. However,
when there's an ongoing OpenGL query, resetting the batch will make the
snapshots_landed never be flipped, so iris_get_query_result() gets stuck
in the while loop forever.

Since there's no guarantee that the next batch after resetting won't hit
ENOMEM, so instead of resetting the batch, be patient and wait until kernel has
enough memory. Once the batch is submiited and snapshots_landed gets
flipped, iris_get_query_result() can proceed normally.

Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/6851
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20449>

intel/compiler: add comment about workaround on simd width

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21619>

radv: use new EVENT_WRITE_ZPASS packet3 on GFX11

Ported from RadeonSI.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21621>

radv: ignore alpha_is_on_msb on GFX11 because the hw ignores it

Ported from RadeonSI.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21621>

nir/lower_mediump: don't use fp16 for constants if the result is denormal

Image stores are not required to preserve denorms.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21622>

aco: use v_fma_mix_f32 for v_fma_f32 with 2 fp16 representable, different literals

We can pack two fp16 literals into one 32bit literal and use opsel to select
the correct value. Note that LLVM currently disassembles these instructions
incorrectly.

Foz-DB Navi21:
Totals from 13365 (9.91% of 134913) affected shaders:
VGPRs: 840880 -> 840016 (-0.10%); split: -0.11%, +0.01%
SpillSGPRs: 724 -> 722 (-0.28%)
CodeSize: 82439364 -> 82451336 (+0.01%); split: -0.06%, +0.08%
MaxWaves: 244858 -> 244980 (+0.05%)
Instrs: 15265976 -> 15247201 (-0.12%); split: -0.13%, +0.01%
Latency: 223316180 -> 223272495 (-0.02%); split: -0.03%, +0.02%
InvThroughput: 41981375 -> 41969917 (-0.03%); split: -0.04%, +0.01%
VClause: 266775 -> 266558 (-0.08%); split: -0.14%, +0.06%
SClause: 646602 -> 645996 (-0.09%); split: -0.16%, +0.07%
Copies: 794703 -> 776075 (-2.34%); split: -2.46%, +0.12%
Branches: 296317 -> 296316 (-0.00%)
PreSGPRs: 658796 -> 656479 (-0.35%); split: -0.35%, +0.00%
PreVGPRs: 744014 -> 743679 (-0.05%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20587>

aco: mark mad definition as precise if the mul/add were precise

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20587>

amd,ac/rgp: fix SQTT memory types

This crashed on Steam Deck because the memory type is LPDDR5 and it
wasn't not handled in the switch. It seems the kernel changed the
memory type returned for VanGogh because it used to work.

Fixes: aef7ea868fe ("ac/gpu_info: handle LPDDR4 and 5 in ac_memory_ops_per_clock")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21627>

gallivm: Add some notes about other invocation_0_must_be_active usages.

So that the next person trying to cut down LLVM compile times doesn't trip
over this.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21142>

gallivm: Use first_active_invocation for scalar SSBO loads.

Again, this should reduce the complexity of the LLVM IR we emit in some
cases.  We don't use it for shared loads, due to the noted corner case.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21142>

gallivm: Use first_active_invocation for ubo/kernel memory loads.

If we're just loading memory, we can take the scalar offset_is_uniform
paths even the first active invocation is nonzero, saving a bunch of
looping and bounds checking for per-element loads.  And, if we don't have
an active invocation, doing the load for element 0 (which is
bounds-checked to return 0 if element 0 had a bad value in it) before
throwing away the result is still better than doing bounds-checked loads
for each element before throwing away the result.

dEQP-VK.ubo.random.16bit.scalar.92 goes from 16.5 to 14.0 seconds.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21142>

gallivm: Return 0 for first active invocation when no invocations are active.

gallivm doesn't actuially jump across branches where no invocations are
active, so my previous assertion about the exec mask being nonzero was
incorrect.  This means that we'll always use a defined invocation for the
various LLVMBuildExtractElements using the result value, which is an
improvement over my even the code before my cttz change that would use
undefined values for the element to be extracted.

Fixes: 8c2493d041c4 ("gallivm: Use cttz instead of a loop for first_active_invocation().")
Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21142>

ci/radv: remove dEQP-VK.image.sample_texture.* fails/flakes

They were fixed since commit 11b2a063bf1 ("vulkan: Unconditionally add
barriers for missing external subpass deps").

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21616>

intel/dev: generate helpers to identify platform workarounds

Workarounds for defects in Intel silicon have been manually
implemented:

 - consult defect database for the current platform

 - add workaround code behind platform ifdef or devinfo->ver checks

Some bugs have occurred due to the manual process.  Typical failure
modes:

 - defect database is updated after a platform is enabled

 - version checks are overly broad (eg gfx11+) for defects that were
   fixed (eg in gfx12)

 - version checks are too narrow for defects that were extended to
   subsequent platforms.

 - missed workarounds

This commit automates workaround handling:

 - Internal automation queries the defect database to collate and
   summarize defect documentation in json.

 - mesa_defs.json describes all public defects and impacted platforms.
   Defects which are extended to subsequent platforms are listed under
   the original defect.

 - gen_wa_helpers.py generates workaround helpers to be called
   in place of version checks:

   - NEEDS_WORKAROUND_{ID} provides a compile time check suitable for
     use in genX routines.

   - intel_device_info_needs_wa() provides a more precise runtime
     check, differentiating platforms within a generation and
     platform steppings.

Internal automation will generate new mesa_defs.json as needed.
Workarounds enabled with these helpers will apply correctly based on
updated information in Intel's defect database.

Reviewed-by: Dylan Baker <dylan@pnwbakers>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20825>

util: add macro to support gcc/clang poison

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20825>

intel/dev: create a helper dependency for libintel_dev

This ensures that users of libintel_dev.a won't be compiled until
include files are generated, and that they are recompiled when the
header changes.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Mark Janes <markjanes@swizzler.org>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20825>

glthread: fix a perf regression due to draw_always_async flag, fix DrawIndirect

Performance regressed by 31% in one VP2020/Creo subtest because
the draw_always_async flag wasn't implemented correctly. Remove it
instead of fixing it.

While removing it, I noticed that our DrawIndirect async conditions
were incorrect. I fixed them.

Fixes: 3b897719e632a165f - glthread: add ctx->GLThread.draw_always_async to simplify draw checking

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21566>

glx: fix indirect initialization crash

Fixes: b090246a ("glx: Only compute client GL extensions for indirect contexts")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8393

Reviewed-by: Adam Jackson <ajax@redhat.com>
Signed-off-by: Illia Polishchuk <illia.a.polishchuk@globallogic.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21591>

d3d12: Use memcmp for full tcs/gs variant keys

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21610>

d3d12: Cache varying info to reduce compare/copy cost

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21610>

d3d12: Compare shader key common parts with memcmp, instead of if's

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21610>

d3d12: Compare shader keys with union-encompassing fields all at once

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21610>

d3d12: Compare shader keys with a switch, instead of cascading if's

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21610>

anv: use the parameter passed to the macro

The two points calling this macro pass dyn->rs.provoking_vertex to it,
which is why it works, but it's cleaner to use the parameter instead.

Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21613>

intel/mi: use 64bit constant for bitshift

Coverity complains that we could end up rolling over on a 32bit
platform, which isn't really true because of the assertion, but there's
also no harm in ensuring that we have exactly the same behavior for both
32 bit and 64 bit platforms.

CID: 1515989
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21572>

venus: fix VK_EXT_image_view_min_lod feature query

Fixes: 8b810985196 ("venus: enable VK_EXT_image_view_min_lod")

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21617>

pvr: Setup SPM background object

Signed-off-by: Karmjit Mahil <Karmjit.Mahil@imgtec.com>
Reviewed-by: Frank Binns <frank.binns@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21575>

glsl: remove do_copy_propagation_elements() optimisation pass

Since 13b859de do_copy_propagation_elements() has a flaw where
the time it takes to complete grows exponentially slowers as the number
of nested loops increases. It can also hurt rather than help verses
just letting NIR optimise the code. So if the NIR linker is enabled we
let it handle it instead.

shader-db results Iris (BDW):

total instructions in shared programs: 11177181 -> 11199739 (0.20%)
instructions in affected programs: 119424 -> 141982 (18.89%)
helped: 109
HURT: 65
total cycles in shared programs: 368946819 -> 372277173 (0.90%)
cycles in affected programs: 116539428 -> 119869782 (2.86%)

total spills in shared programs: 3983 -> 8785 (120.56%)
spills in affected programs: 2072 -> 6874 (231.76%)
helped: 0
HURT: 6

total fills in shared programs: 2016 -> 6068 (200.99%)
fills in affected programs: 230 -> 4282 (1761.74%)
helped: 0
HURT: 6

LOST:   85
GAINED: 77

freedreno results:

total instructions in shared programs: 11011122 -> 11011620 (<.01%)
instructions in affected programs: 939829 -> 940327 (0.05%)
total full in shared programs: 762725 -> 762674 (<.01%)
full in affected programs: 1096 -> 1045 (-4.65%)
total constlen in shared programs: 1772092 -> 1771596 (-0.03%)
constlen in affected programs: 2780 -> 2284 (-17.84%)
total stp in shared programs: 4040 -> 4058 (0.45%)
stp in affected programs: 3656 -> 3674 (0.49%)
total ldp in shared programs: 2160 -> 2178 (0.83%)
ldp in affected programs: 1748 -> 1766 (1.03%)
stp HURT:   shaders/robclark-shaders/gfxbench5/gl_5_high_off/13.shader_test CL: 1231 -> 1234 (0.24%)
stp HURT:   shaders/robclark-shaders/gfxbench5/gl_5_normal_off/13.shader_test CL: 1231 -> 1234 (0.24%)
stp HURT:   shaders/robclark-shaders/gfxbench5/gl_5_high_off/15.shader_test CL: 453 -> 456 (0.66%)
stp HURT:   shaders/robclark-shaders/gfxbench5/gl_5_normal_off/15.shader_test CL: 453 -> 456 (0.66%)
stp HURT:   shaders/robclark-shaders/gfxbench5/gl_5_high_off/17.shader_test CL: 144 -> 147 (2.08%)
stp HURT:   shaders/robclark-shaders/gfxbench5/gl_5_normal_off/17.shader_test CL: 144 -> 147 (2.08%)

however, those stp counts are misleading -- gfxbench gl-5-normal actually
gets its scratch (ldp/stp) stored as 16 bits instead of 32 thanks to
better NIR copy prop, and the result is 2.64398% +/- 0.0991923% perf
improvement!

i915 results:

total instructions in shared programs: 510528 -> 510489 (<.01%)
instructions in affected programs: 3303 -> 3264 (-1.18%)
total tex_indirect in shared programs: 16708 -> 16717 (0.05%)
tex_indirect in affected programs: 134 -> 143 (6.72%)
total temps in shared programs: 30181 -> 30169 (-0.04%)
temps in affected programs: 1268 -> 1256 (-0.95%)
LOST:   0
GAINED: 1

i915 highlights:
instructions HURT:   shaders/closed/steam/legend-of-grimrock/47.shader_test FS: 141 -> 144 (2.13%)
instructions HURT:   shaders/closed/steam/steamworld-dig/22.shader_test FS: 84 -> 108 (28.57%)
temps HURT:   shaders/closed/steam/left-4-dead-2/medium/3682.shader_test FS: 7 -> 13 (85.71%)

r300 results:

total instructions in shared programs: 1340439 -> 1340845 (0.03%)
instructions in affected programs: 32354 -> 32760 (1.25%)
total temps in shared programs: 179394 -> 179329 (-0.04%)
temps in affected programs: 1505 -> 1440 (-4.32%)
total consts in shared programs: 1177742 -> 1177885 (0.01%)
consts in affected programs: 1107 -> 1250 (12.92%)
total lits in shared programs: 24992 -> 25019 (0.11%)
lits in affected programs: 138 -> 165 (19.57%)
instructions HURT:   shaders/closed/steam/legend-of-grimrock/26.shader_test FS: 47 -> 52 (10.64%)
instructions HURT:   shaders/closed/steam/sanctum-2/6072.shader_test FS: 43 -> 48 (11.63%)
instructions HURT:   shaders/closed/steam/champions-of-regnum/2378.shader_test VS: 35 -> 40 (14.29%)

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13288>

nir/split_64bit_vec3_and_vec4: Handle 64-bit matrix types.

The offset handling should already work for flattening to our split vars,
just need to make sure we have enough (or any!) array elements.

Fixes: #7154
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13288>

spirv: Add skip_os_break_in_debug_build option to use in unit tests

When running in the CI environment, instead of crashing the test
binary, it is preferable to just fail gracefully (in this case return
a NULL shader) like is done in release mode, so other tests continue
to be executed.

For convenience add a variable break_on_failure to the test so the
breaking behavior can be re-enable in individual tests when debugging.

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21512>

spirv/tests: Add some basic control flow tests

The DISABLED test currently fails parsing.

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21512>

spirv/tests: Parametrize stage in get_nir() helper

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21512>

spirv/tests: Add script to generate C array from SPIR-V source

This is useful for generating the C code to embed the SPIR-V
when adding a new test.

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21512>

spirv/tests: Subclass spirv_test helper to namespace the tests

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21512>

ci: implement unified sections

in after_script, variable $SCRIPTS_DIR is lost

Acked-by: Daniel Stone <daniels@collabora.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20272>

ci: alpine: install bash and coreutils for date -d

bash is needed for consistency.
coreutils is needed for `date -d`.

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20272>

ci/alpine: keep the curl inside the image

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20272>

nir/opt_algebraic: add patterns for iand/ior of feq/fneu with 0

Foz-DB Navi21:
Totals from 1245 (0.92% of 134913) affected shaders:
VGPRs: 66232 -> 66248 (+0.02%); split: -0.01%, +0.04%
CodeSize: 5874976 -> 5868168 (-0.12%); split: -0.17%, +0.05%
MaxWaves: 25278 -> 25274 (-0.02%); split: +0.01%, -0.02%
Instrs: 1087502 -> 1085267 (-0.21%); split: -0.21%, +0.00%
Latency: 6531489 -> 6531672 (+0.00%); split: -0.04%, +0.05%
InvThroughput: 1531774 -> 1532327 (+0.04%); split: -0.02%, +0.05%
VClause: 22218 -> 22202 (-0.07%); split: -0.08%, +0.00%
SClause: 45906 -> 45873 (-0.07%); split: -0.08%, +0.01%
Copies: 64004 -> 64102 (+0.15%); split: -0.24%, +0.39%
Branches: 21529 -> 21534 (+0.02%); split: -0.00%, +0.03%
PreSGPRs: 51936 -> 51850 (-0.17%)
PreVGPRs: 55393 -> 55398 (+0.01%); split: -0.02%, +0.03%

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21576>

radv: only expose EXT_pipeline_library_group_handles if RT is enabled

VK_EXT_pipeline_library_group_handles requires
VK_KHR_ray_tracing_pipeline to be enabled.

Fixes dEQP-VK.info.device_extensions.

Fixes: ed768337050 ("radv: Implement & expose VK_EXT_pipeline_library_group_handles.")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21618>

spirv: Don't specify nir_var_uniform or nir_var_mem_ubo in barriers

These are constant read-only data and don't need to be synchronized.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21517>

iris: Fix to release BO immediately if not busy

Currently the iris driver is adding the buffer objects to zombie list
without checking if it is busy or not. It checks for it after 1 second
which adds delay to buffer release.

This fix checks if the bo is busy or not before adding it to zombie list.

Without this fix, the applications expecting immediate buffer release would
fail.

The fix is identified while debugging below android cts tests:
android.graphics.cts.BitmapTest#testDrawingHardwareBitmapNotLeaking
android.graphics.cts.BitmapTest#testHardwareBitmapNotLeaking

Signed-off-by: Sai Teja Pottumuttu <sai.teja.pottumuttu@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21460>

genxml: fix border color offset field on Gfx12+

I wonder if the docs are correct for Gfx11 because this is the
generation that gave us the Bindless Sampler Heap of 4Gb. So it would
make sense that the border colors can also be placed anywhere in that
4Gb heap.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21600>

genxml: Fix STATE_BASE_ADDRESS::BindlessSurfaceStateSize field size

BSpec 44507

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21600>

radv: add video format support to format probing.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21457>

radv: Replace radv_trap_handler_shader with radv_shader.

Now that the upload memory is tied to the shader itself, the trap handler
shader no longer needs an additional wrapper.

This is a cleanup to ease introduction of a new shader uploading code path.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21541>

anv: always pick graphics queue to execute prime blits on.

This will change when we get transfer queues but this should avoid
video queues being picked by accident.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21204>

asahi: Flush USC caches on the first draw

It seems that when batches are submitted back to back, the USC can
retain cache contents between them. This causes a problem when the CPU
updates a VBO between batches, since some of those updates might not be
visible to the USC.

Looks like the VDM barrier command with one magic bit set fixes this, so
let's try that.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21538>

asahi: Identify USC cache invalidate

Signed-off-by: Asahi Lina <lina@asahilina.net>
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21538>

asahi: Add readonly BO flag

Signed-off-by: Asahi Lina <lina@asahilina.net>
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21538>

asahi: Add BO_SHAREABLE flag

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21538>

anv: remove more Gfx7 code

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21599>

iris: fix Wa_16011107343 for Gfx12

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: df9f71d79574 ("iris: emit 3DSTATE_HS for each primitive on gfx12")
Acked-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21605>

anv: fixup Wa_16011107343 for Gfx12 only

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 75968398f37c ("anv: emit 3DSTATE_HS for each primitive on gfx12")
Acked-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21605>

ci: Reuse MESA_CI_PERFORMANCE_ENABLED in performance-rules

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21492>

ci: Use workflow to make CI aware of performance jobs

With the workflow keyword, we can have more control over how pipelines
are created.

One of the features is to set a variable for the entire pipeline
depending on the rule. These variables would be available for all jobs
manifest and can be used inside job rules, for example.

We can use that to set a variable to enable performance jobs in the
pipeline, both at the YAML and script levels.

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21492>

ci: Use release builds in perf jobs

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21492>