radv: skip compilation when possible with GPL fast-linking

When all shader stages have already been imported it's possible to
skip radv_graphics_pipeline_compile() entirely. This makes GPL
fast-linking VERY fast.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21068>

radv: determine the last VGT API stage earlier

It can be computed right after the active stages are known. While we
are at it, simplify the code.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21068>

radv: stop using the graphics pipeline key after compilation

Only the blend state was relying on the graphics pipeline key. This
will allow us to skip generating it when there is no compilation at
all (for fast-linking with GPL).

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21068>

radv: return a boolean value in radv_pipeline_needs_dynamic_ps_epilog()

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21068>

radv: pass the lib flags for generating the pipeline key

No functional change.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21068>

v3dv: mark dEQP-VK.api.command_buffers.record_many_draws_secondary_2 as flaky

Signed-off-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21070>

radv: remove one unused variable in radv_graphics_lib_pipeline_init()

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21042>

radv: allow to create a noop FS in a library with GPL

Otherwise, a noop FS will be always compiled during linking if not
provided by the application and that is too slow for fast-linking.

This should be improved to use a global noop FS but it's really tricky
because NIR linking doesn't do anything when the next stage is unknown,
and hence doesn't remove unused varyings.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21042>

zink: rework descriptor buffer templating to use offsets

compute programs can be reused across contexts, which means storing any
pointers directly like this is going to lead to desync and crash

instead, make this like regular descriptor templates and calculate the offset
from the current context to ensure that everything works as it should

fixes #8201

Fixes: 7ab5c5d36d2 ("zink: use EXT_descriptor_buffer with ZINK_DESCRIPTORS=db")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21020>

asahi: Split off macOS support into its own file

All the ifdef __APPLE__ is getting really silly. Let's split off the
macOS UAPI abstraction into its own file, so we can have parallel
implementations.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21058>

asahi: Split off common BO code into its own file

In preparation for splitting off the macOS backend implementation into
its own file, pull out the shared BO code from agx_device.c into
agx_bo.c.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21058>

asahi: Use non-UAPI specific BO create flags

So we're not tied to the macOS or Linux UAPIs and are not translating awkwardly
from one to the other when creating BOs. They're not quite equivalent -- macOS
doesn't include writeback information in this flag field, and Linux doesn't have
a executable flag. (Maybe we should add one, though? Then we can enforce W^X.)

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21058>

zink/ci: allow running manual jobs again on RADV

Fixes: f6c06ef2f66a ("ci: Add manual rules variations to disable.")
Reviewed-by: Emma Anholt <emma@anholt.net>
Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21036>

ci/core-manual-rules: enclose the whole condition in quotes

Quoting a condition is apparently an effective way of working around
YAML parsing weirdness. However, the quotes need to surround the whole
expression, not just parts of it.

Fixes: f6c06ef2f66a ("ci: Add manual rules variations to disable.")
Suggested-by: Michel Dänzer <mdaenzer@redhat.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21036>

agx: Centralize texture lowering

Lowering buffer textures will interact with multiple of our existing lowerings,
and it's convenient to have it all in one place. This also keeps the pass
ordering dependencies centralized.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21060>

Revert "CI: Lima farm is offline"

This reverts commit 0733aafa2271fee6a6724467ec7f2e50754d5a9d.

Signed-off-by: Erico Nunes <nunes.erico@gmail.com>
Acked-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21035>

zink: fix zink_mem_type_idx_from_bits()

at some point this used to work, but it no longer does what it's supposed
to do, which is return a memtype from a heap+flags

Fixes: d702a503ad5 ("zink: support multiple heaps per memory type")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21025>

zink: only set VkPipelineColorBlendStateCreateInfo::attachmentCount without full ds3

this should be ignored by drivers/layers, but it isn't, and the crashing is immense

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21025>

lavapipe: try harder to reuse pipeline layouts during merge

the original code was quite conservative and always created a new layout,
but many times this is unnecessary, and the original layout can just be refcounted
since it doesn't need to be merged

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21051>

lavapipe: delete lvp_pipeline::mem_ctx

this is no longer used

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21051>

lavapipe: delete unused pipelines immediately

deferring these can cause memory ballooning and oom

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21051>

lavapipe: create gfx gallium csos at pipeline bind

this should minimize pipeline creation time and make fast-linking "fast"

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21051>

lavapipe: break out (and slightly refactor) gallium shader cso creation

there's also now a(n unused) flag to indicate that the csos have been created

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21051>

lavapipe: refcount nir shaders instead of cloning

this is just about ownership, not modification, so refcounting saves time

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21051>

lavapipe: add refcounting for shader nir

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21051>

lavapipe: move noop fs creation to device

this avoids creating a separate noop fs for every pipeline

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21051>

freedreno: support UBWC scanout

On sway+xwayland, both explicit and implicit modifiers are advertised.
While dri3proto says nothing about it, zwp_linux_dmabuf_v1 says

  A compositor that sends valid modifiers and DRM_FORMAT_MOD_INVALID for
  a given format supports both explicit modifiers and implicit
  modifiers.

"glmark2 -b build:model=bunny --fullscreen" goes from 468 to 598fps on
a618 @ 2160x1440.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20892>

freedreno: add has_implicit_modifier helper

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20892>

nir/opt_algebraic: Add optimization for ieq/ine and right-shift.

Fossil DB stats on GFX11:
Totals from 1343 (1.00% of 134913) affected shaders:
SpillSGPRs: 7145 -> 7137 (-0.11%)
CodeSize: 20737744 -> 20739148 (+0.01%); split: -0.02%, +0.03%
Instrs: 4010443 -> 4008449 (-0.05%); split: -0.05%, +0.00%
Latency: 50021520 -> 50021105 (-0.00%); split: -0.00%, +0.00%
InvThroughput: 6354371 -> 6354112 (-0.00%); split: -0.00%, +0.00%
VClause: 63035 -> 63038 (+0.00%); split: -0.01%, +0.01%
SClause: 121162 -> 121166 (+0.00%)
Copies: 251354 -> 251058 (-0.12%); split: -0.18%, +0.06%
PreSGPRs: 137283 -> 137299 (+0.01%)

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20936>

anv: Perform load_constant address math in 32-bit rather than 64-bit

We lower NIR's load_constant to load_global_constant, which uses A64
bindless messages.  As such, we do the following math to produce the
address for each load:

   base_lo@32 <- BRW_SHADER_RELOC_CONST_DATA_ADDR_LOW
   base_hi@32 <- BRW_SHADER_RELOC_CONST_DATA_ADDR_HIGH
   base@64 <- pack_64_2x32_split(base_lo, base_hi)
   addr@64 <- iadd(base@64, u2u64(offset@32))

On platforms that emulate 64-bit math, we have to emit additional code
for the 64-bit iadd to handle the possibility of a carry happening and
affecting the top bits.

However, NIR constant data is always uploaded adjacent to the shader
assembly, in the same buffer.  These buffers are required to live in a
4GB region of memory starting at Instruction State Base Address.  We
always place the base address at a 4GB address.  So the constant data
always lives in a buffer entirely contained within a 4GB region, which
means any offsets from the start of the buffer cannot possibly affect
the high bits.

So instead, we can simply do a 32-bit addition between the low bits of
the base and the offset, then pack that with the unchanged high bits.

On anv, INSTRUCTION_STATE_POOL_MIN_ADDRESS is 8GB, so the high bits are
always 0x2.  We don't even need to patch that portion of the address and
can just use an immediate value.  We do still need to pack, however.

fossil-db on Icelake indicates the following for affected shaders:

   Instrs: 10830023 -> 10750080 (-0.74%)
   Cycles: 1048521282 -> 1046770379 (-0.17%); split: -0.33%, +0.16%
   Subgroup size: 103104 -> 103112 (+0.01%)
   Send messages: 570886 -> 570760 (-0.02%)
   Loop count: 14428 -> 14429 (+0.01%)
   Spill count: 14246 -> 14244 (-0.01%); split: -0.06%, +0.04%
   Fill count: 22802 -> 22794 (-0.04%); split: -0.04%, +0.01%
   Scratch Memory Size: 654336 -> 662528 (+1.25%)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20999>

iris: Perform load_constant address math in 32-bit rather than 64-bit

We lower NIR's load_constant to load_global_constant, which uses A64
bindless messages.  As such, we do the following math to produce the
address for each load:

   base_lo@32 <- BRW_SHADER_RELOC_CONST_DATA_ADDR_LOW
   base_hi@32 <- BRW_SHADER_RELOC_CONST_DATA_ADDR_HIGH
   base@64 <- pack_64_2x32_split(base_lo, base_hi)
   addr@64 <- iadd(base@64, u2u64(offset@32))

On platforms that emulate 64-bit math, we have to emit additional code
for the 64-bit iadd to handle the possibility of a carry happening and
affecting the top bits.

However, NIR constant data is always uploaded adjacent to the shader
assembly, in the same buffer.  These buffers are required to live in a
4GB region of memory starting at Instruction State Base Address.  We
always place the base address at a 4GB address.  So the constant data
always lives in a buffer entirely contained within a 4GB region, which
means any offsets from the start of the buffer cannot possibly affect
the high bits.

So instead, we can simply do a 32-bit addition between the low bits of
the base and the offset, then pack that with the unchanged high bits.

On iris, IRIS_MEMZONE_SHADER is at [0, 4GB) so the high bits are always
zero.  We don't even need to patch that portion of the address and can
simply use u2u64 to promote the 32-bit add result to a 64-bit value
where the top bits are 0.

shader-db on Icelake indicates that this:
- Helps instructions: -1.13% in 135 affected programs
- Helps spills/fills: -4.08% / -4.18% in 4 affected programs
- Gains us 1 SIMD16 compute shader instead of SIMD8

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20999>

radv: Don't place CS in VRAM when bandwidth is too low.

People who use RADV on eGPU have reported poor performance by default.
They also noted that the "nosam" option helps.

This commit disables placing CS objects in VRAM when the bandwidth is
below that of PCIe 3.0 x8. Note that eGPUs are typically PCIe 3.0 x4.

Contributes-to: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7340
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20842>

ac/gpu_info: Add has_pcie_bandwidth_info.

This is so that we can tell whether the current kernel
has the PCIe bandwidth info available or not.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20842>

vulkan/wsi/win32: Support tearing (immediate) and VSync (FIFO) present modes

Reviewed-by: Giancarlo Devich <gdevich@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20945>

vulkan/wsi: Add a wsi_device param to get_present_modes

The Win32 WSI will want to query capabilities of the device to
determine what's available.

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20945>

intel/fs: Always stall between the fences on Gen11+

Be conservative in Gfx11+ and always stall in a fence.  Since there are
two different fences, and shader might want to synchronize between them.

This change also brings back the original code block for the stall
between the fence and comment from the commit
b390ff35170fdc2b7f1fb1709a79d81edcd56981.

v2: (Caio)
 - Re-arrange code block.
 - Adjust comment.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6958

Fixes: f7262462 ("intel/fs: Rework fence handling in brw_fs_nir.cpp")
Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Tested-by: Mark Janes <markjanes@swizzler.org>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20996>

ci: Fix perf job condition.

We were supposed to be checking that the job had "performance" in the
name, not that the user (which we already checked is marge) has
"performance" in their name.

Fixes: f6c06ef2f66a ("ci: Add manual rules variations to disable irrelevant driver jobs.")
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21002>

ci: Fix perf jobs blocking Marge pipelines.

They got accidentally disabled entirely, so they didn't block merge, but
once they re-enable then they'll block us again.  The problem was that I
moved allow_failure to a .performance-rules section, but we only ever
inherit the rules from that location, not the rest of yml.

This is basically a revert of 67547a04b602 ("ci: Move the performance
jobs' allow_failure:true to the gl rules."), though I still keep the
allow_failure in a more common location with comments, since perf jobs are
a huge trap.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21002>

radv: remove radv_pipeline_stage::spirv::sha1

This is no longer used.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21048>

radv: remove redundant zero initialization of pipeline layout

It's already zeroed in radv_pipeline_layout_init().

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21048>

radv: optimize radv_pipeline_layout_add_set() slightly

That value is already computed when a descriptor set layout is created.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21048>

venus: log upon device creation

Log the deviceName and driverInfo gated behind VN_DEBUG=log_ctx_info

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21030>

nir: mark progress when removing trailing unused load_const channels

When the unused channels were at the end and so no reswizzling was
needed, we wouldn't correctly mark the progress.

Fixes: 3305c960
Signed-off-by: Pavel Ondračka <pavel.ondracka@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21014>

nir: mark progress when removing trailing unused alu channels

When the unused channels were at the end and so no reswizzling was
needed, we wouldn't correctly mark the progress.

Fixes: cb7f2012
Signed-off-by: Pavel Ondračka <pavel.ondracka@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21014>

nir: nir opt_shrink_vectors whitespace fix

Signed-off-by: Pavel Ondračka <pavel.ondracka@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21014>

intel/compiler: use lower_image_samples_to_one

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewer-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Signed-off-by: Amber Amber <amber@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20813>

ir3: use lower_image_samples_to_one

This is necessary to properly support ARB_shader_texture_image_samples
fixes crash in KHR-GL45.shader_texture_image_samples_tests.image_functional_test

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Rob Clark <robclark@freedesktop.org>
Reviewer-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Signed-off-by: Amber Amber <amber@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20813>

nir: support lowering nir_intrinsic_image_samples to a constant load

This can be used by multiple drivers that do not support ms images

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Rob Clark <robclark@freedesktop.org>
Reviewer-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Signed-off-by: Amber Amber <amber@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20813>

radv: Fix creating accel structs with unbound buffers

If the buffer hasn't been bound to memory yet, we will dereference a
NULL pointer in radv_CreateAccelerationStructureKHR.

cc: mesa-stable

Closes: #8199
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21019>

d3d12: Honor suggested driver profile/level for H264/HEVC encode

Fixes some H264 <-> HEVC transcode cases where the wrong level/profile was assigned to the output bitstream

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21043>

aco: limit VALUPartialForwardingHazard search

Complicated CFG and lots of SALU can cause this to take an extremely long
time to finish.

Fixes
dEQP-VK.graphicsfuzz.cov-value-tracking-selection-dag-negation-clamp-loop
and Monster Hunter Rise demo compile times.

fossil-db (gfx1100):
Totals from 57 (0.04% of 134574) affected shaders:
Instrs: 170919 -> 171165 (+0.14%)
CodeSize: 860144 -> 861128 (+0.11%)
Latency: 961466 -> 961505 (+0.00%)
InvThroughput: 127598 -> 127608 (+0.01%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8153
Fixes: 5806f0246fd ("aco/gfx11: workaround VALUPartialForwardingHazard")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20941>

intel/ds: Fix crash when allocating more intel_ds_queues than u_vector was initialized

u_vector_add() don't keep the returned pointers valid.
After the initial size allocated in u_vector_init() is reached it will
allocate a bigger buffer and copy data from older buffer to the new
one and free the old buffer, making all the previous pointers returned
by u_vector_add() invalid and crashing the application when trying to
access it.

This is reproduced when running
dEQP-VK.synchronization.signal_order.timeline_semaphore.* in DG2 SKUs
that has 4 CCS engines, INTEL_COMPUTE_CLASS=1 is set and of course
perfetto build is enabled.

To fix this issue here I'm moving the storage/allocation of
struct intel_ds_queue to struct anv_queue/iris_batch and using
struct list_head to maintain a chain of intel_ds_queue of the
intel_ds_device.
This allows us to append or remove queues dynamically in future if
necessary.

Fixes: e760c5b37be9 ("anv: add perfetto source")
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20977>

hasvk: Let spirv_to_nir() set UBO/SSBO base cast alignments

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21027>

anv: Let spirv_to_nir() set UBO/SSBO base cast alignments

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21027>

vtn: Set alignment on initial UBO/SSBO casts

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21027>

freedreno/a6xx: Remove excess CS flushing

Also requires fixing where we emit barriers, and flushing pending
barriers at the end of the batch.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20975>

freedreno/a6xx: Also FLUSH_CACHE on image barrier

For the same reason we need to on an UPDATE_BUFFER barrier.  Fixes
KHR-GLES31.core.compute_shader.pipeline-post-fs once the hard-coded
cache-flush is removed from launch_grid path.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20975>

freedreno/a6xx: Make shader state independent of grid info

Eventually we want to move this into a state group, so we can pre-bake
the cmdstream and re-emit it via CP_SET_DRAW_STATE when it is dirty.
But in order to do that it needs to not depend on grid info.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20975>

freedreno: Don't open-code setting dirty CS state

There is actually no issue with setting FD_DIRTY_PROG, since all state
is marked dirty when we switch from compute to 3d.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20975>

freedreno/a6xx: Don't double-write SP_CS_OBJ_START

Also SP_CS_INSTRLEN.  This is already done in fd6_emit_shader().

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20975>

freedreno: Skip flush_resource with explicit sync

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20975>

freedreno: nondraw-batch

Allow multiple compute grids to be combined into a single non-draw
batch.  This will allow us to optimize state emit and remove excess
flushing between compute jobs.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20975>

freedreno/a6xx: Add CS instrlen workaround

Based on !19023.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20975>

freedreno/a6xx: Add missing CS_BINDLESS mapping

Fixes: e51975142c0 ("freedreno/a6xx: Add bindless state"
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20975>

freedreno/ir3: Scalarize load_ssbo

The benefits of turning it into isam (which needs to be scalar as the
SSBO is sampled as a single component R32 texture) outweigh the benefits
of vectorizing.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20975>

freedreno/a6xx: LRZ for MSAA

We don't need to fall off the LRZ path when we fall back to clearing
depth with a u_blitter draw, since u_blitter uses zsa state to achieve
the depth/stencil clear and this is entirely compabile with LRZ.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20975>

freedreno/decode: Increase size of offsets table

The offsets table stores offsets of a buffer (such as cmdstream) that
we've already dumped.  The suballoc pool results in more suballocated
cmdstream allocated from a single backing buffer, meaning that we need
to increase the size of this table.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20975>

aco: use s_pack_ll_b32_b16 for constant copies

Totals from 2 (0.00% of 134913) affected shaders:
CodeSize: 28636 -> 28628 (-0.03%)

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20970>

aco: use s_bfm_64 for constant copies

Foz-DB Navi21:
Totals from 1025 (0.76% of 134913) affected shaders:
CodeSize: 1436752 -> 1432412 (-0.30%)

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20970>

aco/spill: always end spill vgpr after control flow

To fix a hypothetical issue:

v0 = start_linear_vgpr
if (...) {

} else {
   use_linear_vgpr(v0)
}
v0 = phi

We need a p_end_linear_vgpr to ensure that the phi does not use the same
VGPR as the linear VGPR.

This is also much simpler.

fossil-db (gfx1100):
Totals from 1195 (0.89% of 134574) affected shaders:
Instrs: 4123856 -> 4123826 (-0.00%); split: -0.00%, +0.00%
CodeSize: 21461256 -> 21461100 (-0.00%); split: -0.00%, +0.00%
Latency: 62816001 -> 62812999 (-0.00%); split: -0.00%, +0.00%
InvThroughput: 9339049 -> 9338564 (-0.01%); split: -0.01%, +0.00%
Copies: 304028 -> 304005 (-0.01%); split: -0.02%, +0.01%
PreVGPRs: 115761 -> 115762 (+0.00%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20621>

aco/tests: add setup_reduce_temp.divergent_if_phi

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20621>

aco: end reduce tmp after control flow, when used within control flow

In the case of:

v0 = start_linear_vgpr
if (...) {

} else {
   use_linear_vgpr(v0)
}
v0 = phi

We need a p_end_linear_vgpr to ensure that the phi does not use the same
VGPR as the linear VGPR.

fossil-db (gfx1100):
Totals from 3763 (2.80% of 134574) affected shaders:
MaxWaves: 90296 -> 90164 (-0.15%)
Instrs: 6857726 -> 6856608 (-0.02%); split: -0.03%, +0.01%
CodeSize: 35382188 -> 35377688 (-0.01%); split: -0.02%, +0.01%
VGPRs: 234864 -> 235692 (+0.35%); split: -0.01%, +0.36%
Latency: 47471923 -> 47474965 (+0.01%); split: -0.03%, +0.04%
InvThroughput: 5640320 -> 5639736 (-0.01%); split: -0.04%, +0.03%
VClause: 93098 -> 93107 (+0.01%); split: -0.01%, +0.02%
SClause: 214137 -> 214130 (-0.00%); split: -0.00%, +0.00%
Copies: 369895 -> 369305 (-0.16%); split: -0.31%, +0.15%
Branches: 164996 -> 164504 (-0.30%); split: -0.30%, +0.00%
PreVGPRs: 210655 -> 211438 (+0.37%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20621>

ac/gpu_info: add PCIe info

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20790>

amd: update amdgpu_drm.h

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20790>

radv: pass pCreateInfo to radv_graphics_pipeline_compile()

This removes some duplicated code.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20990>

radv: pass radv_compute_pipeline to radv_compute_pipeline_compile()

Similar to graphics.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20990>

radv: move retained shaders info to radv_graphics_pipeline

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20990>

radv: pass radv_graphics_pipeline to radv_graphics_pipeline_compile()

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20990>

radv: add helpers for capturing shaders and statistics

Instead of duplicating the logic everywhere.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20990>

radv: simplify pipeline_has_ngg during graphics shaders compilation

The is_ngg field is copied during shader info linking for GS, so
after radv_shader_fill_info() is performed, it's possible to use it.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20990>

radv: remove useless check about CS in radv_lower_io()

This function is now called only for graphics pipeline.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20990>

anv: expose EXT_load_store_op_none

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21018>

mailmap: Remap name and email for Val Packett

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21032>

intel: enable existing workaround for ICL platform

Patch changes comment to refer to the lineage 14014097488, this
workaround applies for ICL as well.

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20952>

radv: Improve the BVH size estimation

The previous estimation was from before we had proper LBVH and PLOC.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20988>

zink: use VK_EXT_multisampled_render_to_single_sampled for EXT_multisample_render_to_texture

this extension was added for the purpose of emulating the GL ext,
and using it is reasonably straightforward

the only (somewhat) invasive part is modifying the renderpass/dynamic hashes
to have samplecounts in the key, but this is also not too much work

now only fbfetch requires real renderpasses, and everything else is dynamic

fixes #7559

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20883>

zink: shrink zink_render_pass_state::msaa_expand_mask

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20883>

zink: hook up VK_EXT_multisampled_render_to_single_sampled

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20883>

meson: turn android-libbacktrace into a feature option

Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20915>

freedreno/gmem: Fix for partial z/s fast-clear

If we have a combined depth+stencil buffer, but fast-clear just one of
the two channels, we need to mark the other as needing restore.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20998>

nir: Add sha1 hash for nir shaders converted from spir-v

The sha1 hash inside nir structure
makes it easier to find bad shader in games.

For example INTEL_DEBUG=fs will show not zero
source_sha1 field for shaders with vulkan applications

Signed-off-by: Illia Polishchuk <illia.a.polishchuk@globallogic.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21012>

util/format: Fix wrong colors when importing YUYV and UYVY

This changes the swizzling so that importation of YUYV dmabuf without
dedicated blitter HW can work.

v2: fix the other format too, update test results

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20815>

util/disk_cache: Switch to multipart mesa-db cache

Replace single file mesa-db cache with multipart mesa-db cache.
Each part of the multipart cache essentially is a single file
mesa-db cache, aka database shard. Multipart cache brings much
more optimized cache eviction times in comparison to a single file
cache.

Acked-by: Timothy Arceri <tarceri@itsqueeze.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20256>

util/mesa-db: Introduce multipart mesa-db cache

Whenever a single file mesa-db cache hits max size limit, a half of cache
is evicted and the cache file is defragmented. The downside of this eviction
strategy is that it causes high disk IO usage during eviction if mesa-db
cache file size is large.

In order to mitigate this downside, we will split mesa-db into multiple
part such that only one part will be evicted at a time. Each part will be
an individual single file mesa-db cache, like a DB shard. The new multipart
mesa-db cache will merge the parts into a single virtual cache.

This patch introduces two new environment variables:

1. MESA_DISK_CACHE_DATABASE_NUM_PARTS:
Controls number of mesa-db cache file parts. By default 50 parts will be
created. The old pre-multipart mesa-db cache files will be auto-removed
if they exist, i.e. Mesa will switch to the new DB version automatically.

2. MESA_DISK_CACHE_DATABASE_EVICTION_SCORE_2X_PERIOD:
Controls the eviction score doubling time period. The evicted DB part
selection is based on cache entries size weighted by 'last_access_time' of
the entries. By default the cache eviction score is doubled for each month
of cache entry age, i.e. for two equally sized entries where one entry is
older by one month than the other, the older entry will have x2 eviction
score than the other entry. Database part with a highest total eviction
score is selected for eviction.

This patch brings x40 performance improvement of cache eviction time using
multipart cache vs a single file cache due to a smaller eviction portions
and more optimized eviction algorithm.

Acked-by: Timothy Arceri <tarceri@itsqueeze.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20256>

util/cache_test: Remove dummy cache entry added by cache_exists()

The cache_exists() function adds a dummy cache entry that may affect cache
eviction testing results. Remove this entry.

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20256>

util/mesa-db: Support removal of cache entries

Add support for removal of cache entries to mesa-db cache. This allows
to have a more predictable unit tests by removing dummy entries that
are added to cache when test-cache is created by unit-testing framework.

Acked-by: Timothy Arceri <tarceri@itsqueeze.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20256>

util/mesa-db: Don't account header size

In order to ease writing mesa-db eviction unit tests, stop accounting
mesa-db cache file header size during checking whether cache file reached
the size limit. This change ensures that older unit tests will keep working
whenever cache header version/size will change.

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20256>

util/cache_test: Unset env vars left after Cache.List test

The environment variables are persistent and must be explicitly unset by
each cache test, otherwise next test will fail. Add the missing unsets to
the Cache.List test.

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20256>

mailmap: Lina is Chad's new name

If you can longer find chadversary or chadv on the interwebs, then
search for linyaa or versalinyaa.

Egg-crAcked-By: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Egg-Cracked-By: Faith Ekstrand <faith@gfxstrand.net>
Egg-Cracked-By: Lyude Paul <lyude@redhat.com>
Egg-Cracked-By: Wann
Egg-Cracked-By: Zach Lesher
Egg-Cracked-By: 初音ミク
Acked-by: Daniel Stone <daniels@collabora.com>

intel: add missing PS restriction on BDW+

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20949>