nvc0: enable fp helper invocation memory loads on Turing+

Starting with GSP this will be the only way of doing so, for older gens
we'll just fix it up on the kernel side.

Cc: mesa-stable
Acked-by: M Henning <drawoc@darkrefraction.com>
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21989>

radv: use serialized NIR for graphics libs with the RETAIN flag

Cloning NIR shaders consumes too much RAM and this can easily explode
in memory for games that create a ton of graphics libraries. Using
serialized NIR shaders help considerably.

This reduces RAM usage in dota2 with GPL from 3GiB to 400MiB.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22143>

ci/broadcom: slightly increase coverage of vk tests

The previous commit has freed up a couple of runners, so let's repurpose
them to make vk test jobs take less time; with that spare time, let's
increase the coverage a little bit.

Most jobs now take 10-12 minutes, just like they used to.

Stress-tested over 40+ runs.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21902>

ci/broadcom: consolidate v3d-rpi4* jobs into a single v3d-rpi4-gl:armhf

Using now 12 instead of 14 runners, most jobs take 11-13 minutes, which
is about the same as when they were all running separately (10-13 min).

Stress-tested over 8 runs.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21902>

ci/broadcom: consolidate vc4-rpi3* jobs into a single vc4-rpi3-gl:armhf

Without reducing the coverage and using 4 runners instead of 9, most
runs take 10-13 minutes instead of 12-13 minutes for the egl job, 9-11
minutes for the piglit job, and 6-8 minutes for the deqp job.

Stress-tested over 40+ runs.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21902>

Revert "CI: Disable freedreno"

This reverts commit 6a16ed8d7949424f160cb2f1a17bc71d892ee66e.

The proxy has been rebooted

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22168>

radv: Move I/O lowering functions into a new file.

Also ran clang-format on the affected code.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21971>

radv, ac/nir: Move sin/cos lowering to a common pass.

Also ran clang-format on the affected code.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21971>

radv: Move radv_nir_export_multiview to new file.

Also ran clang-format on the affected code.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21971>

radv: Move radv_nir_lower_viewport_to_zero to new file.

Also ran clang-format on the affected code.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21971>

radv: Move radv_nir_lower_view_index to new file.

Also ran clang-format on the affected code.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21971>

radv: Move radv_nir_lower_intrinsics_early to new file.

Also ran clang-format on the affected code.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21971>

radv: Move radv_nir_lower_fs_intrinsics to new file.

Also ran clang-format on the affected code.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21971>

radv: Move radv_nir_lower_primitive_shading_rate to new file.

Also ran clang-format on the affected code.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21971>

radv: Move radv_nir_* to a new folder.

Also ran clang-format on the affected code.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21971>

kopper: apply ancillary invalidation through glthread on swapbuffers

this fixes invalidations with tc's renderpass parsing so they get picked
up before the renderpass ends

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21910>

glthread: add _mesa_glthread_invalidate_zsbuf()

this allows frontends to trigger ancillary invalidation before syncing

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21910>

mapi: add InternalInvalidateFramebufferAncillaryMESA

this allows glthread to handle ancillary buffer invalidation

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21910>

lavapipe: move default rasterizer state values to rendering_state init

these should always be set

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21866>

lavapipe: avoid uniformly unsetting gs_output_lines

instead fix the conditional now that it's possible to tell whether
a gs is bound

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21866>

lavapipe: only unset tess_states pointers on tes bind

this is otherwise irrelevant

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21866>

lavapipe: only update shader access for bind/unbind stages

also squash a loop

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21866>

lavapipe: split out gfx stage unbinding

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21866>

lavapipe: don't double unbind gfx stages on pipeline bind

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21866>

lavapipe: dynamically bind noop fs at draw time when needed

this is simpler than special casing it everywhere

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21866>

lavapipe: unify lvp_pipeline_nir creation

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21866>

lavapipe: add a ref for the tess_ccw nir on creation

not sure if it matters but good to be consistent

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21866>

lavapipe: refactor pipeline destroy a bit

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21866>

lavapipe: delete unused struct member

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21866>

lavapipe: break out all the important parts of gfx pipeline setting for reuse

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21866>

lavapipe: pull out dynamic tess origin check in gfx pipeline bind

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21866>

lavapipe: don't access pipeline shader structs as much during bind

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21866>

lavapipe: PIPE_SHADER_ -> MESA_SHADER_

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21866>

lavapipe: merge some loops in handle_graphics_pipeline()

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21866>

lavapipe: refactor compute shader binding

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21866>

lavapipe: stop using rendering_state::pipeline

access shaders directly now

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21866>

lavapipe: add a device member to rendering_state

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21866>

lavapipe: track bound shader stages on rendering_state

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21866>

lavapipe: refactor shader compile functions to not take pipeline params

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21866>

lavapipe: don't memcpy tess_ccw when copying pipeline library shaders

this gets handled later

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21866>

lavapipe: always copy streamout info when creating shaders

llvmpipe should be smart enough to figure this out internally

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21866>

ci/radeonsi: Skip really slow tests on stoney

These allocate way more memory than is reasonable, a bunch of times. I'd
guess they pushed the machine pretty deep into memory pressure which is
why it was all taking like 3 minutes.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22193>

ci/radeonsi: sort and dedup stoney skips

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22193>

aco: add tests for neg(mul) with opsel

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco/tests: run optimize.mad_mix.input_conv.modifiers on gfx11

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco: add test for min/max combining with opsel

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco: add tests for cmp ordering with opsel

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco: add tests for swap operand with opsel

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco: add tests for dpp with opsel

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco: add tests for fma with opsel

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco/optimizer: remove to_SDWA

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco: keep label_mul/usedef/minmax in apply_extract

16bit int mad/fma/minmax combining can work with opsel set.

All other optimizations should already check if the instruction uses sdwa,
because we don't check this when applying the label initially.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco/optimizer: use opsel for VOP12C

Foz-DB GFX1100:
Totals from 11759 (8.72% of 134864) affected shaders:
VGPRs: 848288 -> 844556 (-0.44%); split: -0.44%, +0.00%
SpillSGPRs: 8527 -> 8543 (+0.19%)
SpillVGPRs: 1411 -> 1423 (+0.85%); split: -0.21%, +1.06%
CodeSize: 114337120 -> 113882472 (-0.40%); split: -0.40%, +0.01%
Scratch: 128768 -> 129024 (+0.20%); split: -0.20%, +0.40%
MaxWaves: 250962 -> 252014 (+0.42%)
Instrs: 22187426 -> 22062378 (-0.56%); split: -0.57%, +0.00%
Latency: 232655375 -> 232376977 (-0.12%); split: -0.20%, +0.08%
InvThroughput: 28292530 -> 28217699 (-0.26%); split: -0.45%, +0.18%
VClause: 352463 -> 352364 (-0.03%); split: -0.12%, +0.10%
SClause: 659282 -> 659354 (+0.01%); split: -0.02%, +0.04%
Copies: 1371369 -> 1342340 (-2.12%); split: -2.30%, +0.19%
Branches: 495903 -> 495941 (+0.01%); split: -0.00%, +0.01%
PreSGPRs: 867295 -> 863664 (-0.42%)
PreVGPRs: 793480 -> 790549 (-0.37%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco/gfx11: allow opsel for VOP12C

Foz-DB GFX1100:
Totals from 515 (0.38% of 134864) affected shaders:
CodeSize: 2768228 -> 2761076 (-0.26%)
Instrs: 520301 -> 518523 (-0.34%)
Latency: 5190860 -> 5187254 (-0.07%)
InvThroughput: 2120844 -> 2119447 (-0.07%)
Copies: 57238 -> 56101 (-1.99%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco: don't label mul with opsel as abs/neg

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco/ir: copy opsel when converting to DPP

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco: swap opsel when swapping VOP2/C operands

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco: return true in usesModifiers for VOP12C with opsel

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco: support neg(mul)/abs(mul) optimization in more cases

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco: support v_cvt_f32_f16 with opsel in combine_mad_mix

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco: update match_op3_for_vop3 for VOP12C opsel

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco: handle opsel in combine_constant_comparison_ordering

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco: handle opsel in combine_ordering_test

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco: handle opsel in combine_comparison_ordering

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco/optimizer: preserve opsel when fusing fma

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco/ra: prepare for VOP12C opsel

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco/to_hw_instr: use VOP1 opsel for v_mov_b16

Foz-DB GFX1100:
Totals from 4661 (3.46% of 134864) affected shaders:
CodeSize: 36500568 -> 36391704 (-0.30%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco: validate VOP12C opsel

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco/assembler: support VOP12C opsel

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco/vn: hash opsel for VOP12C

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco/ra: don't reallocate VOP3 instruction for non-vcc lane mask

This would need to copy opsel soon but we can just reuse the old instruction.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>

aco: Un-swap addressable VGPRs/SGPRs in RT prolog

Fixes: 6446b791681 ("aco: implement select_rt_prolog()")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22188>

radv/rt: Also adjust the SGPR count in postprocess_rt_config

Fixes: bea022d1f6a ("radv/rt: Add shader config combination/postprocessing utils")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22188>

perfetto: Move intel's cmdbuf/queue annotation code to the shared util.

This will let other drivers use the same way of presenting annotations
without duplicating the whole hash table thing.

Reviewed-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22157>

perfetto: Deduplicate clock sync packet emit from renderstage sources.

This is way more horrifying than I hoped -- I can't figure out a way to
have the method be on TraceContext, so it's a static method of the
datasource, but then you have to name the templated types over and over.
You have to pass in a TraceContext because intel emits the clock sync
packet within a Trace(), and perfetto just silently corrupts the trace if
you Trace() in a Trace().

Reviewed-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22157>

perfetto: Make a MesaRenderpassDataSource with common setup/start/stop.

Deduplicates some code from intel/tu/freedreno, and will be a common place
to put other shared code.

The downside I can see is this logging:

[013.129]      tu_perfetto.cc:122 Tracing started
[013.129]  intel_driver_ds.cc:133 Tracing started

("oh, huh, apparently data sources for both drivers are registered?  wild")

becomes:

[142.906] erfetto_renderpass.h:50 Tracing started
[142.907] erfetto_renderpass.h:50 Tracing started

("huh, why is my driver's data source being started twice?").
Unfortunately we can't easily get a string for the data source type due to
not having rtti.

Reviewed-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22157>

intel/perfetto: Drop unused "pipelined" field.

Reviewed-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22157>

perfetto: Add a .clang-format for the directory.

I was frustrated trying to write code and not be able to just mash ^K^F to
format what I'd written.  This .clang-format is just cargo-cult of turnip
with a few tweaks to reduce the diff to the current directory contents.
The remaining deltas in the reformat look decent to me, and mostly bring
things closer to mesa-vague-consensus style.

Reviewed-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22157>

anv: Add assert in functions not supported by Xe kmd

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22171>

anv: Disable anv_bo_sync_type for Xe kmd

anv_bo_sync_type is only supported with i915, if necessary it will be
implemented for Xe kmd.

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22171>

anv: Implement Xe version of anv_queue_exec_locked() and queue_exec_trace()

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22171>

anv: Move to a function code to clflush batch buffers

This also need to be executed in Xe kmd, so moving it to a function.
No changes in behavior intended here.

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22171>

util/disk_cache: use posix_fallocate() for index files

ftruncate() allocates disk space lazily. If the disk is full and it is
unable to allocate disk space when accesed via mmap(), it will crash
with a SIGBUS.

Switch to posix_fallocate(), which ensures disk space is allocated
otherwise it fails if there isn't enough disk space. The disk cache
won't be enabled in this case.

For normal cases, a small increase in disk usage as the 1.3MB index
file will be fully allocated when initialized now.

fallback to ftruncate() if posix_fallocate() isn't found.

Signed-off-by: Juston Li <justonli@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22097>

anv: hash immutable sampler conversion data not pointers

The conversion pointer has a vk_object_base containing pointers.

Previous commits had a similar issue, storing an internal pointer to
the array of formats in anv_format.c

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Reviewed-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22160>

nir/algebraic: Undistribute fsat from fmax

To be helpful, the thing inside the fsat has to be used with and without
the fsat. Otherwise it just moves a saturate destination modifier
around. To not be harmful, the fsat has to only be used by the bcsel.

All Broadwell and newer Intel platforms had similar results. (Ice Lake shown)
total instructions in shared programs: 20174475 -> 20174449 (<.01%)
instructions in affected programs: 3913 -> 3887 (-0.66%)
helped: 13 / HURT: 0

total cycles in shared programs: 866844832 -> 866844719 (<.01%)
cycles in affected programs: 46037 -> 45924 (-0.25%)
helped: 10 / HURT: 1

All Intel platforms had similar results. (Ice Lake shown)
Instructions in all programs: 161491468 -> 161491372 (-0.0%)
helped: 31 / HURT: 8

Cycles in all programs: 10933090736 -> 10933024716 (-0.0%)
helped: 32 / HURT: 18

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22169>

intel/fs: Don't copy propagate from saturate to sel

There are already NIR algebraic optimizations (see also ac6646129f7
("nir: Move fsat outside of fmin/fmax if second arg is 0 to 1.") that
will try to remove the saturate from things like

    fmax(0.5, fsat(x))

This basically reverts 40aeb558ce8 ("i965/fs: Allow propagation of
instructions with saturate flag to sel"). That commit message had no
shader-db information, so it's unclear whether this actually helped
anything ever.

No shader-db changes on any Intel platform.

One shader in Far Cry New Dawn was affected.

Cycles in all programs: 10933090738 -> 10933090736 (-0.0%)
Cycles helped: 1

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22169>

radv: fix setting radv_shader_info::user_data_0 with rt

Fixes raytracing pipelines.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Fixes: 0e81ec98ce9 ("radv: move user_data_0 to the shader info pass")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22187>

ci/zink: Reduce zink-tu-a618-traces parallelism

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22155>

ci/zink: Fix zink-tu-a618-traces perf job rules

The pipeline should create this job only when it is under performance
enabled workflow, represented by the presence of
MESA_CI_PERFORMANCE_ENABLED variable.

For example:
The following pipeline
https://gitlab.freedesktop.org/gallo/mesa/-/pipelines/840525 has
`zink-tu-a618-traces-performance` job, even if it was not triggered by
marge-bot, which means that performance jobs should be disabled.

This job should appear in pipelines with performance jobs enabled
https://gitlab.freedesktop.org/gallo/mesa/-/pipelines/840529

Fixes: 93e3d37b47c77b19b330e3d5c6d2440108636952
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22155>

radeonsi: simplify binning settings to work around GPU hangs

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22084>

radeonsi/gfx11: reduce MSAA samples to 8 for no-attachment framebuffer

EQAA is unsupported

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22084>

amd: set the correct LLVM processor name for gfx1036

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22084>

amd: rename GFX1036 -> RAPHAEL_MENDOCINO

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22084>

amd: update addrlib

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22084>

zink/ci: remove `primitive-id-no-gs-quads` from radv-vangogh-fails

Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21238>

zink: unbind generated gs in `bind_last_vertex_stage`

Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21238>

zink: improve generated gs unbinding

Avoid looping by using the new `parent` field to check if a generaetd gs
is bound and use `bind_gs_state` insted of `bind_gfx_stage` so that
`bind_last_vertex_stage` is automatically called

Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21238>

zink: zink: add `parent` to `zink_shader::non_fs`

For a generated gs this field will store a pointer to the shader that
"owns" it.

Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21238>

zink: unified `zink_set_primitive_emulation_keys` and `zink_create_primitive_emulation_gs`

Those two functions shared most of the code

Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21238>

zink: prevent crash when freeing

If the same vertex shader is used for more than one pipeline where for some a gs
is generated but not for others then the logic to free pipeline
libraries might use the incorrect stage_mask and try to free a non
existing gs.

Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21238>