ci/farms: always compare the code against main repository

`rules:changes:compare_to` resolved firstly pushed branch pipelines,
which always evaluated `rules:changes` as true which breaks the workflow

Since we now explicitely say, that we compare against `main` repository,
GitLab can evaluate against real changes.

Fixes: 79f7882fc604 ("ci: add quirk for GitLab assuming changes is always true for scheduled runs")

Reviewed-by: Eric Engestrom <eric@igalia.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24305>

anv: simplify buffer address+size loads from descriptor buffer

Only found a couple titles that have been helped by this :

 PERCENTAGE DELTAS Shaders   Instrs    Cycles
 cyberpunk_2077    10388     -0.00%    -0.00%
 -----------------------------------------------
 All affected      1         -2.24%    -0.39%
 -----------------------------------------------
 Total             10388     -0.00%    -0.00%

 PERCENTAGE DELTAS    Shaders   Instrs    Cycles
 red_dead_redemption2 5949      -0.10%    -0.00%
 --------------------------------------------------
 All affected         111       -0.74%    -0.14%
 --------------------------------------------------
 Total                5949      -0.10%    -0.00%

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23318>

isl: add ability to store buffer size in unused RENDER_SURFACE_STATE fields

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23318>

intel/fs: add more UNDEFs around SEND messages

lower_find_live_channel() in particular is used a lot in control flow
to find the live channel for the surface/sampler handle. Adding UNDEFs
on the temporary registers used for finding the live channels helps
reduce the liveness of those temporary registers, especially in loops.

Some titles affected :

Rise Of The Tomb Raider:
Totals from 2780 (22.58% of 12311) affected shaders:
Instrs: 1294455 -> 1294592 (+0.01%); split: -0.15%, +0.16%
Cycles: 1473136441 -> 1471302617 (-0.12%); split: -1.52%, +1.40%
Max live registers: 144282 -> 143595 (-0.48%)
Max dispatch width: 22200 -> 22232 (+0.14%)

Red Dead Redemption 2:
Totals from 435 (7.28% of 5972) affected shaders:
Instrs: 488472 -> 487594 (-0.18%); split: -0.31%, +0.14%
Cycles: 11354732 -> 11384928 (+0.27%); split: -0.44%, +0.71%
Spill count: 1217 -> 1172 (-3.70%)
Fill count: 3521 -> 3447 (-2.10%)
Scratch Memory Size: 64512 -> 62464 (-3.17%)
Max live registers: 35997 -> 35798 (-0.55%)

Fallout 4:
Totals from 8 (0.49% of 1638) affected shaders:
Instrs: 41908 -> 40509 (-3.34%)
Cycles: 3638464 -> 3555680 (-2.28%); split: -2.67%, +0.39%
Spill count: 717 -> 665 (-7.25%)
Fill count: 2542 -> 2438 (-4.09%)
Scratch Memory Size: 32768 -> 16384 (-50.00%)
Max live registers: 567 -> 534 (-5.82%)

Cyberpunk 2077:
Totals from 2984 (28.97% of 10301) affected shaders:
Instrs: 3888874 -> 3891600 (+0.07%); split: -0.20%, +0.27%
Cycles: 67906489 -> 67767721 (-0.20%); split: -0.68%, +0.47%
Spill count: 200 -> 98 (-51.00%)
Fill count: 237 -> 90 (-62.03%)
Scratch Memory Size: 10240 -> 8192 (-20.00%)
Max live registers: 215715 -> 212727 (-1.39%)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24282>

intel/fs: consider UNDEF as non-partial write

A few titles show max live register reductions, but nothing
significant in instruction count or other stats.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24282>

radv: rename radv_pipeline_stage to radv_shader_stage

It's more generic and it will fit shader object just well.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24313>

radv: cleanup pipeline compute emit helpers

Merge both functions together and rename the function.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24313>

radv: rework determining the NGG stage without a graphics pipeline

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24313>

radv: simplify lowering NGG GS intrinsics

The is_ngg field is already set correctly for GS.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24313>

radv: rename graphics pipeline linking helpers

There is no pipeline dependency.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24313>

radv: move removing all varyings when the FS is a noop

This allows us to remove one more pipeline dependency.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24313>

radv: stop passing radv_graphics_pipeline to radv_fill_shader_info()

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24313>

radv: rework considering force VRS without relying on graphics pipeline

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24313>

radv: set next_stage to MESA_SHADER_NONE if there is no FS

This follows the same convention as shader object where the last stage
would have nextStage to 0. This will allow more refactoring.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24313>

radv: initialize stage/next_stage earlier

This will allow more refactoring.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24313>

intel/vec4: fix log_data pointer

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 3384f029be ("intel/compiler: rework input parameters")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9421
Acked-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24307>

ac: Switch to use nir_foreach_function_impl in function analyze_shader_before_culling

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23940>

radeonsi: Convert to use nir_foreach_function_impl

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23940>

microsoft/clc/compiler: Convert to use nir_foreach_function_impl when possible

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23940>

microsoft/compiler: convert to use nir_foreach_function_with_impl in function emit_module

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23940>

vulkan/runtime: Add helper functions for VK_EXT_host_image_copy

Add helper functions vk_memory_to_image_copy_layout() and
vk_image_to_memory_copy_layout(), which will be useful in
VK_EXT_host_image_copy implementations.

vk_memory_to_image_copy_layout() is similar to
vk_image_buffer_copy_layout(), except the second parameter is
VkMemoryToImageCopyEXT instead of VkBufferImageCopy2.

vk_image_to_memory_copy_layout() is similar to
vk_image_buffer_copy_layout(), except the second parameter is
VkImageToMemoryCopyEXT instead of VkBufferImageCopy2.

Signed-off-by: Rebecca Mckeever <rebecca.mckeever@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24290>

gm107/ir: fix SULDP for loads without a known format

Signed-off-by: Karol Herbst <git@karolherbst.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24312>

intel/compiler: call brw_nir_adjust_payload from brw_postprocess_nir

Calling anything after nir_trivialize_registers() risks undoing some of
its work.
In this case, brw_nir_adjust_payload() will do a constant folding pass
if any payload adjusting happened, and that can turn a bunch of
@store_regs into basically noops.

Fixes dEQP-VK.subgroups.*task

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24325>

intel/fs: Constant fold OR and AND

The path taken in fs_visitor::swizzle_nir_scratch_addr for DG2 generates
some AND and OR instructions before the SHL. This commit folds those so
the whold calculation becomes a constant (like on older platforms).

v2: Fix return type of src_as_uint. Noticed by Marcin.

shader-db results:

DG2
total instructions in shared programs: 23190475 -> 23179540 (-0.05%)
instructions in affected programs: 36026 -> 25091 (-30.35%)
helped: 7 / HURT: 0

total cycles in shared programs: 841196807 -> 841142563 (<.01%)
cycles in affected programs: 1660670 -> 1606426 (-3.27%)
helped: 7 / HURT: 0

No shader-db changes on any older Intel platforms.

fossil-db results:

DG2
Totals:
Instrs: 197780372 -> 197773966 (-0.00%)
Cycles: 14066410782 -> 14066399378 (-0.00%); split: -0.00%, +0.00%
Subgroup size: 8438104 -> 8438112 (+0.00%)
Send messages: 8049445 -> 8049446 (+0.00%)
Scratch Memory Size: 14263296 -> 14264320 (+0.01%)

Totals from 9 (0.00% of 668055) affected shaders:
Instrs: 24547 -> 18141 (-26.10%)
Cycles: 1984791 -> 1973387 (-0.57%); split: -0.98%, +0.40%
Subgroup size: 88 -> 96 (+9.09%)
Send messages: 867 -> 868 (+0.12%)
Scratch Memory Size: 69632 -> 70656 (+1.47%)

No fossil-db changes on any older Intel platforms.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23884>

intel/fs: Constant fold SHL

This is a modified version of a commit originally in !7698. This version
add the changes to brw_fs_copy_propagation. If the address passed to
fs_visitor::swizzle_nir_scratch_addr is a constant, that function will
generate SHL with two constant sources.

DG2 uses a different path to generate those addresses, so the constant
folding can't occur there yet. That will be addressed in the next
commit.

What follows is the commit change history from that older MR.

v2: Previously this commit was after `intel/fs: Combine constants for
integer instructions too`.  However, this commit can create invalid
instructions that are only cleaned up by `intel/fs: Combine constants
for integer instructions too`.  That would potentially affect the
shader-db results of each commit, but I did not collect new data for
the reordering.

v3: Fix masking for W/UW and for Q/UQ types. Add an assertion for
!saturate. Both suggested by Ken. Also add an assertion that B/UB types
don't matically come back.

v4: Fix sources count. See also ed3c2f73dbb ("intel/fs: fixup sources
number from opt_algebraic").

v5: Fix typo in comment added in v3. Noticed by Marcin. Fix a typo in a
comment added when pulling this commit out of !7698. Noticed by Ken.

shader-db results:

DG2
No changes.

Tiger Lake, Ice Lake, and Skylake had similar results (Ice Lake shown)
total instructions in shared programs: 20655696 -> 20651648 (-0.02%)
instructions in affected programs: 23125 -> 19077 (-17.50%)
helped: 7 / HURT: 0

total cycles in shared programs: 858436639 -> 858407749 (<.01%)
cycles in affected programs: 8990532 -> 8961642 (-0.32%)
helped: 7 / HURT: 0

Broadwell and Haswell had similar results. (Broadwell shown)
total instructions in shared programs: 18500780 -> 18496630 (-0.02%)
instructions in affected programs: 24715 -> 20565 (-16.79%)
helped: 7 / HURT: 0

total cycles in shared programs: 946100660 -> 946087688 (<.01%)
cycles in affected programs: 5838252 -> 5825280 (-0.22%)
helped: 7 / HURT: 0

total spills in shared programs: 17588 -> 17572 (-0.09%)
spills in affected programs: 1206 -> 1190 (-1.33%)
helped: 2 / HURT: 0

total fills in shared programs: 25192 -> 25156 (-0.14%)
fills in affected programs: 156 -> 120 (-23.08%)
helped: 2 / HURT: 0

No shader-db changes on any older Intel platforms.

fossil-db results:

DG2
Totals:
Instrs: 197780415 -> 197780372 (-0.00%); split: -0.00%, +0.00%
Cycles: 14066412266 -> 14066410782 (-0.00%); split: -0.00%, +0.00%

Totals from 16 (0.00% of 668055) affected shaders:
Instrs: 16420 -> 16377 (-0.26%); split: -0.43%, +0.17%
Cycles: 220133 -> 218649 (-0.67%); split: -0.69%, +0.01%

Tiger Lake, Ice Lake and Skylake had similar results. (Ice Lake shown)
Totals:
Instrs: 153425977 -> 153423678 (-0.00%)
Cycles: 14747928947 -> 14747929547 (+0.00%); split: -0.00%, +0.00%
Subgroup size: 8535968 -> 8535976 (+0.00%)
Send messages: 7697606 -> 7697607 (+0.00%)
Scratch Memory Size: 4380672 -> 4381696 (+0.02%)

Totals from 6 (0.00% of 662749) affected shaders:
Instrs: 13893 -> 11594 (-16.55%)
Cycles: 5386074 -> 5386674 (+0.01%); split: -0.42%, +0.43%
Subgroup size: 80 -> 88 (+10.00%)
Send messages: 675 -> 676 (+0.15%)
Scratch Memory Size: 91136 -> 92160 (+1.12%)

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23884>

intel/fs: Always do opt_algebraic after opt_copy_propagation makes progress

opt_copy_propagation can create invalid instructions like

    shl(8) vgrf96:UD, 2d, 8u

These instructions will be cleaned up by opt_algebraic.  The irony is
opt_algebraic converts these to simple mov instructions that
opt_copy_propagation should clean up.  I don't think we want a loop like

   do {
      progress = false;
      if (OPT(opt_copy_propagation)) {
         OPT(opt_algebraic);
         OPT(dead_code_eliminate);
      }
   } while (progress);

But maybe we do?

Maybe this would be sufficient:

   while (OPT(opt_copy_propagation))
      OPT(opt_algebraic);
   OPT(dead_code_eliminate);

No shader-db or fossil-db changes (yet) on any Intel platform.  This is
expected.

v2: Do opt_algebraic immediately after every call to
opt_copy_propagation instead of being clever. Suggested by Lionel.

Tested-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23884>

ci/a5xx: Add another GPU hanging piglit test to the skips.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23485>

Uprev Piglit to ed58dfbd12be34fa3dab97a7a2987b890e0637f1

https://gitlab.freedesktop.org/mesa/piglit/-/compare/5036601c43fff63f7be5cd8ad7b319a5c1f6652c...ed58dfbd12be34fa3dab97a7a2987b890e0637f1

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23485>

tu: Fix data race in userspace VMA management.

The sequence was two threads A and B on a shared VkDevice:

A: move a BO to zombie VMA list
A: drop the BO VMA lock
B: prepare to allocate a BO
B:   Lock BO VMA lock
B:     call tu_free_zombie_vma_locked()
B:       close the gem handle from the VMA list
B:   Drop BO VMA lock
B: allocate a BO, getting the recently-closed handle back.
B: initialize the BO struct for the new handle.
A: memset the BO struct to 0.

Multithreading in C is the worst.

Closes: #9049, #9247
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24324>

iris: Request Xe KMD to place BOs to CPU visible VRAM when required

This is required to support discrete GPUs placed in systems with large
PCI bar or resizeble PCI bar not available or disabled.

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23781>

anv: Request Xe KMD to place BOs to CPU visible VRAM when required

This is required to support discrete GPUs placed in systems with large
PCI bar or resizeble PCI bar not available or disabled.

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23781>

intel/dev/xe: Add support for small-bar setups

This adds support for discrete GPUs placed in systems with large PCI
bar or resizeble PCI bar not available or disabled.

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23781>

intel: Sync xe_drm.h

Sync with commit aef50195664a ("drm/xe/uapi: add the userspace bits for small-bar")

Link: https://patchwork.freedesktop.org/series/115515/
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23781>

ci/tu: Drop some xfails for !24086

Fixes: 99e58460efb9 ("tu: Fix zombie VMAs array not initialized when first BOs may be freed")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24322>

ci/tu: Mark descriptor_buffer.basic.limits as failing in gmem too.

Noticed in a full run.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24322>

ci/tu: Add more crash cases for the multithreading bugs caught on a630.

Weirdly, we don't see this group on a618.  Different CPU timings/core
counts just getting unlucky?

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24322>

nir/lower_blend: Optimize out PIPE_LOGICOP_NOOP

Just drop the store. Written while debugging
dEQP-VK.pipeline.monolithic.logic_op.r8_uint.no_op.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Italo Nicola <italonicola@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24252>

nir/lower_blend: Fix 32-bit logicops

nir_const_value_for_int asserts signed bounds on the input, but we pass in an
unsigned value that would be out-of-bounds for 32-bit channels, causing the
assert to fail for 32-bit channel formats.

Fixes dEQP-VK.pipeline.monolithic.logic_op.r32_uint.* on AGXV (and probably
PanVK).

Fixes: dbd0615e7ad ("nir/lower_blend: Avoid useless iand with logic ops")
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Italo Nicola <italonicola@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24252>

panfrost: Disable blending for no-op logic ops

Prevents regression from the series, since we don't support empty blend
shaders. This could be fixed more generically but I'm not inclined to compile
more blend shaders than needed so shrug.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Italo Nicola <italonicola@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24252>

rusticl: fix warnings with newer rustc

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24315>

intel/fs: Assume NIR is in SSA form

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24310>

intel/fs: Rework the overlapping mov/vec case

Now that we're using load/store_reg intrinsics, the previous checks for
registers aren't what we want.  Instead, we need to be looking for a mov
or vec where both the destination and a source are load/store_reg with
matching decl_reg.

Fixes: b8209d69ffdb ("intel/fs: Add support for new-style registers")
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24310>

intel/fs: Use write masks from store_reg intrinsics

Fixes: b8209d69ffdb ("intel/fs: Add support for new-style registers")
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24310>

broadcom/compiler: Convert to new-style NIR registers

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24153>

nir/schedule: Support load/store_reg

These are tracked the same way as register reads and writes, allowing
them to be re-arranged as long as they respect dependencies within the
same reg.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24153>

vc4: Convert to new-style NIR registers

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24153>

nir/trivialize: Move decl_reg to the start of the block

This makes it so we never find a reg_decl in between a reg_store and the def
for its value, which helps avid inserting copy movs.

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24153>

nir/trivialize: Handle more RaW hazards

Consider the snippet of NIR:

   div 32    %447 = @load_reg (%442) (base=0, legacy_fabs=0, legacy_fneg=0)
   div 32    %463 = @load_reg (%442) (base=0, legacy_fabs=0, legacy_fneg=0)
   con 32    %409 = iadd %17 (0x3), %447
                    @store_output (%182 (0x601), %463) (base=0, wrmask=x, component=0, src_type=invalid...
                    @store_reg (%409, %442) (base=0, wrmask=x, legacy_fsat=0)

The load_reg's are trivial, so the %442 read will get folded into store_output.
But under the old definition, the store_reg is also trivial so it gets folded
into the iadd... causing a read-after-write hazard and invalid code generation.

The fix is to amend our definition of store_reg triviality to account for loads
getting folded in. It's not good enough that there's no intervening load_reg,
there can also be no intervening source that gets chased to a load_reg. Handle
that case as well.

Identified in dEQP-VK.geometry.input.basic_primitive.triangles_adjacency on
V3DV.

Fixes: d313eba94ef0 ("nir: Add pass for trivializing register access")
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reported-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24153>

nir/trivialize: Trivialize cross-block loads

In order for a register load to be trivial, it cannot be used in any
block other than the one in which it is loaded.  We're not currently
explicitly doing anything to ensure this invariant holds.  It may be
that it holds regardless but I couldn't find any documented reason why
it should so let's explicitly handle that case.  Worst case, the newly
added code does nothing.

Fixes: d313eba94ef0 ("nir: Add pass for trivializing register access")
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24153>

nir/trivialize: Maintain divergence information

Because this pass is intended to be run after out-of-SSA and directly
before injesting the NIR into the back-end, it may come after divergence
analysis and needs to preserve the divergence information. Fortunately,
since all we ever do is insert nir_op_mov, this is easy.

Fixes: d313eba94ef0 ("nir: Add pass for trivializing register access")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24153>

nir: Properly handle divergence for load_reg

This commit makes three changes:

 1. Default all newly created registers divergent because this is the
    safer default.

 2. Make divergence analysis do something sane with register divergence.
    It's not perfect because divergence analysis isn't able to prove
    registers divergent based on stores but at least if someone uses
    registers a bit they'll end up with safe defaults.  This matches
    what they'd get with nir_ssa_def_init().

 3. Make the load_reg() helper automatically propagate divergence from
    the register.  Because the defaults for both nir_ssa_def_init() and
    nir_decl_reg() are to mark everything divergent, this only means
    that nir_load_reg() of a uniform reg is now uniform.

Putting all these together, nir_from_ssa should now be producing
load_reg intrinsics with the proper uniform information.

Fixes: 7229bffcb133 ("nir: Add intrinsics for register access")
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24153>

pan/bi: Remove leftover include

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24153>

intel/compiler/test: fix crashes when TEST_DEBUG is set

Dumping instructions requires that ISA info is not empty.

Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24274>

lavapipe: fixes indent of function lvp_inline_uniforms

The indent fixes are in separate patch is for easier to review

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24316>

lavapipe: Convert to use nir_foreach_function_impl

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24316>

zink: Convert to use nir_foreach_function_impl when possible

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24316>

freedreno: Switch to use nir_foreach_function_impl in tu_shader.cc

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24316>

lima: Convert to use nir_foreach_function_impl when possible

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Erico Nunes <nunes.erico@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24316>

rusticl/core: Make convert_spirv_to_nir output pair (KernelInfo, NirShader)

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23898>

rusticl/core: Delete KernelDevState and KernelDevStateInner

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23898>

rusticl/program: New helper functions to NirKernelBuild

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23898>

rusticl: Move NirKernelBuild to ProgramDevBuild

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23898>

rusticl/compiler: Remove unnecessary functions

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23898>

rusticl: Move Cso to Program

Commit got huge, but couldn't figure out a better way to split without
breaking stuff.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23898>

rusticl/compiler: Add NirPrintfInfo

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23898>

rusticl/kernel: Add CsoWrapper

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23898>

rusticl/kernel: Removing unnecessary clone in kernel launch

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23898>

ci/freedreno: add a530 flake vs-lessthanequal-uvec4-uvec4

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24314>

state_tracker: fix dereference before null check

Coverity error
CID 1528178 (#1 of 1): Dereference before null check (REVERSE_INULL)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Illia Polishchuk <illia.a.polishchuk@globallogic.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20893>

glx: fix dead code when gc var cannot be null due to earlier check

CID 1528170 (#1 of 1): Logically dead code (DEADCODE)
At condition gc, the value of gc cannot be NULL.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Illia Polishchuk <illia.a.polishchuk@globallogic.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20893>

s/Intel: fix/anv: fix: potentially overflowing expression in genX

CID 1528164 (#1 of 1): Unintentional integer overflow (OVERFLOW_BEFORE_WIDEN)
overflow_before_widen: Potentially overflowing expression
pool->n_passes * pool->khr_perf_preamble_stride with type
unsigned int (32 bits, unsigned) is evaluated using 32-bit arithmetic,
and then used in a context that expects an expression of type uint64_t (64 bits, unsigned).

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Illia Polishchuk <illia.a.polishchuk@globallogic.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20893>

iris: remove NULL check for already dereferenced pointer earlier

Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Illia Polishchuk <illia.a.polishchuk@globallogic.com>
Found by Coverity.

CID: 1528158
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20893>

i915: change format in dbg string
Actually, uintptr_t is of type unsigned long, but the
debug line uses the %d format specifier, which expects an int.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Illia Abernikhin <illia.abernikhin@globallogic.com>
Found by Coverity.

CID: 1515961
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20893>

state_tracker: moving initialisation of whandle out from if statement
whandle initialization inside if statement but used also outside

Signed-off-by: Illia Abernikhin <illia.abernikhin@globallogic.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Found by Coverity.

CID: 1516746
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20893>

lavapipe: Advertise samplerYcbcrConversion

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24295>

lavapipe: Implement samplerYcbcrConversion

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24295>

lavapipe: Fix binding immutable samplers with desc buffers

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24295>

lavapipe: Store immutable_samplers as lvp_sampler array

We will need this to access the ycbcr conversion.

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24295>

lavapipe: Remove dummy sampler ycbcr conversion

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24295>

gallivm: Ignore nir_tex_src_plane

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24295>

gallivm: Fix subsampled format sampling under Vulkan

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24295>

draw: Do not restart the primitive_id at 0

Otherwise the primitive_id will wrap around to 0 if more than 4096
patches are drawn.

cc: mesa-stable

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24295>

radv: pass submit info to radv_check_gpu_hangs()

This will allow to dump preambles/postambles CS and eventually even
more CS.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24191>

radv/amdgpu: rename old_ib to ib in radv_amdgpu_winsys_cs_dump()

Forgot this variable when I renamed the ib_buffers array.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24191>

radv/amdgpu: fix dumping CS with the chained IBs path

ib_buffer is now NULL in both paths, and the first IB is the beginning
of the chain.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24191>

radv: use next_stage for determining the stage to lower NGG

If the next stage is FS, it's also the last VGT API stage.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24273>

radv: simplify getting next VS stage for VS prologs

It's the VS shader info stage.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24273>

radv: determine as_ls earlier by using the next stage

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24273>

radv: determine ES info for VS/TES with GS earlier

By using the next stage, it's possible to compute these information
earlier without having to link shaders info.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24273>

radv: use the number of GS linked inputs to compute the ESGS itemsize

It's similar.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24273>

radv: add a helper to compute the ESGS itemsize

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24273>

radv: remove the pipeline dependency for creating a GS copy shader

This is unnecessary. While we are at it, stop passing the array of
shaders and use the GS stage only.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24273>

intel/common: Only set op mask on instructions in decoder

When a default value of a struct's field, which is in the
higher half of the first dword, is specified in a gen xml
file, setting op mask makes decoder treat the field as a
header (intel_field_is_header()). As a result, it won't
output the field in batch dump. This is not a common case
but can happen once a gen xml file includes such fields.

The op mask is only meaningful to instructions, so we fix
the above issue by not setting op mask of structs (also
registers).

Signed-off-by: Jianxun Zhang <jianxun.zhang@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24268>

iris: Handle clear color compatibility in prepare_render

Before this patch, iris_resource_render_aux_usage would disable
compression when the clear color did not support format
reinterpretation.

With this patch, iris now replaces the clear color with zero and keeps
compression enabled. Disabling fast clears would be enough for most aux
usages, but replacement is also done to handle ISL_AUX_USAGE_FCV_CCS_E.

Note that this also fixes a bug. Format reinterpretation with
incompatible clear colors previously was not handled for the MCS aux
usages.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23676>

iris: Create BLORP surfaces after resource preparation

iris_resource_prepare_render will soon gain the ability to change a
resource's clear color. iris_blorp_surf_for_resource will keep a copy of
that clear color, so make sure calls to it happen after the render
preparation helper. At the moment, this shouldn't have an impact besides
improving debugging.

While we're here, do the same for the generic access preparation helper.
We may convert those to more specific helpers at a later time.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23676>

iris: Pass the render format to prepare_render

This will be used in an upcoming patch.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23676>

iris: Reorder render_aux_usage parameters

Match the order of the parameters for iris_resource_texture_aux_usage.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23676>

intel/blorp: Ambiguate after CCS resolves on gfx7-8

ISL's state-machine of CCS_D describes full resolves as leaving the aux
buffer in the pass-through state. Hardware doesn't behave this way on
gfx8 however. On that platform, full resolves transition the aux buffer
to the resolved state. This was verified by dumping the CCS before and
after a full resolve on BDW (gfx7 is simply assumed to behave the same).
Ambiguate after resolving to match driver expectations.

Prevents iris from failing piglit's fcc-write-after-clear on BDW with a
future patch which relies on fast-clear encodings being removed after a
resolve. The avoided failure is:

   Testing implicit read of partial block UNORM -> SNORM
   Probe color at (0,1,0)
     Expected:  1.000000 1.000000 1.000000 1.000000
     Observed:  0.000000 0.000000 0.000000 0.000000

Cc: mesa-stable
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23676>

intel/fs: don't try to rebuild sequences of non ssa values

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 04777171e0 ("intel/fs: try to rematerialize surface computation code")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9378
Reviewed-by: Illia Polishchuk <illia.a.polishchuk@globallogic.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24228>

meson: Ensure that LLVMSPIRVLib is not required for Clover

Fixes: cb588d5d6ee ("compiler/clc: Move related NIR passes to the common mesa clc")
Closes: #9391
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24244>