radeonsi: fix a regression causing GPU hang with GLCTS using streamout

Move the streamout code into the streamout-only branch. The code must be
guarded by si_shader_uses_streamout(). Using xfb_stride is not enough.

Fixes: 003cbddfee3 - radeonsi: use native shader info when init streamout args

Reviewed-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20147>

dzn: Don't crash when libd3d12.so can't be found

`dzn_instance_create` will call `dzn_instance_destroy` when the d3d12
library fails to load. Just like the issue in `d3d12_screen`, this will
lead to a crash because `d3d12_mod` is NULL.

To fix this, only close the library after if it was actually opened.

Cc: mesa-stable
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20145>

d3d12: Don't crash when libd3d12.so can't be found

`d3d12_destroy_screen` is called by `d3d12_create_dxcore_screen` after
`d3d12_init_screen_base` fails and attempts to call `util_dl_close` on
a NULL pointer, leading to an abort.

To fix this, only close the library after if it was actually opened.

Cc: mesa-stable
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20145>

radeonsi: fix ngg lds base load intrinsic llvm implementation

Otherwise we get llvm assertion.

Fixes: 7e1b804992d ("radeonsi: implement two lds base load intrinsics")
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20142>

anv: Defer flushing PIPE_CONTROL bits forbidden in CCS while in GPGPU mode

Fixes: 313aeee8 ("anv: Use pending pipe control mechanism in flush_pipeline_select()
")

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7816
Signed-off-by: Sviatoslav Peleshko <sviatoslav.peleshko@globallogic.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20124>

dxil: Use nir_const_value_for_uint in dxil_nir_lower_int_samplers

This change should avoid any accidental rounding issues because of
border colors getting stored in a float in dxil_wrap_sampler_state.  It
also switches us to using the correct helpers for nir_const_value so we
can avoid any weird uninitialized data failures that can be caused by
filling out the fields in the struct directly.

Fixes: b9c61379ab4c ("microsoft/compiler: translate nir to dxil")
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19689>

r600/nir: Fix u64vec2 immediate lowering

There were a couple of issues here:

 1. We should be using nir_const_value_for_uint instead of setting the
    union fields directly to ensure the rest of the union is zeroed.

 2. It was always filling out the first two components of val even if
    the incoming constant had 2 64-bit components.

Fixes: 165fb5117bf7 ("r600/sfn: add lowering passes to get 64 bit ops lowered to 32 bit vec2")
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19689>

st/mesa: Use nir_const_value_for_bool() in ATIFS

Fixes: 0a179bb6e26b ("st/mesa: Generate NIR for ATI_fragment_shader instead of TGSI.")
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19689>

nir: Use nir_const_value_for_int in nir_lower_subgroups

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7670
Fixes: e4e79de2a420 ("nir/subgroups: Support > 1 ballot components")
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19689>

llvmpipe: Use get_first_non_void_channel more often

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18634>

virgl: Use get_first_non_void_channel more often

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18634>

radeonsi: Use get_first_non_void_channel more often

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18634>

r600: Use get_first_non_void_channel more often

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18634>

r300: Use get_first_non_void_channel more often

Reviewed-by: Pavel Ondračka <pavel.ondracka@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18634>

radv: Use get_first_non_void_channel more often

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18634>

venus: add VN_PERF_NO_CMD_BATCHING

Make it easier to debug object lifetime issues in Venus.

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19830>

venus: fix draw cmd batch accounting

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19830>

venus: add VN_PERF_NO_MEMORY_SUBALLOC

Make it easier for memory alignment debugging.

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19830>

venus: avoid no_async_buffer_create related caches with the perf option

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19830>

radv: Only create bvh pipelines when using rt

Saves some time when creating non-rt devices.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20110>

vulkan: Use vk_image_sanitize_extent

We set the image type before the extend which allows us to use
vk_image_sanitize_extent.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20112>

microsoft/clc: Add a test for nested function-temp arrays

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20111>

microsoft/compiler: Handle nested arrays correctly for emitting global consts

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20111>

compiler: Handle nested arrays correctly for computing CL size/alignment

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Sil Vilerino <sivileri@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20111>

gallium/u_debug_flush: Fix incompatible function signature warnings

Looks like an MSVC update started complaining that pipe_error is
no longer a compatible return type for function pointers that are
supposed to be int.

Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20102>

radv/rra: Get rid of annoying memory aliasing warning

Such cursed behavior is almost non existent in practise. When capturing
a Doom Eternal, this warning spams the output for no reason.

The warning is also unnecessary since we copy acceleration structures
right after building them now.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20047>

radv/rra: Fix setting some offsets

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20047>

radv/rra: Refactor rra_fill_accel_struct_header_common

No need to re-do the offset calculation for every field.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20047>

radv/rra: Set the metadata size correctly

Fixes: 5749806 ("radv: Add Radeon Raytracing Analyzer trace dumping utilities")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20047>

radv/rra: Remove an obsolete comment

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20047>

radv/rra: Defer destroying accel struct data

This allows us to dump acceleration structures that were destroyed
before present.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20047>

radv/rra: Copy accel structs directly after build

This is the second step of decoupling acceleration structure dumping
from lifetimes. It also simplifies the logic a bit.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20047>

radv/rra: Introduce radv_rra_accel_struct_data

This will be useful for dumping acceleration structures that were
destroyed before submit.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20047>

radv: Add hash_table_foreach to .clang-format

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20047>

microsoft/spirv_to_dxil: Properly handle load- and is_helper_invocation

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19908>

clc: fetch clang resource dir at runtime

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19617>

clc: generate sources only with with_microsoft_clc

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19617>

nir/nir_opt_offsets: Prevent offsets going above max

In try_fold_load_store when trying to extract const addition from
non-const offset source, we should take into account that there is
already a constant base offset, which should count towards the limit.

The issue was found in "Monster Hunter: World" running on Turnip.

Fixes: cac6f633b21799bd1ecc35471d73a0bd190ccada
("nir/opt_offsets: Use nir_ssa_scalar to chase offset additions.")

Well, the issue was present before this commit but it made a lot
of changes in surrounding code.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20099>

ci/amd: re-enable previously OOM tests

Since we have ZRAM now, we can enable previously failing tests on OOM.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19535>

ci/kernel: enable ZRAM on all archs

Let's enable ZRAM with 2G. Should help prevent peak OOM scenarios.

For more info see: https://www.kernel.org/doc/html/latest/admin-guide/blockdev/zram.html

Reviewed-by: Eric Engestrom <eric@igalia.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19535>

aco: Don't use v_lshrrev_b64 for moves on GFX11.

Looking at VOPD things, shifts are not very likely to get dual issued
but plain moves are. Looking at RDNA2 v_lshrrev_b64 are half the perf
of v_mov_b32 (but you need twice as many moves), so on GFX11 this likely
reaches the threshold where moves are faster.

Totals from 68400 (50.70% of 134906) affected shaders:

CodeSize: 275489516 -> 275459536 (-0.01%); split: -0.01%, +0.00%
Instrs: 51775474 -> 51991286 (+0.42%)
Latency: 589884847 -> 589066439 (-0.14%); split: -0.15%, +0.01%
InvThroughput: 127154986 -> 126037619 (-0.88%); split: -0.88%, +0.00%
Copies: 3756157 -> 3976193 (+5.86%)
Branches: 1259604 -> 1260072 (+0.04%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19633>

aco: Use more detailed wave64 timing for GFX10+.

Also nabbed some dual issue stuff for GFX11 from LLVM.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19633>

docs: update calendar and link releases notes for 22.3.0

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20129>

docs: add release notes for 22.3.0

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20129>

Revert "ci: disable Collabora's LAVA lab for maintance"

This reverts commit 3964a77454b616a91c78d7867d0d8cba4ffe6b63.

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20128>

ac/nir/ngg: merge multi stream gs shader queries

Before this commit each stream will emit a query block, now
we merge them to a single block.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20074>

anv: enable VK_KHR_ray_tracing_maintenance1

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20011>

anv: implement new queries for VK_KHR_ray_tracing_maintenance1

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20011>

anv: implement vkCmdTraceRaysIndirect2KHR

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20011>

anv: refactor ray tracing dispatch

Preparing for vkCmdTraceRaysIndirect2KHR

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20011>

intel/rt/nir: add support for RayCullMaskKHR

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20011>

intel/rt/nir: enable the trampoline shader to load the indirect ray shader bsr

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20011>

anv: correctly predicate ray tracing

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 7479fe6ae093 ("anv: Implement vkCmdTraceRays and vkCmdTraceRaysIndirect")
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20011>

anv/genxml: make gen_rt more like other genxml files

The main goal is to be able to generate genX_bits.h for those
structures so we can get generated field offsets.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20011>

hasvk: remove coarse pixel checks

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19852>

hasvk: Drop more DG2 code

v2: remove unused devinfo (Lionel)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19852>

hasvk: Rip out local memory support

Things could probably be simplified further but this at least gets rid
of most of the dead code and the dead flags and fields.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19852>

hasvk: Rip out scratch surfaces

These are a DG2+ thing

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19852>

hasvk: Drop SKL+ features

Most of these have already had all the code removeed.  We just need to
remove the feature bits and queries.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19852>

hasvk: Drop support for atomic_int64 and atomic_float2

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19852>

hasvk: Drop bindless image support

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19852>

hasvk: Drop A64 descriptor set support

It's only used by task/mesh and ray-tracing.  Also drop a couple
remaining ray query things and a task/mesh we left behind.

v2: Fix incorrect use of nir_load_desc_set_address_intel (Lionel)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19852>

hasvk: Drop remnants of ray queries

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19852>

hasvk: Drop CCS_E support

Oh, for the days of Broadwell and earlier where compression was called
fast-clear.  That was a simpler time.  The birds sang in the trees, the
oceans weren't brown from oil spills, and Intel surface compression was
actually comprehendable by humans.  To help the reviewer, keep the
following in mind:

 1. CCS_E is SKL+
 2. Implicit CCS is TGL+
 3. The AUX TT (AKA aux map) is TGL+
 4. HIZ+CCS, stencil CCS, and CCS for storage images are all TGL+
 4. CCS_D surfaces only ever get full resolves and MCS surfaces only
    ever get partial resolves

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19852>

hasvk: Rip out primitive replication

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19852>

hasvk: Rip out remaining traces of CPS/FSR

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19852>

hasvk/gpu_memcpy: Rip out SKL+

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19852>

hasvk/state: Rip out SKL+

v2: Fix incorrectly removed l3cr.SLMEnable setting (Lionel)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19852>

hasvk/blorp: Rip out SKL+

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19852>

hasvk/pipeline: Rip out SKL+

v2: Fix incorrect DispatchMode removal (Lionel)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19852>

hasvk/cmd_buffer: Rip out SKL+ support

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19852>

isl: don't report I915_FORMAT_MOD_Y_TILED_CCS on Gfx8

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19852>

ci: disable Collabora's LAVA lab for maintance

This is to inform you of some planned downtime in the LAVA lab as follows:
    Start: 2022-12-02 08:00 GMT
    End: 2022-12-02 12:00 GMT

Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20103>

ac/llvm,radeonsi: lower attribute ring intrinsics in nir

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010>

ac/llvm,radeonsi: lower nir primitive counter add intrinsics

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010>

nir,ac/llvm: add nir_buffer_atomic_add_amd

Used by radeonsi for lower nir_atomic_add_gen/xfb_prim_count_amd.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010>

ac/llvm,radeonsi: lower nir_load_streamout_buffer_amd

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010>

ac/llvm,radeonsi: lower nir_load_user_clip_plane in abi

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010>

ac/llvm: remove lowered abi->intrinsic_load() intrinsics

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010>

radeonsi: remove si_llvm_load_intrinsic intrinsics lowered

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010>

radeonsi: add si_nir_lower_abi pass

This pass is for lower intrinsics to driver spec nir instructions,
so that each compiler backend don't need to implement their own.
Like radv_nir_lower_abi().

Currently only lower intrinsics in si_llvm_load_intrinsic().

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010>

ac/nir: add ac_nir_unpack_arg

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010>

nir,ac/llvm: add nir_load_smem_buffer_amd

Used by radeonsi to load const buffer.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010>

ac/llvm: nir_load_smem_amd support 32bit base address

For radeonsi which use 32bit address in ac_build_load_to_sgpr().

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010>

radeonsi: separate shader args from llvm

Move shader args out of llvm context, so that we can init
it before get nir. This is for creating a nir lower abi pass
which load args directly in nir.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010>

radeonsi: use native shader info when init streamout args

We are going to init shader args earlier, there is no such
pipe_stream_output_info when that time.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010>

asahi: Use PIPE_CAP_VERTEX_ATTRIB_ELEMENT_ALIGNED_ONLY

The hardware only supports aligned loads and stores. That applies to vertex
buffer loads as well. As such, we need to ensure that the base address of vertex
buffers, the stride, and the offset are all aligned to the vertex buffer format,
ensuring that the load itself is aligned. Mesa has a CAP for that,
PIPE_CAP_VERTEX_ATTRIB_ELEMENT_ALIGNED_ONLY, which ensures that these conditions
are met and will rewrite a vertex buffer on the CPU in the off chance that
they're not.

This is a bug fix compared to the old code, because it requires that offsets and
base addresses are aligned (not just the strides like before). It's also an
optimization compared to the old code, because it does not require 4 byte
alignment for 8-bit and 16-bit formats. In fact, it doesn't require any
alignment for 8-bit formats. This will avoid needless CPU work for smaller
formats.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19996>

agx: Lower VBOs in NIR

Now we support all the vertex formats! This means we don't hit u_vbuf for format
translation, which helps performance in lots of applications. By doing the
lowering in NIR, the vertex fetch code itself can be optimized by NIR (e.g.
nir_opt_algebraic) which can improve generated code quality.

In my first implementation of this, I had a big switch statement mapping format
enums to interchange formats and post-processing code. This ends up being really
unwieldly, the combinatorics of bit packing + conversion + swizzles is
enormous and for performance we want to support everything (no u_vbuf
fallbacks). To keep the combinatorics in check, we rely on parsing the
util_format_description to separate out the issues of bit packing, conversion,
and swizzling, allowing us to handle bizarro formats like B10G10R10A2_SNORM with
no special casing.

In an effort to support everything in one shot, this handles all the formats
needed for the extensions EXT_vertex_array_bgra, ARB_vertex_type_2_10_10_10_rev,
and ARB_vertex_type_10f_11f_11f_rev.

Passes dEQP-GLES3.functional.vertex_arrays.*

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19996>

agx: Lower UBOs in NIR

Simpler than lowering in the backend and makes the sysvals obvious in the NIR.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19996>

agx: Implement 8-bit sign extensions

Long term, I think having i2i16 and i2i32 available with 8-bit sources should
make lowering the rest of 8-bit away a bit easier. Short term, this avoids
special casing 8-bit in the VBO lowering code.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19996>

agx: Allow some 8-bit sources

8-bit sources are useful for int8->float32 conversions, which we can do in a
single hardware instruction.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19996>

agx: Implement formatted loads

These will be generated by the UBO and VBO lowerings. (and eventually by other
lowerings too?)

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19996>

agx: Add shift to device_load

We'll use this as an optimization soon. This acts in addition to the format's
shift.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19996>

asahi: Use NIR_PASS_V for agx_nir_lower_tilebuffer

This ensures that printing shaders before and after the NIR pass still works
with the standard NIR debug options.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19996>

nir: Add intrinsics for lowering UBOs/VBOs on AGX

We'll use formatted loads and some system values to lower UBOs and VBOs to
global memory in NIR, using the AGX-specific format support and addressing
arithmetic to optimize the emitted code.

Add the intrinsics and teach nir_opt_preamble how to move them so we don't
regress UBO pushing.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19996>

intel/nir/rt: switch to workgroup_id_zero_base

RT don't use a base workgroup id so no reason of using workgroup_id.
Additionally the lowering introduced in b4dd3df227 requires something
provides base_workgroup_id which we don't have for RT as it's not
needed.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: b4dd3df227 ("intel/nir: Set has_base_workgroup_id for lower_compute_system_values")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7812
Reviewed-by: Mark Janes <markjanes@swizzler.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20115>

radeonsi: cleanup si_llvm_build_vs_exports gfx11 code

It's now completely handled in ac_nir_lower_ngg.c
export_vertex_params_gfx11.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17109>

ac/llvm: remove unused llvm cull

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17109>

radeonsi: remove unused ngg llvm code

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17109>

radeonsi: replace llvm ngg gs with nir lowering

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17109>