v3dv: Update texture padding logic to match v3d changes

Piglit tests for v3d highlighted issues with the padding
computation when allocating memory for slices. This change
moves the fixes from v3d to v3dv.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23260>

v3d: Add support for ASTC texture compression

Add proper support for ASTC texture compression in the v3d
gallium driver, instead of relying on the fallback software
conversion from gallium, as the hardware has native support
for ASTC compressed textures.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23260>

vulkan: use cmd size array for queued cmd allocations

minor simplification for consistency

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23322>

vulkan/cmd_queue: expose cmd sizes

now that cmds are more precisely allocated, it's necessary for drivers
to have some way to determine what the allocation size is

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23322>

vulkan/cmd_queue: allocate cmds based on the size of the cmd

the base size of a vk_cmd_queue_entry is massive since there are a couple
union entries that have a trillion params. by allocating conditionally using
the union member size, memory can be reduced, which will affect some user-facing
api properties

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23322>

vulkan: reorder vk_cmd_queue_entry

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23322>

zink/ci: update expected results

Remove the now passing arb_compute_variable_group_size tests

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23244>

zink: fix layout(local_size_variable) for vk1.3+

Use the correct exec-mode op for LocalSizeId
Corrected typo `gl_LocalGroupSize` -> `gl_LocalGroupSizeARB`

Fixes: 99bd1eaf ("zink: use spir-v 1.6 local-size when needed")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23244>

docs/venus: advertise VK_EXT_image_2d_view_of_3d

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23090>

venus: enable VK_EXT_image_2d_view_of_3d

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23090>

venus: sync protocol for VK_EXT_image_2d_view_of_3d

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23090>

venus: sync to latest protocol from header v1.3.248

This is to make later protocol update CL to be easily backported to
older branches.

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23090>

venus: silence -Wuninitialized

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23090>

intel/dev: switch defect identifiers to use lineage numbers

Update existing workarounds when necessary to match changed
identifiers.

Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23226>

freedreno: Reallocate on unshared export

If we need to export a handle on a resource which was not originally
allocated with PIPE_BIND_SHARED, then re-allocate with shared flag and
try again.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9110
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23324>

freedreno: Add aux-context support

A global aux-context can be created on-demand for cases where we need to
(for example) blit a resource when we only have a screen ptr.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23324>

freedreno: Handle export error handling

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23324>

freedreno/drm: Don't try to export suballoc bo

Suballoc BOs don't have a real handle, so attempting to dmabuf export
won't work.  Likewise for handle or flink-name export.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23324>

nir/print: Print locations for geometry shader inputs

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23259>

intel/blorp: Use nir_trim_vector

With Coccinelle patch:

@@
expression b, V;
@@

-nir_vec2(b, nir_channel(b, V, 0), nir_channel(b, V, 1))
+nir_trim_vector(b, V, 2)

@@
expression b, V;
@@

-nir_vec3(b, nir_channel(b, V, 0), nir_channel(b, V, 1), nir_channel(b, V, 2))
+nir_trim_vector(b, V, 3)

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23259>

radv/query: Use nir_trim_vector

With Coccinelle patch:

@@
expression b, V;
@@

-nir_vec2(b, nir_channel(b, V, 0), nir_channel(b, V, 1))
+nir_trim_vector(b, V, 2)

@@
expression b, V;
@@

-nir_vec3(b, nir_channel(b, V, 0), nir_channel(b, V, 1), nir_channel(b, V, 2))
+nir_trim_vector(b, V, 3)

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23259>

pan/lower_framebuffer: Use nir_replicate

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Italo Nicola <italonicola@collabora.com>
Acked-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23259>

treewide: Use nir_replicate

Via coccinelle.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23259>

nir/builder: Add nir_replicate helper

Splat a scalar to all components of a vector.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23259>

radeonsi: return kernel queried video capability for HEVC and JPEG

Query and return the values obtained from kernel for VCN_1 and above.
Earlier the HEVC and JPEG capabilities were returned based on
pre-defined values.

Signed-off-by: Veerabadhran Gopalakrishnan <veerabadhran.gopalakrishnan@amd.com>
Reviewed-by: Thong Thai <thong.thai@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23067>

radeonsi: return kernel queried video capability for HEVC and JPEG

Query and return the values obtained from kernel for VCN_1 and above.
Earlier the HEVC and JPEG capabilities were returned based on
pre-defined values.

Signed-off-by: Veerabadhran Gopalakrishnan <veerabadhran.gopalakrishnan@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23067>

gallivm: Fix gather/scatter types for newer llvm

The types changed with opaque pointer support.

Fixes a bunch of lavapipe regressions.
Cc: mesa-stable
Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23293>

docs/calendar: add 23.2 branchpoint and release candidates

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23205>

docs: update calendar for 23.0.4

At this point I'm calling 23.0 done. Please use 23.1 for future updates

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23325>

docs: Add sha256 sum for 23.0.4

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23325>

docs: add release notes for 23.0.4

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23325>

microsoft/compiler: Use image formats to determine texture types

Fixes some tests when bindless is disabled, where the image format is
R32, we do atomics on it, but we didn't set the "typed UAV load with
additional formats" feature bit because when we loaded from it, we
only loaded one component. Since the image format on the DXIL side
was declared as U32x4, the DXIL validator said that we should have.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23266>

spirv2dxil: Assign formats to image vars before lowering to bindless

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23266>

microsoft/compiler: Add a pass to assign image formats based on number of components

For loads/stores without formats, let's guess one based on how it's used.
The actual format doesn't matter, we just want to use it for the number
of components it has.

Also copy image formats from variables to intrinsics, to ensure that
deref-based intrinsics have formats assigned and lowered intrinsics
are up to date.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23266>

microsoft/compiler: Enable emitting type info for textures with <4 comps

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23266>

radv/amdgpu: Report 48-bit VAs in bo logs.

More ergonomic when correlating with page fault addresses.

Signed-off-by: Hans-Kristian Arntzen <post@arntzen-software.no>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23256>

radv: fix gl_SampleMaskIn for sample shading

When sample shading, we need

  gl_SampleMaskIn = SampleCoverage & (PsIterMask << gl_SampleID);

Add a new shader arg, ps_iter_mask, to pass PsIterMask to ps.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23265>

ac, radeonsi: add and use ac_get_ps_iter_mask

It is more natural for ac_get_ps_iter_mask to take sample count.
Replace samplemask_log_ps_iter by ps_iter_smples in
ac_nir_lower_ps_options accordingly.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23265>

aco: fix alignment check in emit_load

align_offset already takes const_offset into consideration.  Remove the
adjustment.

Fixes: 542733dbbf7 ("aco: add emit_load helper")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9097
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23242>

anv: remove unused functions

I thought I removed those, it seems my rebase got screwed up :(

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 64f20cec28 ("anv: prepare image/buffer views for non indirect descriptors")
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23317>

anv: add support for VK_EXT_dynamic_rendering_unused_attachments

Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23294>

zink: do not lower line-smooth for non-lines

We used to do this correctly, but it seems this accidentally got
dropped.

Fixes: d80a35a7f76 ("zink: unified `zink_set_primitive_emulation_keys` and `zink_create_primitive_emulation_gs`")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23316>

zink: compute correct location for line-smooth gs

The GS and the FS needs to agree on the driver_location. But we just
used the num_outputs variable for the GS instead of matching the logic
from lower_aaline_instr in nir_draw_helpers.c.

This does that, but cleans up our copy slightly to avoid computing the
needless location, as well as using unsigned values.

This used to *mostly* work before, but only because we were lucky and
not too much crazy stuff went on with the inputs / outputs in
smooth-line cases.

Fixes: edecb66b018 ("nir: avoid generating conflicting output variables")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23316>

zink: keep gl46_optimal extensions/features sorted

These are mostly sorted, which makes it easy to know where to insert new
stuff. Let's make them completely sorted.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23316>

zink: explicitly avoid ci errors due to unrecognized extensions in VVL

lavapipe still had this from shader object, but this should always be in
place for all drivers so that old VVL doesn't block merges

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23315>

zink: update profiles schema

We technically need the 251 version, but the schema for that version
is not released yet, so this is the best we can do.

See https://github.com/KhronosGroup/Vulkan-Profiles/issues/437

This gets rid of some JSON schema-validation warnings.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23308>

nir/opt_if: use nir_alu_instr_is_comparison directly

Since 2d6233d0 ("nir: Check all sizes in nir_alu_instr_is_comparison"),
nir_alu_instr_is_comparison already returns true for comparisons with 32bit
result.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23287>

aco: improve printing of s_delay_alu

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23213>

aco: insert s_delay_alu on the linear CFG

fossil-db (gfx1100):
Totals from 10498 (7.87% of 133428) affected shaders:
Instrs: 22274711 -> 22277717 (+0.01%); split: -0.01%, +0.03%
CodeSize: 114557040 -> 114569064 (+0.01%); split: -0.01%, +0.02%
Latency: 236505186 -> 236497338 (-0.00%); split: -0.00%, +0.00%
InvThroughput: 33425052 -> 33423876 (-0.00%); split: -0.00%, +0.00%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23213>

aco: use pass_flags to recover s_delay_alu cycles

This is simpler and more accurate.

fossil-db (gfx1100):
Totals from 11678 (8.75% of 133428) affected shaders:
Instrs: 25448655 -> 25436028 (-0.05%)
CodeSize: 130364728 -> 130314220 (-0.04%)
Latency: 325247603 -> 325231064 (-0.01%); split: -0.01%, +0.00%
InvThroughput: 45901166 -> 45900022 (-0.00%); split: -0.00%, +0.00%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23213>

aco: fix update_alu(clear=true) for exports

For:
   v_mov_b32_e32 v0, 1.0
   exp mrtz v0, off, off, off
we should completely remove the ALU entry before creating the EXP's WaR entry for v0.
Otherwise, the two will be combined into an entry which will wait for
expcnt(0) for later uses of v0.

gen_alu() should also be before gen(), since gen_alu() performs the clear
while gen() creates the WaR entry.

fossil-db (gfx1100):
Totals from 3589 (2.69% of 133428) affected shaders:
Instrs: 5591041 -> 5589047 (-0.04%); split: -0.04%, +0.00%
CodeSize: 28580840 -> 28572864 (-0.03%); split: -0.03%, +0.00%
Latency: 65427923 -> 65427543 (-0.00%); split: -0.00%, +0.00%
InvThroughput: 11109079 -> 11109065 (-0.00%); split: -0.00%, +0.00%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23213>

d3d12: HEVC Encode - Fix num_subregions_per_scanline rounding

num_subregions_per_scanline need to be round up

Signed-off-by: Teng, Jin Chung <jin.chung.teng@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23247>

pvr: Fix page faults in occlusion query tests

This does not fix the tests completely, but does allow them to run to
completion and fail "properly".

Also contains a few trivial bugfixes in the same codepath.

Signed-off-by: Matt Coster <matt.coster@imgtec.com>
Reported-by: James Glanville <james.glanville@imgtec.com>
Reviewed-by: Karmjit Mahil <Karmjit.Mahil@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23100>

pvr: Handle barrier load and store flags.

This commit adds handling for {s,z}loaden and {s,z}storeen to
control loading from and storing to the stencil and depth buffer.

This commit also addressed the FIXMEs around barrier_{load,store}
which control the {s,z}{load,store}en.

Signed-off-by: Karmjit Mahil <Karmjit.Mahil@imgtec.com>
Reviewed-by: Frank Binns <frank.binns@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20487>

pvr: Reorder execution in pvr_cmd_buffer_end_sub_cmd()

This allows sub_cmd->job.run_frag to be setup before calling
pvr_sub_cmd_gfx_requires_split_submit().

Signed-off-by: Matt Coster <matt.coster@imgtec.com>
Reported-by: James Glanville <james.glanville@imgtec.com>
Reviewed-by: Karmjit Mahil <Karmjit.Mahil@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23125>

pvr: Fix out of range stream errors for geometry-only jobs on pvrsrvkm

Signed-off-by: Matt Coster <matt.coster@imgtec.com>
Reported-by: James Glanville <james.glanville@imgtec.com>
Reviewed-by: Karmjit Mahil <Karmjit.Mahil@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23125>

pvr: Do not free deferred pvr_transfer_cmd instances

Deferred pvr_transfer_cmd instances are allocated from a dyn_array on
the owning pvr_cmd_buffer and must not be freed directly.

Signed-off-by: Matt Coster <matt.coster@imgtec.com>
Reported-by: James Glanville <james.glanville@imgtec.com>
Reviewed-by: Karmjit Mahil <Karmjit.Mahil@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23125>

pvr: Rename shadowing loop variable in pvr_add_deferred_rta_clear()

Signed-off-by: Matt Coster <matt.coster@imgtec.com>
Reviewed-by: Karmjit Mahil <Karmjit.Mahil@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23125>

pvr: Use correct surface for deferred RTA clear

Signed-off-by: Matt Coster <matt.coster@imgtec.com>
Reported-by: James Glanville <james.glanville@imgtec.com>
Reviewed-by: Karmjit Mahil <Karmjit.Mahil@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23125>

pvr: Correct error flow in pvr_compute_pipeline_compile()

Fixes:
  dEQP-VK.api.object_management.alloc_callback_fail.compute_pipeline

Signed-off-by: Matt Coster <matt.coster@imgtec.com>
Reviewed-by: Karmjit Mahil <Karmjit.Mahil@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23130>

pvr: Correct error flow in pvr_graphics_pipeline_compile()

Fixes:
  dEQP-VK.api.object_management.alloc_callback_fail.graphics_pipeline

Signed-off-by: Matt Coster <matt.coster@imgtec.com>
Reviewed-by: Karmjit Mahil <Karmjit.Mahil@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23130>

pvr: Fix memory leaks on realloc failure in pvr_pipeline.c

Signed-off-by: Matt Coster <matt.coster@imgtec.com>
Reviewed-by: Karmjit Mahil <Karmjit.Mahil@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23130>

pvr: Fix allocation scopes in vkCreateRenderPass2() code path

These previously COMMAND scoped allocations are stored on the
VkRenderPass and must therefore be OBJECT scoped.

Fixes: dEQP-VK.api.object_management.single_alloc_callbacks.render_pass

Signed-off-by: Matt Coster <matt.coster@imgtec.com>
Reviewed-by: Karmjit Mahil <Karmjit.Mahil@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23130>

v3d: only warn about bining sync for indirect draw once

Reviewed-by: Juan A. Suarez <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23306>

anv: override vendorID for Cyberpunk 2077

A recent update to Cyberpunk 2077 enables XeSS code for Intel GPUs
which is causing the game to crash in the XeSS libraries.  As a
temporary work around, stop identifying as Intel for Cyberpunk so
XeSS falls back to the cross-vendor path.

References: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8860
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23271>

anv: enable direct descriptors on platforms with extended bindless offset

Aka. DG2

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: ensure descriptor addresses are used with bindless stages

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: descriptor binding for direct descriptors

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: bring back the max number of sets to 8

Not sure we bumped it to 32 for the right reasons. This generates more
push constant data and because we're not tighly packing our push
constant data this can generate more register pressure.

We could tightly pack things at the cost of some CPU cycles but only
for some stages. RT stages would have to retain the current "sparse"
version of push constants.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: add direct descriptor support to apply_layout

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: track descriptor data size

descriptor_stride includes multiple plane size, this new field tracks
just the data of one plane.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: simplify ycbcr bti computations

To make BTI indexing simpler with ycbcr samplers, stop doing packing
calculations in the apply_layout. We'll insert NULL bindings for the
few ycbcr cases where it's needed.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: implement binding table emission for direct descriptors

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: factor out dynamic buffer bti emission

No functional change. Will reuse in the followup commit.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: handle null surface in the binding table with direct descriptors

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: add helpers to build pipeline bindings

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: add support for direct descriptor in allocation/writes

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: prepare image/buffer views for non indirect descriptors

When in direct descriptor mode, the descriptor pool buffers will hold
surface states directly. We won't allocate surface states in image &
buffer views.

Instead views will hold a packed RENDER_SURFACE_STATE ready to copied
into the descriptor buffers.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: bound load descriptor mem better

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

nir: expose a couple of address format add helpers

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: add a pass to partially lower resource_intel

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: new structure to hold surface states

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: reduce push constant size for descriptor sets

Now that descriptor sets are located a in a 1Gb area, we can avoid
storing the whole address to the descriptor and add the base address
of the area to a 32bit offset.

Replay a bunch of fossils with this and changes not really significant
one way or another :

Totals:
Instrs: 9278246 -> 9277148 (-0.01%); split: -0.01%, +0.00%
Cycles: 3547598421 -> 3547579435 (-0.00%); split: -0.00%, +0.00%

Totals from 353 (1.14% of 31021) affected shaders:
Instrs: 581546 -> 580448 (-0.19%); split: -0.23%, +0.04%
Cycles: 25885422 -> 25866436 (-0.07%); split: -0.31%, +0.24%

No difference on send messages or spills/fills.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: create a pool for indirect descriptors

We'll use the fact that the pool is aligned to 4Gb to limit the amount
of address computations to build the address in the shaders.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: introduce a new descriptor set layout type

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: add an option for using indirect descriptors

This is the default for now. It needs to be part the pipeline hashing
as we will allow this to be tweaked per application.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

docs/anv: some binding table explanations

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: toggle extended bindless surface state on Gfx12.5+

We bump the max surfaces to ~16 million instead of ~1 million on
Gfx9-12. We could do more but that'll come later.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: increase workaround BO so that we can hold a full 4Kb page of 0s

At the beginning of the buffer is located the driver identifier for
error states.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: move pipeline active_stages to common structure

And fill it out for all types of pipelines.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: track pipeline in anv_cmd_pipeline_state

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: bail flush_gfx_state when not gfx push constant is dirty

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: remove incorrect ifdef

This is a leftover from a previous fix attempt. We don't need this.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: fix null descriptor handling with A64 messages

global load/store (or A64 messages) need the NIR bound checking which
is enabled by "robust" behavior even when robust behavior is disabled.

Many thanks to Christopher Snowhill for pointing out the pushed
constant related issue with the initial version of this patch.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

anv: remove unused define

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

intel/fs: try to rematerialize surface computation code

This helps a lot with accessing surface handles in control flow. Our
resource_intel intrinsic has a non_uniform flag, in which case we
cannot apply this optimization. But in uniform cases, this is just a
massive win. We drop all kind of pipeline stalls due to
find_live_channel. We also reduce register pressure by doing the
surface handle computation in a single GRF (instead of 2 or 4).

There are some regressions in max dispatch width but those I think are
only on SIMD32 and due to the current heuristic disabling it after
throughput comparison with SIMD16. We know this heuristic is not
perfect, it should probably be updated in another change.

Here are some stats (all titles seem to have similar gains) :

 PERCENTAGE DELTAS    Shaders   Instrs    Cycles  Subgroup size Send messages Spill count Fill count Scratch Memory Size Max live registers Max dispatch width
 red_dead_redemption2 5860     -36.80%    -5.67%      +0.77%        +0.06%      -81.26%     -79.16%        -70.62%             -8.63%             -6.93%
 ---------------------------------------------------------------------------------------------------------------------------------------------------------------
 All affected         4716     -37.29%    -5.67%      +0.95%        +0.07%      -81.26%     -79.16%        -70.62%             -9.15%             -8.47%
 ---------------------------------------------------------------------------------------------------------------------------------------------------------------
 Total                5860     -36.80%    -5.67%      +0.77%        +0.06%      -81.26%     -79.16%        -70.62%             -8.63%             -6.93%

 PERCENTAGE DELTAS          Shaders   Instrs    Cycles  Subgroup size Send messages Spill count Fill count Scratch Memory Size Max live registers Max dispatch width
 rise_of_the_tomb_raider_g2 12010    -37.19%   -22.12%      +0.01%        +0.00%      -99.01%     -99.14%        -98.65%             -7.62%             -4.96%
 ---------------------------------------------------------------------------------------------------------------------------------------------------------------------
 All affected               11732    -37.27%   -22.14%      +0.01%        +0.00%      -99.01%     -99.14%        -98.65%             -7.67%             -5.11%
 ---------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Total                      12010    -37.19%   -22.12%      +0.01%        +0.00%      -99.01%     -99.14%        -98.65%             -7.62%             -4.96%

 PERCENTAGE DELTAS    Shaders   Instrs    Cycles  Spill count Fill count Scratch Memory Size Max live registers Max dispatch width
 total_war_warhammer2 462      -27.45%   -12.42%    -82.35%     -88.46%        -66.67%             -5.52%             -5.62%
 -----------------------------------------------------------------------------------------------------------------------------------
 All affected         335      -28.31%   -12.77%    -82.35%     -88.46%        -66.67%             -6.25%             -7.24%
 -----------------------------------------------------------------------------------------------------------------------------------
 Total                462      -27.45%   -12.42%    -82.35%     -88.46%        -66.67%             -5.52%             -5.62%

 PERCENTAGE DELTAS Shaders   Instrs    Cycles  Subgroup size Send messages Spill count Fill count Scratch Memory Size Max live registers Max dispatch width
 witcher_3_dxvk_g2 1049     -36.94%   -57.82%      +0.06%        +0.01%      -98.52%     -97.29%        -98.10%             -7.81%             -1.00%
 ------------------------------------------------------------------------------------------------------------------------------------------------------------
 All affected      693      -41.93%   -58.45%      +0.09%        +0.01%      -98.52%     -97.29%        -98.10%             -10.25%            -1.33%
 ------------------------------------------------------------------------------------------------------------------------------------------------------------
 Total             1049     -36.94%   -57.82%      +0.06%        +0.01%      -98.52%     -97.29%        -98.10%             -7.81%             -1.00%

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

intel/fs: enable uniform block accesses through bindless heap

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

intel/fs: enable bindless sampler state offsets

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

intel/fs: enable extended bindless surface offset

Gives use 4Gb of bindless surface state on Gfx12.5+ instead of 64Mb.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>

intel/fs: enable get_buffer_size on bindless heap

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>