u_format: Add restrict to fn pointer and manual format pack/unpack/fetch

MSVC warns (loudly) about a mismatch between the generated functions in
u_format_table.c and the definition of util_format_[un]pack_description,
specifically having 'restrict' in the function but not in the pointer
type in the struct.

So, add 'restrict' everywhere - to the function types, and to the rest
of the implementations that are assigned to those structs.

v2: util_format_unpack_description is used in gallium/auxiliary/translate

Fixes: 5785fdac ("u_format: Mark the generated pack/unpack src/dst args as restrict.")
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9670>

zink: handle nir_intrinsic_image_deref_samples

same as the texop but for images

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9682>

compiler/spirv: fix image sample queries

this was only implemented for textures (I assume because drivers which implement
the corresponding intrinsic don't support multisampled images), but it's also
used for shader images

Fixes: 22fdb2f8551 ("nir/spirv: Update to the latest revision")

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9682>

zink: enable spirv extension for post depth coverage

Fixes: 3c72c867422 ("zink: Wire up ARB_post_depth_coverage")

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9683>

aco: Fix constant address offset calculation for ds_read2 instructions.

Cc: mesa-stable
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9678>

gallium/st: reserve space in default uniform block for lowered constants

If we don't reserve these, we risk these lowering passes eating up more
uniforms than we have available.

This fixes a crash due to an assert in Zink, because we end up trying to
use a too large UBO after lowering.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9631>

gallium/st: fix shader_has_one_variant

I think we need to care about this here as well, otherwise the variant
logic might be short-circuited.

Fixes: 7eb5fd98fd1 ("mesa/st: handle running nir lower passes for ucp and psiz in tess stage")
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9631>

gallium/st: fix shader_has_one_variant

I think we need to care about this here as well, otherwise the variant
logic might be short-circuited.

Fixes: ad0037fcfe1 ("mesa/st: set lower_point_size for tes/gs during program update")
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9631>

nir_to_tgsi: Fix indices for CMP in nir_to_tgsi for nir_op_fcsel

nir_to_tgsi utiliy was using wrong src for nir_op_fcsel.

Fixes regression with solidworks2012_viewport.trace and mudbox-2010sp1-basic-head.trace

v2: as per suggested by Brian, instead of relying assumption that
value will be 0.0 or 1.0 all the time, use -abs(src) to make any non-zero
value as negative.

Reviewed-by: Brian Paul <brianp@vmware.com>
Reviewed-by: Charmaine Lee <charmainel@vmware.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9658>

nir_to_tgsi: Respect PIPE_SHADER_CAP_TGSI_SQRT_SUPPORTED.

If the driver can't do it, we shouldn't be putting it in the TGSI,
regardless of what the NIR compiler above us decided.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Neha Bhende <bhenden@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9662>

zink: clear framebuffer state on context destroy

ensure we don't leak surface references

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9626>

zink: move fence reset to zink_fence_init()

fence stuff goes in fence file

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9626>

zink: remove query batch-tracking init from begin_query()

this is always created on startup now

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9626>

zink: use macro to streamline batch struct member init

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9626>

zink: move other batch-tracking implementations to unified codepath

prep for monotonic batch ids

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9626>

zink: also move batch destructor into zink_batch.c

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9626>

zink: move batch init into zink_batch.c

this consolidates more batch code into the batch file

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9626>

zink: move active query pruning to batch reset

this didn't ever really need to be on the fence in the first place

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9626>

zink: split out batch resource-set clearing into separate function

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9626>

zink: return enum zink_queue from zink_batch_reference_resource_rw()

this is a little cleaner

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9626>

zink: abstract zink_get_resource_usage() and move it to be internal

I'll be rewriting how resource tracking works, so abstracting it and removing
direct uses is going to reduce the chances of breaking things as well as code churn

plus it's a bit easier to use

downside is that until that rewrite happens, this will be a (very small) perf hit and
there's some kinda gross macros involved to consolidate code

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9626>

zink: convert ZINK_RESOURCE_ACCESS defines to enum

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9626>

zink: refactor resource_sync_writes_from_batch_usage() to manage batch id internally

this is always used the same way, so we can simplify the code a bit for future use

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9626>

zink: add enum for different queues

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9626>

zink: add a pipe_context::fence_server_sync hook

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9626>

zink: more consolidation for null sampler/image view hashing

by using the new helper functions, this is even more consistent

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9625>

zink: ci updates

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9673>

zink: always use requested format for sampler view creation

this should be fine and good

Fixes: c768c5297a8 ("zink: force stencil format for stencil-only samplers and swizzle the right component")

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9673>

gallium/u_vbuf: use updated pipe_draw_start_count while using draw_vbo

new_draw has updated draw count. Stale info in draw caused regression
with piglit gl-3.0-multidrawarrays-vertexid -indirect

fixes piglit test ./gl-3.0-multidrawarrays-vertexid -indirect

Fixes: 1cd455b17b7a ("gallium: extend draw_vbo to support multi draws")

Reviewed-by: Brian Paul <brianp@vmware.com>
Reviewed-by: Charmaine Lee <charmainel@vmware.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9656>

android: util: create some standalone compression helpers

Fixes the following building errors:

FAILED: out/target/product/x86_64/obj/SHARED_LIBRARIES/vulkan.android-x86_intermediates/LINKED/vulkan.radv.so
...
ld.lld: error: undefined symbol: util_compress_inflate
>>> referenced by disk_cache_os.c:459 (external/mesa/src/util/disk_cache_os.c:459)
...
ld.lld: error: undefined symbol: util_compress_max_compressed_len
>>> referenced by disk_cache_os.c:614 (external/mesa/src/util/disk_cache_os.c:614)
...
ld.lld: error: undefined symbol: util_compress_deflate
>>> referenced by disk_cache_os.c:622 (external/mesa/src/util/disk_cache_os.c:622)

Fixes: d7ecbd5bf837 ("util: create some standalone compression helpers")
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9613>

mesa: set states in fast path for restoring light attributes

Since states were not updated in fast path for restoring light attributes,
seen darker images in solidworks2012_viewprt.trace

Fixes regression seen with solidworks2012_viewport.trace

Fixes: 7fa9d9d06c4 ("mesa: add a fast path for restoring light attributes")

Reviewed-by: Brian Paul <brianp@vmware.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9657>

anv: Use fallback paths if DRM_I915_QUERY_ENGINE_INFO fails

Anvil can handle if this call fails, but not if we assert. :)

Reported-by: Brian Paul <brianp@vmware.com>
Fixes: 5d84c764fde ("anv: Gather engine info from i915 if available")
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9664>

intel/fs: Add support for 16-bit A64 float and integer atomics

The messages for those 16-bit operations still use 32-bit sources and
destinations, so expand them accordingly when building the payload.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8750>

spirv: Add support for SPV_EXT_shader_atomic_float_min_max

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8750>

nir: Add image atomic_fmin/fmax intrinsics

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8750>

nir: Handle deref_atomic_fadd in a couple of passes

Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8750>

spirv: Update headers and metadata from latest Khronos commit

This corresponds to bcf55210f13a4fa3c3d0963b509ff1070e434c79
("Merge pull request #178 from orbea/datadir") in
https://github.com/KhronosGroup/SPIRV-Headers.

Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8750>

lavapipe: bump maxMemoryAllocationCount

not sure why this was so low

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9644>

st/mesa: add a driconf option to transcode ETC2 to DXTC

for performance analysis

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9357>

intel/fs/vec4: add missing dependency in write-on-write fixed GRFs

If we load constant data using pull constant SENDS, and we later load
that register with some other data, we can end up in a situation where
we don't track the initial fixed register write and therefore end up
using uninitialized registers.

This tracks write-on-write of fixed GRFs like we do for normal virtual
GRFs.

v2: Fix post_alloc_reg case (Jason)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: <mesa-stable@lists.freedesktop.org>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9667>

zink: Wire up ARB_post_depth_coverage

Just a matter of passing the bits through in the right place.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9637>

mesa: fix parameter reservation size

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9360>

mesa: clear reserved parameter storage because it's stored in the shader cache

The elements might not be initialized and we don't want random bytes
in the shader cache.

Discovered by valgrind.

Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9360>

mesa: don't overallocate ParameterValues 4 times (v2)

The additional memory was never used.

v2: rework

Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9360>

mesa: fix a oldNum typo in reallocation in _mesa_reserve_parameter_storage

oldNum was incorrect. oldValNum is the correct number of elements
to copy inside realloc. (oldNum is for Parameters, not ParameterValues)

Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9360>

mesa: add assertions for buffer reference counts

Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9360>

mesa: fix Blender crash due to optimizations in buffer reference counting

The problem was that I assumed that deleted zombie buffers can't have any
references in the context, so buffers were released sooner than they should
have been.

The fix is to count the non-atomic references in the new field
gl_buffer_object::CtxRefCount. When we detach the context from the buffer,
we can just add CtxRefCount to RefCount to re-enable atomic reference
counting. This also allows removing code that was doing a similar thing.

Fixes: e014e3b6be63 "mesa: don't count buffer references for the context that created them"
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4259

Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9360>

radeonsi: use pipe_sampler_state::border_color_is_integer to simplify stuff

We don't need the separate integer sampler state if we know the border
color type.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9577>

ci: Don't run meson tests in strace for meson-mingw32-x86_64 job

There have been repeated timeouts:
https://gitlab.freedesktop.org/mesa/mesa/-/issues/3437#note_842273

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9648>

radeonsi: fix leak when the in-memory cache is full

When the hw_binary is not put in the in-memory
cache it must be freed.

Fixes: 8283ed65cfd ("radeonsi: Limit the size of the in-memory shader cache")

Signed-off-by: Axel Davy <davyaxel0@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9587>

gbm: remove fprintf calls in gbm_dri_bo_create

These errors can be handled by the caller. The caller can't guess
whether the GBM implementation supports modifiers, for instance.

Signed-off-by: Simon Ser <contact@emersion.fr>
References: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7601#note_778845
Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8715>

gbm: fail early when modifier list only contains INVALID

The current check only accomodates for a list with a single INVALID
item. However the driver won't be able to pick any modifier if the
list only contains INVALID. This includes the following cases:

- The modifier list is empty (count == 0)
- The modifier list contains more than a single item, but all items
  are INVALID

In these cases, also fail early.

Signed-off-by: Simon Ser <contact@emersion.fr>
References: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7601#note_778845
Reviewed-by: Daniel Stone <daniel@fooishbar.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8715>

zink: switch to deqp-runner for piglit jobs

There's a few changes in the expected results. First of all, there's a
few failures that are now interpreted as crashes. These test are:

- glx@glx-visuals-depth
- glx@glx-visuals-depth -pixmap
- glx@glx-visuals-stencil
- glx@glx-visuals-stencil -pixmap

Secondly, and more surprisingly, there's three tests that were
previously passing, but are now failing. These are all EGL-related, so
it's likely that there's some EGL interaction that is different with the
new runner. These tests are:

- spec@egl 1.4@eglterminate then unbind context
- spec@egl_khr_surfaceless_context@viewport
- spec@egl_mesa_configless_context@basic

commit log and skiplist by Erik Faye-Lund <erik.faye-lund@collabora.com>

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9630>

ci/freedreno: Mark the rest of the glx_arb_sync_control@timing as flakes.

IIRC I've seen -msc-delta 2 flake set, so just complete the set.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9660>

ci/freedreno: Mark an a630 piglit flake from async shader compiling.

It seems that right around when we enabled piglit, some timing also
changed so this one started flaking.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9660>

anv/apply_pipeline_layout: Add support for A64 descriptor access

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv: Do UBO loads with global addresses for bindless

This makes UBO loads in the variable pointers or bindless case work just
like SSBO loads in the sense that they use A64 messages and 64-bit
global addresses.  The primary difference is that we have an
optimization in anv_nir_lower_ubo_loads which uses a (possibly
predicated) block load message when the offset is constant so we get
roughly the same performance as we would from plumbing load_ubo all the
way to the back-end.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv: Add a pass for lowering A64 UBO access

Instead of load_global_constant_offset/bounded, we want to use the
Intel-specific block load intrinsic whenever we can.  This way we get
the same wide block loads that we usually use for constant offset UBO
pulls with a binding table.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

nir/lower_io: Support global addresses for UBOs in nir_lower_explicit_io

For nir_address_format_64bit_global_32bit_offset and
nir_address_format_64bit_bounded_global, we use a new intrinsics which
take the base address and offset as separate parameters.  For bounds-
checked access, the bound is also included in the intrinsic.  This gives
the drive more control over the bounds checking so that UBOs don't
suddenly become massively more expensive.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv/apply_pipeline_layout: Use the new helpers for images

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv/apply_pipeline_layout: Use the new helpers for early lowering

This also means that some of the newly added helpers need to grow a bit
to support VK_DESCRIPTOR_TYPE_INLINE_UNIFORM_DATA_EXT.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv/apply_pipeline_layout: Rework the desc_addr_format helper

We're about to add a new helper which is more detailed.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv/apply_pipeline_layout: Refactor all our descriptor address builders

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv/apply_pipeline_layout: Apply dynamic offsets in load_ssbo_descriptor

This function has exactly two call sites.  The first is where we had
these calculations before.  The second only cares about the size of the
SSBO so all the extra code we emit will be dead.  However, NIR should
easily clean that up and this lets us consolidate things a bit better.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv: Zero out the last dword of UBO/SSBO descriptors in the shader

This way, NIR can constant fold it.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv: Rework the 64bit_bounded_global resource index format

Instead of packing the descriptor offset into the packed portion, use
that unused channel we have lying around.  This potentially allows for
larger descriptor sets.  We also re-arrange the components a bit to make
it more like the 64bit_bounded_global memory address format.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv: Use 64bit_global_32bit_offset for SSBOs

This has the advantage of giving us cheaper address calculations because
we can calculate in 32 bits first and then do a single 64x32 add.  It
also lets us delete a bunch of code for dealing with descriptor
dereferences (vulkan_resource_reindex, and friends) because our bindless
SSBO pointers are now vec4s regardless of whether or not we're doing
bounds checking.  This also unifies UBOs and SSBOs.  The one down-side
is that, in certain variable pointers cases, it may end up burning more
memory and/or increasing register pressure.  This seems like a worth-
while trade-off.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

nir: Add a new 64+32-bit address format

This is a global address format where you have a 64-bit base pointer and
a 32-bit offset.  It's intentionally identical to 64bit_bounded_global
except nir_lower_explicit_io does no bounds checking with it.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv/apply_pipeline_layout: Add some switch statements

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv/apply_pipeline_layout: Plumb through a UBO address format

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv/apply_pipeline_layout: Move bounds checking later for index/offset

Instead of doing the array check at the load_vulkan_resource_index
intrinsic, stuff it in the vec2 and handle it at load_vulkan_descriptor
time.  This allows the bounds check to take any re-index intrinsics into
account.  This only affects variablePointers + SSBOs + Gen7.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv/apply_pipeline_layout: Run DCE between the early and late passes

This allows us to ignore UBOs in the late code going forward.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv/apply_pipeline_layout: Lower UBO loads in the early pass

We're about to enable bindless UBOs via A64 memory access like we do for
SSBOs.  In order to prevent 100% of UBOs from hitting that path, we
enable them in the early lowering.  This way we'll still get binding
table-based UBO access for any non-bindless ones.  In particular, we
need this for UBO pushing to work.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv/apply_pipeline_layout: Rework the early pass index/offset helpers

Rewrite them all to work on an index/offset vec2 instead of some only
returning the index.  This means SSBO size handling is a tiny bit more
complicated but it will also mean we can use them for descriptor buffers
properly.

This also fixes a bug where we weren't bounds-checking re-index
intrinsics because we applied the bounds check at the tail of the
recursion and not at the beginning.

Fixes: 3cf78ec2bd "anv: Lower some SSBO operations in..."
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv/apply_pipeline_layout: Refactor descriptor chasing code

This makes things a bit more generic for use in the next commit.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv: Use nir_shader_instructions_pass in apply_pipeline_layout

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

anv: Use load_global_constant for shader constants

NIR can do a bit better job optimizing this version.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

intel/fs,rt: Add a predicate to load_global_const_block

This allows us to do bounds checked A64 block load without the it being
counted as control-flow by NIR.  This means that NIR optimizations like
CSE will be able to work on these the same as a regular load.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>

ci/bare-metal: Restart a run on intermittent kernel lockups.

Since enabling SMP on db820c and cranking up how many tests we run, we've
been seeing lockups like this a couple of times a week.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9655>

freedreno/drm: Avoid unitialized timestamp in submit fail

Saw a flood of "waiting on invalid fence" with a completely bogus
looking fence # in a log of a rather strange low-memory crash.  Not
sure if it is coming from memory corruption in userspace, but if a
submit ioctl is failing due to failed allocation (or other reason)
we would get left with random stack garbage as the fence #.  Let's
not have that as a potential problem.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9638>

aco: use a single instruction for uadd32_sat() on GFX8

fossil-db (GFX8):
Totals from 8 (0.01% of 147787) affected shaders:
SGPRs: 352 -> 368 (+4.55%)
CodeSize: 49576 -> 48788 (-1.59%)
Instrs: 9487 -> 9318 (-1.78%)
Latency: 49935 -> 49607 (-0.66%)
InvThroughput: 138493 -> 137443 (-0.76%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9598>

aco: use uadd32_sat() helper for nir_op_uadd_sat

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9598>

aco: implement 64-bit VGPR {u,i}find_msb

This can be created by subgroupBallotFindMSB().

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4458
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9598>

ac/gpu_info: fix more non-coherent RB and GL2 combinations

It ignored non-harvested chips with a non-power-of-two memory bus.

Fixes: abed921ce71 - amd: add support for Navy Flounder

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9568>

zink: fix free of ralloced pointer

When we alloc with ralloc, we also need to free with it.

But let's take a step back here; we don't just need to use ralloc, we
also need to destroy all other screen-resources. So let's call the
destructor here instead.

Fixes: 2643f9ed284 ("zink: ralloc screen objects")
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9647>

zink: fix emulation of no mipfilter

This approach is taken from the Vulkan spec[1], where a robust maxLod
of 0.25 is proposed instead of 0.0.

This has the effect of allowing room for both minification and
magnification filters, yet still rounding down to the right miplevel in
the end.

[1]: https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkSamplerCreateInfo.html#_description

Fixes: 8d46e35d16e ("zink: introduce opengl over vulkan")
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9649>

aco: Delete superfluous tess and ESGS I/O code.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

radv/llvm: Delete superfluous tess and ESGS I/O code.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

radv/llvm: Only store TCS outputs where they are really needed.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

radv: Use new, NIR-based I/O lowering.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

radv: Reorder some NIR optimizations in preparation for the I/O changes.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

radv: Fill some tess shader info earlier.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

radv: Determine tcs_in_out_eq in radv_pipeline instead of the compiler.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

radv: Calculate tess patches and LDS use outside the backend compilers.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

radv: Save I/O usage data to both shader infos for merged stages.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

radv: Lower IO and set driver locations earlier.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

ac: Add NIR passes to lower ES->GS I/O to memory accesses.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

ac: Add NIR passes to lower VS->TCS->TES I/O to memory accesses.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

ac/llvm: Emit more efficient code for load_shared.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>

ac/llvm: Add constant offset to load/store_shared.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>