iris: crocus: Use shader_info::is_arb_asm flag

...instead of looking for "ARB" in the name of the shader.  This matches
the behavior of i965.  Using "ARB" was added in a1ebac3750e ("iris:
Implement ALT mode for ARB_{vertex,fragment}_shader"), but there's no
explanation of why that method was used.

v2: Just use shader_info::is_arb_asm everywhere instead of
iris_uncompiled_shader::use_alt_mode.  Suggested by Ken.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12858>

freedreno/a6xx: Pre-bake SO-disable stateobj

No need to re-create this every time we transition from stream-out
enabled to disabled.

Creation of streamout_disable_stateobj is deferred until we create
a program state using streamout to avoid creating it unnecessarily
and because fd6_prog_init() is called before ctx->pipe is created.
(Changing that ordering is complicated by the fact that u_blitter
copies pctx->bind_fs_state(), and friends.)

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12918>

zink: don't leak drm fd on drmPrimeFDToHandle failure

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12911>

freedreno: Add perf warning for WC readback

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11176>

freedreno: Used cached coherent for staging resources

These are really only accessed by the GPU once, so CPU access speed is
more important.  Especially for PIPE_MAP_READ.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11176>

freedreno/drm: Use cached-coherent for control bo

Userspace frequently reads the elapsed fence, but the GPU only writes it
once per submit.  So this should be another useful place for cached-
coherent.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11176>

freedreno/drm: Use cached-coherent cmdstream buffers

Some more extreme examples, like gl_driver2_off, can be bottlenecked on
writes to cmdstream.  OTOH the CP is pretty pipelined in how it slurps
in memory, so the penalty of using coherent buffers should not be so
much.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11176>

freedreno/drm: Add cached-coherent bo support

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11176>

freedreno/drm: Don't return shared/control bo's to cache

They can never be allocated from the cache, as fd_bo_state() would
return FD_BO_STATE_UNKNOWN

Fixes: 7dabd624649 ("freedreno/drm: Userspace fences")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11176>

freedreno/drm: Consider allocation flags in bo-cache

It hasn't really mattered until now, as we keep a separate cache for
cmdstream (which is FD_BO_GPU_READONLY), and the only other flag so
far is FD_BO_SCANOUT (which the bo cache probably messes up, but it
does not matter on most hw, and on hw where it does the scanout buffer
will be imported (and therefore won't end up in the bo cache).

But when we add cached-coherent (or if we wanted to use GPU_READONLY
more) it starts to matter.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11176>

freedreno/drm: Rename bo->flags to bo->reloc_flags

Next patch adds alloc_flags, lets rename bo->flags first to make it
clear *which* flags these are.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11176>

freedreno/drm: Garbage collect unused bo_cache

The ring_cache that is actually used is in fd_device.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11176>

ci/bare-metal: add etnaviv

Add deqp gles2 CI run for GC2000.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Acked-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12852>

ci/bare-metal: add support for eth008 power relay

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Acked-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12852>

ci/bare-metal: add telnet based serial

Makes it possible to use e.g. a ser2net server in the lan.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Acked-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12852>

ci: update kernel

Switch to v5.13-rc5-for-mesa-ci-2bb5d9ffd79c branch, which includes
etnaviv MMU patches.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Acked-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12852>

ci: include etnaviv support in ARMHF container.

Build the kernel with CONFIG_DRM_ETNAVIV=y and include
imx6q-cubox-i.dtb.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Acked-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12852>

nir/algebraic: distribute fmul(fadd(a, b), c) when b and c are constants

This allows for more MAD/FMA instructions to be created.

fossil-db (Sienna Cichlid):
Totals from 50134 (33.46% of 149839) affected shaders:
VGPRs: 2436536 -> 2436000 (-0.02%); split: -0.05%, +0.03%
SpillSGPRs: 13136 -> 13135 (-0.01%); split: -0.02%, +0.02%
CodeSize: 206621424 -> 206278292 (-0.17%); split: -0.23%, +0.07%
MaxWaves: 1116804 -> 1117448 (+0.06%); split: +0.07%, -0.01%
Instrs: 38977460 -> 38862886 (-0.29%); split: -0.33%, +0.04%
Latency: 832425389 -> 827432260 (-0.60%); split: -0.63%, +0.03%
InvThroughput: 184193457 -> 183563350 (-0.34%); split: -0.37%, +0.03%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7458>

freedreno: Remove dead fd_batch_reset().

Unused since 58f5605124a7 ("freedreno: Handle full blit discards by
invalidating the resource.")

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11511>

freedreno: Use a BO bitset for faster checks for resource referenced.

When moving the batch cache to the context, I added hash table lookups
from batch to rsc for "is this resource in use" because we could no longer
store data in the rsc bo under the batch cache's lock.

We can save that cost by tracking a bitfield of resources referenced by
the batch, which gives us very cheap checks in the draw path at a minor
cost in memory.  We can just use the GEM BO handle, since it's a nice
small integer already (we can't use the TC buffer ID, because the frontend
changes that, and we're in the driver thread).

This required moving the !pending() assert up in resource shadowing, since
the BO swap meant we were checking pending on the wrong resource.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11511>

freedreno: Remove the submit lock locking.

I think the whole submit lock thing should be possible to remove now, but
just getting rid of the lock since we're no longer sharing batches between
ctxes means another several percent draw overhead improvement.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11511>

freedreno: Move the batch cache to the context.

Our draw call rate was significantly limited by the atomics we had to do
to manage access to the batch cache.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11511>

freedreno: Use TC's flag for whether get_query is in the driver thread.

In moving batch cache to the context, the check for whether there's
pending work being done to this resources ends up accessing the context,
so we can't do it outside of the fd_context_access_begin().  This flag
lets us do the driver-thread asserts before we've decided whether we need
to flush.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11511>

freedreno: Precompute resource pointer hash values.

It was around half a percent of the CPU in the fd-bc-ctx branch, and adds
4b to our 472b struct.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11511>

aco/nops: fix handle_raw_hazard_internal when visiting the current block

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12720>

aco/nops: add State

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12720>

aco/nops: create handle_raw_hazard_instr helper

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12720>

aco/nops: use up-to-date mask_size

fossil-db (Pitcairn):
Totals from 6 (0.00% of 129702) affected shaders:
CodeSize: 8760 -> 8736 (-0.27%)
Instrs: 1714 -> 1708 (-0.35%)
Latency: 12325 -> 12302 (-0.19%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12720>

ir3: Make trig replacement expression exact

This prevents other optimizations from associating the inner multiply,
which can add inaccuracies that can lead to discontinuities around the
boundary of the ffract. We should use exactly the sequence that the blob
uses to avoid problems.

Since fadd + fmul cannot be combined to ffma when exact is specified, we
have to use ffma ourselves.

Fixes artifacts in PixMark Volplosion with !7458.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12912>

aco: allow live-range splits of linear vgprs in top-level blocks

Fixes dEQP-VK.ssbo.phys.layout.random.8bit.all_per_block_buffers.46 on
GFX8.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12172>

aco: implement linear vgpr copies

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12172>

aco/tests: add regalloc.scratch_sgpr.create_vector

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12172>

aco/tests: fix finish_ra_test()

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12172>

aco: find scratch reg for sub-dword psuedo instructions which read sgprs

If there's a sgpr operand before a sub-dword operand, a scratch register
will not be found on GFX6/7.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12172>

aco: find a scratch register for sub-dword copies on GFX7 if scc is empty

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12172>

aco: rewrite print_reg_class()

Make it work for any regclass, and print linear VGPRs differently from
logical ones.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12172>

aco: add and use RegClass::resize helper

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12172>

aco: add RegClass::is_linear_vgpr helper

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12172>

radv: fix pipeline caching with robust buffer access

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12900>

util/tc: rename tc_replace_buffer_storage_func::num_rebinds and document

this parameter is only a hint, as tc provides no method for tracking cases
when a buffer is bound multiple times to the same site (e.g., multiple vertex
buffer slots will be counted as 1 bind), so rename to "minimum" to be more clear

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12898>

features: fix listing for GL_ARB_parallel_shader_compile

this isn't actually all drivers

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12909>

ac/rgp, radv: report wave size for shaders

Fills the "Wave mode" in "Pipelines" for GPUs that supports Wave32.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12896>

ac/rgp, radv: report scratch memory size for shaders

Fills the "Scatch Mem" with "Yes/No" in "Pipelines", this requires
instruction timing to be enabled.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12896>

Revert "ci/v3d: add piglit flake"

This reverts commit 7d5aea9edf358bcfc43dd1517bbebe658dce10c8.

The test isn't flaky: now that the regression is fixed we can
remove it from the flakes list.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12849>

vbo/dlist: reallocate the vertex buffer on vertex upgrade

upgrade_vertex copies save->copied.nr vertices to the vertex buffer,
so we need to make sure it has enough space to accomodate them.

This commit also drops the usage of COPY_CLEAN_4V_TYPE_AS_UNION in
this function because it always writes 4-components for all attributes,
but our buffer might be smaller. Instead, only write the needed
components.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5353
Fixes: cc57156dce0 ("vbo/dlist: rework vertex_store management")
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12849>

r300: assert that array in translate_vertex_program is initialized

Problematic usage is in case RC_OPCODE_ENDLOOP, at line
ret_addr = loops[--loop_depth];

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12880>

util/cache: run basic cache tests on the single file cache

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12745>

freedreno: Handle timeout == PIPE_TIMEOUT_INFINITE and rollover

The fd_fence_finish() may be passed a special timeout value PIPE_TIMEOUT_INFINITE.
This gets propagated all the way to get_abs_timeout(), where it gets converted to
a huge timeout value and passed down to the kernel. At least on iMX53, the kernel
may complain about this value being too large and emit a backtrace. The relevant
piece of information there is the following:

  schedule_timeout: wrong timeout value bf94984b

Per suggestion by Rob Clark, fix this in get_abs_timeout() by picking the same
rollover implementation present in etnaviv. This fixes one part of the problem
where the tv_nsec becomes larger than NSEC_PER_SEC, which is invalid.

However, the PIPE_TIMEOUT_INFINITE is sufficiently large to make tv_secs larger
than KTIME_SEC_MAX, which makes kernel-side ktime_set() return KTIME_MAX and
that in turn triggers the above "wrong timeout value N" message. Fix this by
setting the timeout to large enough value in case of PIPE_TIMEOUT_INFINITE.
While the timeout is not truly infinite, the timeout is long enough as anything
longer than a few seconds means the GPU got hung.

The "util/timespec.h" is added so we can use NSEC_PER_SEC instead of ad-hoc
constant 1000000000 . The "pipe/p_defines.h" is needed for PIPE_TIMEOUT_INFINITE.

This problem can be reliably triggered on iMX53 using Qt5 with EGLFS support,
using the qtbase examples, as follows:

  /usr/share/examples/opengl/qopenglwidget/qopenglwidget -platform eglfs

Fixes: f3cc0d27475 ("freedreno: import libdrm_freedreno + redesign submit")
Signed-off-by: Marek Vasut <marex@denx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12886>

zink: zero VkImageCreateInfo::queueFamilyIndexCount on creation

this explodes gfxreconstruct

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12905>

ci/freedreno: Drop minetest from a3xx trace testing.

It's been flaking approximately weekly since the end of June.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12906>

auxiliary/indices: convert primitive type PIPE_PRIM_PATCHES

Few tessellation related piglit test are crashing. This patch
fixes unhandled case

Tested with piglit

Reviewed-by: Charmaine Lee <charmainel@vmware.com>
(cherry picked from commit 0e1c962cc105a9330caf22266e1962b049c13454)

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12882>

lavapipe: enable KHR_spirv_1_4

One test fails but I think it's a missing barrier in the test

Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12888>

freedreno: Enable Adreno 508, 509 and 512

These GPUs attained kernel support in:
https://git.kernel.org/torvalds/c/1d832ab30ce64abe30571bc12931a296a8a27c4d

Signed-off-by: Marijn Suijten <marijn.suijten@somainline.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12874>

iris: Finish aux import in iris_resource_from_handle

This allows us to delete iris_resource_unfinished_aux_import, which
incorrectly assumed that a CCS-enabled resource needs an aux BO.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12795>

iris: Add and use get_num_planes

This function counts the number of objects in a pipe_resource linked list.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12795>

iris: Don't add a clear color BO for MC_CCS

It's unusable because ISL_AUX_USAGE_MC doesn't support fast clears.

Instead of performing this change in the if-ladder, replace the
if-ladder with a switch statement to make it clear what's going on.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12795>

iris: Allow NULL aux BOs in aux-state functions

XeHP can use CCS_E without an aux BO.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12795>

iris: Simplify an iris_use_pinned_bo call

Avoid using a helper function to get the resource BO. This fits in
better with the previous iris_use_pinned_bo calls.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12795>

iris: Split clear color and aux BO checks

CCS_E-enabled resources on XeHP have a clear color without an aux BO.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12795>

iris: Support NULL aux BOs in fill_surface_state

XeHP can use CCS_E without an aux BO.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12795>

iris: Delete iris_resource_get_clear_color

This helper simply is a wrapper to the clear color fields in the
iris_resource struct. We choose to delete it for two reasons:

1) It incorrectly asserts that the resource argument has an aux BO.
   This doesn't hold for CCS_E on XeHP.

2) The majority of functions ignore the helper anyway and access these
   fields directly.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12795>

CI/windows: Build shared-glapi, EGL, gles2

Acked-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12881>

meson: Set /Zc:__cplusplus for MSVC

Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12881>

egl: Update Windows .def to include missing exports

Reviewed-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12881>

symbols-check: Fix symbol demangling for Windows

Only strip leading underscores if there's also a trailing @
Fixes shared-glapi symbol check for x64

Reviewed-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12881>

zink: pass all modifiers through to image creation

let the driver figure these out after zink guarantees that at least one of them
will work

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12857>

zink: pre-filter multi-plane modifiers

only single plane modifiers are supported now

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12857>

zink: unbreak dmabuf handling

this does need kms handling to do literally anything.

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12857>

zink: store drm fd to screen

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12857>

nir: Stop sweeping indirects

They're no longer ralloc'd.

Fixes: 879a569884b1 "nir: Switch from ralloc to malloc for NIR instructions."
Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12884>

nir: Properly clean up nir_src/dest indirects

Now that they're no longer ralloc'd, we have to be much more careful
about indirects.  We have to make sure every time a source or
destination is overwritten, its indirect (if any) is freed.  We also
have to choose a memory ownership convention for the rewrite functions.
Assuming that they will be called with the source from some other
instruction, we choose to always make a copy of the indirect (if any).
It's the responsibility of the caller to ensure its copy of the indirect
is freed.

Unfortunately, all this extra logic is going to make
nir_instr_rewrite/move_src/dest more expensive because they now have
all the logic of nir_src/dest_copy instead of a simple struct
assignment.  Fortunately, the vast majority of rewrite calls are done by
nir_ssa_def_rewrite_uses which is an SSA-only fast-path.

Fixes: 879a569884b1 "nir: Switch from ralloc to malloc for NIR instructions."
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12884>

radv: Remove PSIZ output when it isn't needed.

PSIZ output is only needed when:
1. There is a next stage and it reads it.
2. Primitive topology is point list, in the last vertex pipeline stage.

Zink always adds this output in its vertex (and other) shaders,
because it helps Zink avoid recompiling shader variants.

However, this has a performance impact for RADV because
it needs a scalar memory load. That becomes noticeable
at high primitive rates.

The Fossil stats are unremarkable because our DB doesn't include any
shaders from Zink or D9VK, but there are a few affected shaders.

Note that there may be an increase in LDS use in some GS. This is
because with PSIZ removed the ES per-vertex LDS size is smaller, so
we can squeeze more GS threads in the same workgroup.

Fossil DB stats on Sienna Cichlid:

Totals from 14 (0.01% of 128647) affected shaders:
CodeSize: 119884 -> 119732 (-0.13%)
LDS: 235008 -> 228864 (-2.61%); split: -2.83%, +0.22%
Instrs: 23076 -> 23048 (-0.12%)
Latency: 71667 -> 71625 (-0.06%)
InvThroughput: 19155 -> 18870 (-1.49%)
Copies: 1586 -> 1572 (-0.88%)

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10725>

docs: update docs for new llvmpipe/lavapipe features

Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11816>

lavapipe: enable KHR_shader_subgroup_extended_types

Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11816>

gallivm/nir: handle subgroup reduction across all types

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11816>

lavapipe: enable KHR_shader_float16_int8

Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11816>

llvmpipe: enable FP16 and update CL + traces piglit results.

The fails will be addressed later.

This adds a fail in GLSL compiler that is due to a workaround
that fails when fp16 constants are lowered

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11816>

gallivm: use llvm intrinsics for 16-bit round/trunc/roundeven

Otherwise the inf translations don't seem to work, and the VK CTS
fails

Fixes VK CTS dEQP-VK.spirv_assembly.instruction.graphics.float16.arithmetic*

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11816>

gallivm: increase tgsi nesting call stack size

Some VK CTS tests are topping this out around 76, increase it to 80 for now.

Fixes:
dEQP-VK.spirv_assembly.instruction.graphics.float16.opvectorshuffle.*44*

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11816>

gallivm/nir: pass the correct float builder to ddx/y

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11816>

gallivm/nir: call pow with correct flt builder

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11816>

gallivm/nir: handle 16-bit exp/lod using intrinsics.

This just passes the 16-bit float versions to the llvm intrinsics

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11816>

llvmpipe: lower_flrp16

fixes a bunch of spir-v 16-bit tests

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11816>

gallivm: add 16-bit sin/cos via llvm intrinsic

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11816>

gallivvm/nir: handle non-32bit mask scatter stores

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11816>

gallivm/nir: fix f2b32

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11816>

gallivm/nir: handle conversion to 16-bit texel fetch

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11816>

gallivm: add initial support for 16-bit float builder.

This is an initial patch that is needed for OpenCL and Vulkan
support for proper 16-bit floats.

This doesn't enable the cap bit yet

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11816>

zink: cap max shader variants with inlined uniforms

avoid making a new shader for every frame forever

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12842>

zink: simplify shader variant update loop

a single continue makes this much easier to read

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12842>

zink: split out inlined uniform shader variants into separate cache

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12842>

zink: remove default_variants storage in program struct

these should naturally be the first entry in the list when it matters

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12842>

zink: replace shader module hash table with a list

this should be significantly more performant for the majority of cases
since it's rare that shaders have multiple variants outside of unit tests,
so now there can just be a list of shaders being iterated instead where the
first entry is the last used

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12842>

zink: move shader cache to gfx program struct

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12842>

zink: stop using hash table for compute programs

this is pointless since there's no variants yet

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12842>

zink: store shader key to shader module

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12842>

zink: move uniform size calc for shader keys into keybox

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12842>

zink: move shader keys to be persistent on pipeline state

save a cycle or two zeroing and populating this on every recalc

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12842>

zink: move xfb updates to just before draw

it's illegal to bind the pipeline after xfb has begun

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12842>

zink: simplify flagging last vertex stage for updating

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12842>