lavapipe: store binding offset for buffers

cc: mesa-stable

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19274>

crocus: fix off-by-one error when clearing stale syncobjs

This is the same fix as commit 1311eddd526 ("iris: fix off-by-one error
when clearing stale syncobjs"), but for crocus.

Signed-off-by: Nicholas Bishop <nicholasbishop@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19282>

clover/llvm: add kernel arg info for opaque pointers

With opaque pointers you can no longer the info from the pointertype,
so rebuild it.

v2: add a comment

Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19103>

d3d12: Add cache managers for separate buffer usage cases

Mis-matched usage is a large percentage of buffer cache misses when
searching for applicable buffers. Separating these into separate
managers puts them into separate heaps and eliminates a significant
amount of CPU time spent searching.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19283>

freedreno/ir3: Switch to NIR for a5xx's vertex id lowering.

This was similarly done for a3xx & a4xx with 4bdd226ab61
("freedreno/ir3: Switch to NIR for a3xx/a4xx's vertex id lowering.")
already, yet a5xx was missed which I noticed as Minecraft window
titlebar text corruption on OnePlus 5T with Adreno 540.

Cc: mesa-stable
Signed-off-by: Jami Kettunen <jami.kettunen@protonmail.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19263>

crocus: Advertise PIPE_CAP_NATIVE_FENCE_FD

Enables EGL_ANDROID_native_fence_sync.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7253
Signed-off-by: Nicholas Bishop <nicholasbishop@google.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19235>

v3d/ci: Add a flake

I've seen this one flake, add it to flakes.txt to avoid blocking CI.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19259>

freedreno: Passthrough TCS support

Fixes 17 little piglits.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7518
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19259>

freedreno: Add tess_state support

Add support for setting default tess params, used by passthrough TCS.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19259>

freedreno/ir3: Passthrough TCS support

Wire up support for the two intrisics to get default tess levels (which
adds new HS driver params) and add a helper to generate a passthrough
TCS for a given VS.  The passthrough TCS is cached in the VS, indexed by
patch_vertices, as the generated TCS is a function of the VS outputs and
the patch_vertices count.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19259>

freedreno: Dirty the TCS state if patch_vertices change

If using a passthrough TCS shader, we can't rely on having a new TCS
stage bound, so we need to invalidate the TCS state when the # of
patch_vertices changes.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19259>

radeonsi: Use nir passthrough TCS helper

Switch over to using the common nir helper.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19259>

nir: Add helper to create passthrough TCS shader

Based on si_create_passthrough_tcs() as that seemed the most generic of
the various different backend driver implementations.  Uses the
load_tess_level_outer_default and load_tess_level_inner_default
intrinsics to load the gl_TessLevelOuter and gl_TessLevelInner values,
so driver will somehow need to implement those to load the values set
by pipe_context::set_tess_state() or similar.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19259>

ci/panfrost: re-enable traces on t760

Also make a note, why we don't run gles2 piglit.

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19211>

ci/panfrost: Humus Portal trace got fixed, update checksum

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19211>

ci/panfrost: enable piglit-gl on g52 again and deparalelize

The job fits into 15 minutes of runtime, so deparalelize.

Stress-tested.

Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19211>

ci/panfrost: deduplicate gitlab-ci.yml

Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19211>

r300: respect buffer offset in r300_set_constant_buffer

Fixes constants upload with nine.
Thanks to Marek Olšák for pointing this out.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7481
Cc: mesa-stable
Signed-off-by: Pavel Ondračka <pavel.ondracka@gmail.com>
Reviewed-by: Filip Gawin <filip@gawin.net>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19277>

aco: Add ACO_DEBUG=novalidateir option.

This disables IR validation in debug/debugoptimized builds.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18103>

aco: Move is_dead to aco_ir.h to allow it to get inlined.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18103>

aco/optimizer_postRA: Delete dead instructions more efficiently.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18103>

aco/optimizer_postRA: Properly handle vccz/execz/scc in reset_block.

Fixes: a8dd07518c59af0087ed311cee232c31c3e8268c
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18103>

aco/optimizer_postRA: Speed up reset_block() with predecessors.

Copy the information from the first predecessor then check whether
it matches other predecessors and modify the data accordingly.

Marked for backporting to stable to make it possible to also
backport fixes based on this.

Cc: mesa-stable
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18103>

aco/optimizer_postRA: Use unique_ptr + array for instruction indices.

According to perf, this roughly halves the impact of the post-RA
optimizer in ACO's compile times.

Measurement was taken using a debug optimized build using
NIR_DEBUG=novalidate RADV_DEBUG=nocache and replaying the Fossil DB
from the Doom Eternal shaders.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18103>

mesa: Fix stack corruption for PIPE_QUERY_TIMESTAMP

get_query_result expects a pointer to a union pipe_query_result,
which is larger than GLuint64EXT, causing the memset it does to
overwrite the stack.

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19055>

radeonsi: Use available dot product instructions on gfx11.

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19114>

radv: Use available dot product instructions on gfx11.

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19114>

aco: Implement signed idot instructions on GFX11.

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19114>

ac/llvm: Implement signed idot on GFX11.

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19114>

aco: Don't use opsel for p_insert.

This doesn't make sense, opsel preserves the not selected half of the register,
p_insert zeros it.

No Foz-DB changes.

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: 54292e99c78 ("aco: optimize 32-bit extracts and inserts using SDWA")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19253>

aco: fmaak/fmamk can't use SDWA.

Found by inspection.

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19255>

microsoft/spirv2dxil: Support pack_32_2x16 and pack_32_4x8

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19273>

aco: Do not define NOMINMAX as it's already defined in pre_args now

It's a revert of fcd53bebe68 ("aco: Define NOMINMAX in Meson build file")
Because 852d91edcd4 ("windows: Always set NOMINMAX to remove min/max macros")
did the same thing

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19233>

zink: reuse rast_prim for line-rasterization check

The check_warn variable is true a bit too often; in realtity it's not
*either* of these conditions that makes things lines, it's the last one
in the pipeline. But we already have a state for this, so let's reuse
that instead of recomputing.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19048>

zink: rework primitive rasterization type logic

There's a few things that depends on the primitive rasterization type,
like point-sprite lowering and polygon offset. The effective state is a
combination of several other states, and we currently kinda wing it a
bit sometimes.

This should improve the situation. In particular, we now go backwards
through the pipeline, checking one overriding state at the time.

The end result should be that we don't end up lowering point-coord
replacement when not rendering with points.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19048>

microsoft/spirv2dxil: Change `spirv_to_dxil()` to receive SM and validator versions

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19227>

microsoft/spirv2dxil: Refactor to make SM and validator versions public

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19227>

vulkan: wsi: fix musl build

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19270>

radv/rt: Load instance id and custom index on demand

Stats for Quae II RTX:

57fps -> 57fps

Totals from 7 (14.00% of 50) affected shaders:
VGPRs: 800 -> 784 (-2.00%)
CodeSize: 217868 -> 218308 (+0.20%)
MaxWaves: 62 -> 63 (+1.61%)
Instrs: 40384 -> 40420 (+0.09%); split: -0.01%, +0.10%
Latency: 866315 -> 870692 (+0.51%)
InvThroughput: 199189 -> 196595 (-1.30%); split: -1.75%, +0.45%
VClause: 1058 -> 1077 (+1.80%)
SClause: 1126 -> 1130 (+0.36%)
Copies: 5787 -> 5772 (-0.26%); split: -0.40%, +0.14%
PreVGPRs: 764 -> 750 (-1.83%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19159>

aco/ra: only rename fixed Operands if the copy-location matches

With epilogs, the same Operand can be fixed to multiple registers.

Fixes: 061b8bfd2980a5ed903febef42be288ab1eb4e40 ('aco/ra: rework fixed operands')
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18485>

radv: disable shaderBufferFloat64AtomicMinMax on GFX11

buffer_atomic_{fmin,fmax}_x2 aren't supported on this GPU apparently.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19268>

radv: disable {shader,sparse}ImageFloat32AtomicMinMax on GFX11

image_atomic_{fmin,fmax} aren't supported on this GPU apparently.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19268>

ac: fix has_vrs_ds_export_bug for VanGogh

Missed it.

Fixes: 0a8a9d9d638 ("ac: add radeon_info::has_vrs_ds_export_bug")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19272>

st: Remove struct st_api,  st_gl_api_create and st_gl_api.h

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19197>

xlib: Implement xmesa_get_name by return literal "Mesa " PACKAGE_VERSION directly

This literal is consistence with st_api::name comes from global variable
st_gl_api that will be removed in following commits

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19197>

st: Turn st_api::query_versions to be direct accessed function st_api_query_versions

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19197>

st: Turn st_api::create_context to be direct accessed function st_api_create_context

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19197>

st: Turn st_api::make_current to be direct accessed function st_api_make_current

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19197>

st: Turn st_api::get_current to be direct accessed function st_api_get_current

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19197>

st: Turn st_api::destroy_drawable to be direct accessed function st_api_destroy_drawable

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19197>

st: Remove st_api::destroy

It's a global variable and have no need destroy

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19197>

st: OpenVG are removed, remove st_api::feature_mask as it's constant

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19197>

st: OpenVG are removed, remove st_api::profile_mask as it's constant

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19197>

st: move function st_visual_have_buffers into st_manager.c

As it's only accessed in st_manager.c, there is no need expose it

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19197>

hgl: Remove non-nused hgl_create_st_api

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19197>

aco: swap v_perm_b32 operands

I misread the ISA doc and got the order wrong.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Fixes: dae1629778d ("aco: disable sdwa on gfx11")
Fixes: e68e6c75ca1 ("aco: use v_perm_b32 to copy 0xff00/0x00ff/0xff/0x00")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19223>

radv: Use correct primgrp size for gfx11.

From radeonsi, found by inspection.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19222>

radv: Improve gfx11 clear colors for DCC.

When I wrote the function I did unorms wrong. Let's copy radeonsi
harder.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19222>

v3d: replace all TODO around for FIXME

Even if there is a slight difference of meaning between FIXME and
TODO, at some point we agreed to use just FIXME for all pending things
to do, just to make it easier to grepping for things that can be done.

And after all, one could argue that is there is something pending TO
DO, is that needs FIXING.

Acked-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19225>

radeonsi: zero init using the correct components count

This fixes the replay of the trace attached to
https://gitlab.freedesktop.org/mesa/mesa/-/issues/7530 which
would otherwise fail with an LLVM error:
  "All operands to PHI node must be the same type as the PHI node!"

Fixes: ae4379d81e4 ("ac/nir: export some undef as zero")
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19189>

aco,radv/llvm: do not export parameters on GFX11

They will be exported through the attribute ring instead.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19216>

ac/nir,radv/llvm: add support for the attribute ring

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19216>

radv/llvm: do not set COMPR for exports on GFX11

Use 0x3 channel mask instead.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19216>

hasvk: Nuke code around local memory

None of the platforms supported by this driver supports local memory.

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19240>

freedreno: get timestamp from os_time_get_nano instead of os_time_get

Signed-off-by: António Monteiro <antonio.fmr.monteiro@gmail.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19260>

nouveau: get timestamp from os_time_get_nano instead of os_time_get

Signed-off-by: Yusuf Khan <yusisamerican@gmail.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19249>

util: unify FreeBSD futex_wait signature with Linux/OpenBSD/Windows

src/util/futex.c:73:5: error: conflicting types for 'futex_wait'
int futex_wait(uint32_t *addr, int32_t value, struct timespec *timeout)
    ^
src/util/futex.h:50:5: note: previous declaration is here
int futex_wait(uint32_t *addr, int32_t value, const struct timespec *timeout);
    ^

Fixes: 095dfc6caa29 ("util: Move the implementation of futex_wake and futex_wait from futex.h to futex.c")
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19177>

driconf: add mesa_glthread=false for gfxbench

this seems to hurt perf

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19081>

glsl/types: fix dword slots calc for float16 matricies.

The current uniform query uploader for mat3 calcs things as
if the vector elements are f16vec4 wide, so fix the calcs
here to do the same.

Fixes GTF-GL46.gtf21.GL.mat3.mat3arraysimple_frag on llvmpipe
when 16-bit uniform lowering is allowed.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14817>

agx: Use agx_nir_opt_preamble

Now that everything is in place, we can actually take advantage of
preambles. This wins us a crude form of UBO pushing (accounting for most
of the win here), as well as its intended purpose of optimizing
uniform-on-uniform arithmetic.

shader-db results are excellent. The shader that's regressed for instruction
count is a fragment shader that solely consists of `gl_FragColor = uniform`,
which goes from a vectorized UBO load to four scalar moves. That's more
instructions (and more bytes) but presumably faster, since ALU should be much
cheaper than load/store.

total instructions in shared programs: 6502 -> 5764 (-11.35%)
instructions in affected programs: 5136 -> 4398 (-14.37%)
helped: 60
HURT: 1
helped stats (abs) min: 2.0 max: 47.0 x̄: 12.33 x̃: 8
helped stats (rel) min: 0.84% max: 34.48% x̄: 18.69% x̃: 21.05%
HURT stats (abs)   min: 2.0 max: 2.0 x̄: 2.00 x̃: 2
HURT stats (rel)   min: 33.33% max: 33.33% x̄: 33.33% x̃: 33.33%
95% mean confidence interval for instructions value: -14.69 -9.51
95% mean confidence interval for instructions %-change: -20.49% -15.20%
Instructions are helped.

total bytes in shared programs: 42186 -> 38310 (-9.19%)
bytes in affected programs: 33182 -> 29306 (-11.68%)
helped: 60
HURT: 1
helped stats (abs) min: 10.0 max: 272.0 x̄: 64.83 x̃: 50
helped stats (rel) min: 0.72% max: 30.00% x̄: 15.16% x̃: 16.67%
HURT stats (abs)   min: 14.0 max: 14.0 x̄: 14.00 x̃: 14
HURT stats (rel)   min: 31.82% max: 31.82% x̄: 31.82% x̃: 31.82%
95% mean confidence interval for bytes value: -77.73 -49.35
95% mean confidence interval for bytes %-change: -16.66% -12.11%
Bytes are helped.

total halfregs in shared programs: 2370 -> 1639 (-30.84%)
halfregs in affected programs: 1804 -> 1073 (-40.52%)
helped: 60
HURT: 0
helped stats (abs) min: 1.0 max: 40.0 x̄: 12.18 x̃: 8
helped stats (rel) min: 3.85% max: 72.73% x̄: 41.37% x̃: 36.17%
95% mean confidence interval for halfregs value: -14.77 -9.60
95% mean confidence interval for halfregs %-change: -46.00% -36.75%
Halfregs are helped.

Total CPU time (seconds): 2.71 -> 2.80 (3.32%)

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

agx: Add agx_nir_opt_preamble pass

This pass creates preamble shaders. The heavylifting is done by
nir_opt_preamble. We do need to define the cost model for nir_opt_preamble, set
up 16-bit units for the register file, and scalarize the resulting
load/store_preamble intrinsics.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

asahi: Set GPR count accurately

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

asahi: Set uniform counts accurately

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

agx: Report GPRs to the driver

This needs to be passed onto the hardware.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

agx: Avoid reading high uniforms from device_load

This does not seem to be possible architecturally. Exhaustively checked
all bits of the encoding. Avoids regressing

dEQP-GLES3.functional.texture.units.8_units.only_2d.0

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

agx: Handle 64-bit moves

lower_resinfo generates some 64-bit math, so we need to handle it. Even
though we don't have native 64-bit moves, it's convenient to pretend we
do to avoid special cases in the IR. In particular, modelling 64-bit
mov_imm in the IR means our existing small constant propagation code
works, with zero-extension from 8->64.

Fixes dEQP-GLES3.functional.texture.units.2_units.only_2d_array.*

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

agx: Restrict copyprop of uniforms

Some instructions don't accept uniform registers as sources (yet?),
avoid this hazard in the optimizer.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

agx: Handle uniforms passed to COLLECT

It's useful to be able to copyprop uniform registers into COLLECT. That
requires handling of uniform registers in the parallel copy lowering,
which isn't too hard to add.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

agx: Implement scalar load/store_preamble

These need to copy values between GPRs and uniform registers. This is pretty
easy in either direction. This implements scalar versions of the intrinsics. A
backend NIR pass will scalarize for us.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

agx: Strengthen assert for packing ld/st instructions

We really need to autogenerate the packing code... It's on the todo
list, currently in discussions on how to best go about this.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

agx: Allow larger indices for ld/st

For memory load/store instructions, the immediate is 16-bit, not 8-bit
like for ALUs.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

agx: Add uniform_store instruction

Moves data from GPRs to uniform registers. Encoded like load/store
instructions, so reuse the device_load packing code.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

agx: Align preamble shaders

I don't know if it's ok to have unaligned programs, let's not risk it.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

agx: Handle multiple functions

Needed for preambles. Make agx_context map to nir_function_impl and
the (out, key, binary) triple the only global shader state.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

agx: Make push_count public

The driver needs to plumb this information into the hardware. It will
also make it easier to preserve across preamble/main function in a
moment.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

agx: Check that we don't push too much

Currently we don't have a good mechanism for this other than being
careful... assert() at least. The long term plan is to lift the
UBO/VBO lowerings into NIR, making the sysval requirements visible in
the NIR. (And possibly moving to a descriptor set model to better match
Vulkan? I need to talk to jekstrand about what we want to do here long
term.)

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

agx: Be less sloppy about high uniforms

We need 9-bits to index into the uniform file. Fix an overflow and add
some asserts to try to catch these issues earlier.

Sigh, C.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

asahi: Support preamble shaders

Extend the compiler/driver ABI to attach preambles, and plumb them into
the USC hardware when needed. This is the easy part!

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

asahi: Add group tests

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

asahi: Don't use unnecessary test fixture

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

asahi: Rename LOD clamps tests to fit other packing

We'll use for testing the "groups" encoding.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

asahi: Use USC_UNIFORM_HIGH

Abstract it away so we get access to the full uniform file without
special cases.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

asahi: Make agx_varyings a union

More accurate.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

asahi: Remove "padding" field

Trivial.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

asahi: Identify more shader-related fields

The big discovery is the "number of uniform registers" field. I learned
about this one accidentally when my preamble shaders weren't working
right, because we had inadvertently hardcoded "at most 32 registers" :-)

In the course of identifying that field, I found that the pipeline
address is used as a tagged pointer, with some unknown field in the
bottom bits and alignment demanded. The XML is updated to account for
this.

I later found that there's also a "number of general purpose registers
used by the preamble shader" field. I missed this one first, because the
encoding is slightly different from the usual "number of general purpose
registers in the main shader" field. The specification is slightly
coarser. I don't know why the hardware needs that
information anyway -- occupancy of the preamble shader should be
irrelevant -- but it's not a big deal.

Finally I found that the "more than 4 textures?" bit is... not that. I
do not yet know what it is, but it is... not that.

These all use the new groups() modifier for GenXML

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

asahi: Identify "Uniform high" USC word

The start field in the Uniform USC word is only 8-bits, whereas 9-bits
are required to address the entire uniform register file. This other
word gets used for the high half, with start indexed from u128l in
the natural way.

Apparently spending the evening stuffing too many uniforms into Metal is
paying off.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

asahi: Route shader-db stats to debug callback

This way multithreading works correctly in shader-db including CPU time account.
Code from v3d via panfrost.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

asahi: Precompile for shader-db

This gets shader-db's runner working, in conjunction with a shader-db ./run
modified to set ASAHI_MESA_DEBUG=precompile. This flag triggers precompiles of
all shaders witha default key so we can exercise the compiler.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

asahi: Remove AGX_FAKE_DEVICE environment variable

The proper way to fake a device on Linux will be drm-shim.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>

agx: Remove command line compiler

It has not been used in quite some time but adds maintainence burden.
Its function is replaced by drm-shim in conjunction with shader-db's
./run, which goes through the actual driver.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18813>