aco: Fix image instructions with lod when 2d_view_of_3d is enabled on GFX9.

If there's a lod parameter it matter if the image is 3d or 2d because
the hw reads either the fourth or third component as lod. So detect
3d images and place the lod at the third component otherwise.

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18114>

nir/opt_algebraic: Optimize d3d9 pow with fmulz.

Foz-DB Navi21:
Totals from 69 (0.05% of 134913) affected shaders:
CodeSize: 255684 -> 253788 (-0.74%); split: -0.74%, +0.00%
Instrs: 46307 -> 46052 (-0.55%); split: -0.55%, +0.00%
Latency: 533255 -> 530742 (-0.47%); split: -0.48%, +0.01%
InvThroughput: 110001 -> 109156 (-0.77%)
VClause: 839 -> 844 (+0.60%); split: -1.19%, +1.79%
SClause: 1411 -> 1395 (-1.13%)
Copies: 1828 -> 1816 (-0.66%); split: -1.09%, +0.44%
PreSGPRs: 2243 -> 2232 (-0.49%)
PreVGPRs: 2213 -> 2192 (-0.95%)

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18145>

nir/opt_algebraic: Optimize check for single bit.

Foz-DB Navi21:
Totals from 3239 (2.40% of 134913) affected shaders:
SpillSGPRs: 110 -> 102 (-7.27%)
CodeSize: 17426512 -> 17344808 (-0.47%); split: -0.48%, +0.01%
Instrs: 3194264 -> 3179366 (-0.47%)
Latency: 20498012 -> 20481419 (-0.08%); split: -0.08%, +0.00%
InvThroughput: 3311738 -> 3311282 (-0.01%); split: -0.02%, +0.00%
SClause: 145810 -> 145690 (-0.08%)
Copies: 171748 -> 169009 (-1.59%); split: -1.63%, +0.03%
Branches: 86610 -> 86370 (-0.28%)
PreSGPRs: 138036 -> 137104 (-0.68%)
PreVGPRs: 138540 -> 138545 (+0.00%)

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17429>

util: Remove the need of _MTX_INITIALIZER_NP in simple_mtx.h

The usage of _MTX_INITIALIZER_NP is a problem. Sure, the code builds on
Linux and Windows, and it even usually works, but is problematic.
It means that Windows lock debugging tools cannot be used with that code.
So we remove _MTX_INITIALIZER_NP to gain the following benefit:
  Aligning to the C11 standard threads.h definitions.
  Improving portability of the threading code to better support Windows.
This change removes one of the most prolific uses of _MTX_INITIALIZER_NP by
slightly increasing the cost of simple_mtx_lock when targeting platforms that
don't support futex, but it's cost are far less than the cost of syscall,
so the affect are negligible.

The futex code path are not affected. And Windows 8 and upper also
supports the futex codepath, so it's only affect the Windows version < 8, MacOS
or other platforms that doesn't support for futex.

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17122>

util: Add extern "C" guard in simple_mtx.h

This is a header that can be accessed both in C/C++,
And when remove the usage of _MTX_INITIALIZER_NP in latter commits, it's need
implement simple_mtx_init in .c file, so for ABI consistence
Add the extern "C" for cpp files

Also add comment for #endif guard for code readability

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17122>

util: Add api util_call_once_with_context

This is used to remove the need of _MTX_INITIALIZER_NP in simple_mtx.h

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17122>

pvr: Only destroy the physcial device if it exists

Without this, it segfaults on instance destruction which brings down the
Vulkan app if you have the driver installed on your system, even if you
have no PowerVR hardware.

Reviewed-by: Rajnesh Kanwal <rajnesh.kanwal@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18330>

pan/mdg: Print 3 sources for CSEL

The third source exists logically but not architecturally. We still need to
print it. Caught by the assertion.

Fixes: 0ee24c46e0a ("pan/mdg: Only print 2 sources for ALU")
Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18338>

pan/mdg: Always write return address to r1.w

This might not be optimal but it matches our current behaviour and is much more
justified than the "accidental" code before. Caught by the gcc warning:

../src/panfrost/midgard/midgard_schedule.c:1227:48: warning: the comparison will
always evaluate as ‘true’ for the address of ‘writeout_branch’ will never be
NULL [-Waddress]

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18338>

radeonsi/sqtt: set stable pstate if possible

This avoids the need to manually change the power profile.

Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18009>

wayland: avoid writing interface names as free-form strings

Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Simon Ser <contact@emersion.fr>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18335>

pvr: Change indentation from tab to spaces for xml files.

This is done to match other drivers and to use spaces for
indentation throughout the whole of the powervr driver.

Signed-off-by: Karmjit Mahil <Karmjit.Mahil@imgtec.com>
Reviewed-by: Frank Binns <frank.binns@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18319>

pvr: Implement vkResetCommandBuffer

Signed-off-by: Frank Binns <frank.binns@imgtec.com>
Reviewed-by: Rajnesh Kanwal <rajnesh.kanwal@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18317>

pvr: reset all command buffer state in pvr_cmd_buffer_reset()

Command buffers were only being partially reset. Fix this by factoring out the
common parts of pvr_cmd_buffer_reset() and pvr_cmd_buffer_reset() into a common
function, so there's now only a single place to free command buffer state, and
zeroing out parts of the command buffer that were missed.

Signed-off-by: Frank Binns <frank.binns@imgtec.com>
Reviewed-by: Rajnesh Kanwal <rajnesh.kanwal@imgtec.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18317>

virgl/ci: remove some flakes

With the better synchronization we can now remove some flakes.

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Reviewed-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Ryan Neph <ryanneph@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18243>

virgl: when reading back wait first, then do the transfer

Otherwise the transfer from the host might not be finished
when we try to read the data.

v2: always wait before readback, but also keep the wait after
  a possible readback, Itherwise TF fails in the CI.

Fixes: e87186fc679c146c5d521139bb79c8eafa0a22f0
    virgl: add virgl_resource_transfer_prepare

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Reviewed-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Ryan Neph <ryanneph@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18243>

v3dv: expose VK_KHR_synchronization2

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18290>

v3dv: use VkPipelineStageFlagBits2 when checking semaphore wait stage

This adds extra stages, some of which involve geometry stages that are
relevant to the check we do to check for binning sync.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18290>

v3dv: port remaining entry points to KHR_synchronization2

The common code in Mesa will rewrite the old entry points to these
if available.

Since we implement events and timestamps queries in the driver the
API changes don't quite affect us.

vkQueueSubmit2 is already implemented in the common synchronization
framework and the driver works in terms of a submit hook that is
independent of the actual entry point used by the application.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18290>

v3dv: implement vkCmdPipelineBarrier2

Added with KHR_synchronization2. The common code in Mesa will
rewrite vkCmdPipelineBarrier to vkCmdPipelineBarrier2.

With synchronization2 barriers now have a per-barrier stage
and access flags (previously these were shared by all the barriers
in a vkCmdPipelineBarrier commands), so we need to rewrite a bit
the logic to account for this.

Also, stage and access flag bits have been expanded and renamed.
Particularly, some new flags have been added that we need to account
for.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18290>

v3dv: also check GS stage on image access for binning barriers

For some reason we were only checking the binning VS stage, but we
should also check the GS, like we do for other access types.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18290>

zink: request forcing of fp32-samplers to nearest when needed

Setting a linear filtering mode when not supported isn't legal, and can
lead to bad things in the VK driver. Let's ask the state-tracker to fix
this for us!

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18292>

mesa/st: force nearest-filtering for fp32 textures

OpenGL 3.0 requires ARB_texture_float, which technically speaking
requires linear-filtering support for FP32 textures. However, because a
lot of early implementations just ignore this and does nearest filtering
instead due to lack of hardware features for this, this functionality
was never added to the OpenGL CTS.

The result is that FP32 is a feature that is required on paper, but
is unreliable to be used by applications in practice. This is mostly
fine; for most filterable use-cases (e.g colors), FP16 is fine and saves
a bunch of bandwidth, and for cases where you really need the extra bits
(depth-reads etc), filtering is usually not what's wanted.

To save drivers that doesn't support filtering of FP32 filtering from
having to manually whack the state, and get needless CSOs in the CSO
caches, let's force this in early, where we already do the same for
integer textures.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18292>

mesa/st: always force integer filtering to nearest

There's no need to re-check the DRI-config setting here; we either
already got nearest filtering already, or the we applied a work-around
to avoid incomplete textures. In either case, these should be nearest,
no need to check for it again.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18292>

mesa/st: also override mip filter for integer textures

If not, drivers may attempt interpolating these values, which isn't
going to do anything good.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18292>

radv: determine if the topology uses adjacency from the shaders on GFX9+

From the Vulkan spec says:
    "VUID-VkGraphicsPipelineCreateInfo-pStages-00738
     If the pipeline is being created with pre-rasterization shader
     state and pStages includes a geometry shader stage, and does not
     include any tessellation shader stages, its shader code must
     contain an OpExecutionMode instruction specifying an input
     primitive type that is compatible with the primitive topology
     specified in pInputAssembly"

In other words, the GS input primitive and the topology must match, so
it's possible to get this information directly from the GS instead of
from the pipeline key. Main benefit is that the driver can compile all
pre-rasterization stages without knowing the primitive topology.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18282>

radv: allow null descriptor set layouts in pipeline layouts

This is allowed by the Vulkan spec when GPL is used.

Fixes dEQP-VK.pipeline.pipeline_library.graphics_library.misc.other.null_descriptor_set_in_monolithic_pipeline

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18313>

v3dv: implement VK_KHR_zero_initialize_workgroup_memory

This only requires that we call the relevant lowering pass in NIR.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18312>

nir/lower_variable_initializers: implement non-scoped barrier path

Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18312>

radv: Remove redundant radv_QueuePresentKHR.

Does the same thing as the wsi function.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18293>

venus: allow no external memory handle when renderers lacks support

This is to make vtest happy.

Fixes: 9c5a7ffbd8a ("venus: fix external memory ext filtering")

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18331>

nir/lower_mediump: Lower FS outputs to 16-bit when the value was upconverted.

Take this real-world (trimmed) shader:

precision highp float;
in lowp vec4 var_varVertexColor;
layout(location = 0) out vec4 out_FragColor0;
void main() {
    vec4 textureColor0 = vec4(1.000000e+00, 0.000000e+00, 0.000000e+00, 1.000000e+00);
    vec3 color = vec3(1.000000e+00, 1.000000e+00, 1.000000e+00);
    vec4 outColor = vec4(vec3((color).rgb), 1.000000e+00);
    (outColor *= vec4(var_varVertexColor));
    (out_FragColor0 = outColor);
}

After opts, it's just a store from input to output.  If we decide to lower
the input to 16-bit, then as long as the driver can handle 16-bit outputs,
it would be a good idea to demote the output and save the conversions.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18003>

d3d12: Change displayable format logic

Instead of not reporting support for the pixel format at all, just
disable swapchain creation.

Some apps want to create off-screen contexts targeting these formats,
but since WGL doesn't really have "off-screen," the pixel format
enumeration should return support for these.

Reviewed-by: Giancarlo Devich <gdevich@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18326>

anv: Drop anv_pipeline::use_primitive_replication

Instead of this fragile use_primitive_replication bit which we set
differently depending on whether or not we pulled the shader out of the
cache, compute and use the information up-front during the compile and
then always fetch it from the vue_map after that.  This way, regardless
of whether the shader comes from the cache or not, we have the same flow
and there are no inconsistencies.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17602>

intel/compiler: Store the number of position slots in the VUE map

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17602>

nir: Track per-view outputs in shader_info

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17602>

anv: Stop looking at the pipeline in multiview lowering

Passing all the data we need in directly avoids issues where we might
forget what is and isn't set on the pipeline object at the time the
shader call happens.  This will be especially important once we start
splitting things up for pipeline libraries.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17602>

anv: Decouple primitive replication check from the fragment shader

Now that we're using nir_intrinsic_load_layer_id for ViewIndex, we can
use primitive replication even when the FS uses ViewIndex.  Also, only
check that active_stages doesn't contain anything other than VS and FS,
not that it's exactly VS | FS.  This both decouples things more and
allows primitive rplication in the depth-only case where we have no
fragment shader at all.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17602>

anv: Use nir_intrinsic_load_layer_id for ViewId in fragment shaders

Now that fragment shaders are a special case handled up-front, we can
reduce the complexity of some of the logic elsewhere in the pass.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17602>

crocus: Fix memory leaks on iris_resource_create failure paths

We've already allocated the pipe resource, so we ought to free it
before returning.

Fixes: d8a38edc48e ("crocus: fail resource allocation properly.")
Fixes: f3630548f1d ("crocus: initial gallium driver for Intel gfx 4-7")
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18311>

crocus: Fall back if iris_map_copy_region can't create a staging resource

If we can't create a staging resource, then fall back to direct CPU
mapping (possibly with a stall).  This is a rare case, but it could
happen for very large staging buffers.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18311>

crocus: Don't print out XXX messages if resource creation fails

Failing to create staging resources isn't necessarily a critical error,
and we probably don't want to be printing out XXX messages.  I mostly
had this here when bringing up iris and resource creation was failing
because I'd done something wrong when writing the code.  Then it got
copied over to crocus.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18311>

iris: don't create staging resources larger than half the aperture

This is a port of crocus's f1c1fcfd (crocus: don't create staging
resources > half aperture).

I chose to use the whole aperture here rather than 3/4 of it, mostly
to avoid adding another legacy field for aperture stuff.  It's close
enough to work.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2104
Reviewed-by: Dave Airlie <airlied@redhat.com>
Tested-by: Mark Janes markjanes@swizzler.org
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18311>

iris: Fall back if iris_map_copy_region can't create a staging resource

If we can't create a staging resource, then fall back to direct CPU
mapping (possibly with a stall).  This is a rare case, but it could
happen for very large staging buffers.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Tested-by: Mark Janes markjanes@swizzler.org
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18311>

iris: Don't print out XXX messages if resource creation fails

Failing to create staging resources isn't necessarily a critical error,
and we probably don't want to be printing out XXX messages.  I mostly
had this here when bringing up the driver and resource creation was
failing because I'd done something wrong when writing the code.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Tested-by: Mark Janes markjanes@swizzler.org
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18311>

iris: Delete unused iris_screen::aperture_bytes field

This was effectively removed in bb3545a6, but the field wasn't deleted,
leaving it there to troll people.  screen->aperture_bytes is 0?  What?

Fixes: bb3545a6ee4 ("intel: Store the aperture size in devinfo.")
Reviewed-by: Dave Airlie <airlied@redhat.com>
Tested-by: Mark Janes markjanes@swizzler.org
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18311>

radeonsi: use nir_opt_large_constants earlier

Calling it before nir_convert_to_lcssa helps in some cases,
because the NIR is simpler and nir_opt_large_constants can
detect that a variable is constant.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7059
CC: mesa-stable
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18147>

radv: deal with ac_nir_translate failures

Now that ac_nir_translate reports failures instead of aborting,
we need to deal with it in the driver.
For now, call abort to keep the existing behavior but this could
be improved.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18147>

radeonsi: deal with ac_nir_translate failures

ac_nir_translate can fail now so forward the translation result to
si_llvm_compile_shader. If it's a failure, it'll print something like:
  radeonsi: can't compile a main shader part
And the shader won't be used.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18147>

ac/llvm: add a return value to ac_nir_translate

Return a boolean instead of aborting when the NIR->LLVM translation fails.
This way, we can deal with the error at the calling site instead of crashing
the whole application.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18147>

nir: Print selection control for nir_if.

It's useful to see this information now that aco is going to use it.

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18297>

nv50: race free state tracking

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Acked-by: M Henning <drawoc@darkrefraction.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10752>

nvc0: make state handling race free

I am not entirely convinced that contexts can't mess up the state of other
contexts, but with this we at least turn down the amount of races on the
CPU side.

If we hit bugs later we can always look into it then and figure out what
to fix how.

I think we might need a better solution for it in the future as state
tracking might need to become more involved, but for now this should be
good enough.

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Acked-by: M Henning <drawoc@darkrefraction.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10752>

nouveau: make fencing race free

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Acked-by: M Henning <drawoc@darkrefraction.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10752>

nouveau: per context client and pushbuf

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Acked-by: M Henning <drawoc@darkrefraction.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10752>

nouveau/fence: per context fence

This doesn't add a fence list per pipe context, it simply makes us track
the current fence per context so we can safely modify the fence object
per thread.

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Acked-by: M Henning <drawoc@darkrefraction.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10752>

nv50: remove nv50_bufctx_fence call in vbo_kick_notify

No idea why it is there. nvc0 doesn't do anything like it and it doesn't
regress any tests. Also removing is easier than handling the fence locking
inside the kick notify handler.

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10752>

nv50: remove nouveau_fence_signalled prior nouveau_fence_wait

nouveau_fence_wait handles signalled fences just as well.

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: M Henning <drawoc@darkrefraction.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10752>

nouveau/fence: make the fence struct part of the fence API

This makes it easier to implement proper locking of the fence struct as we
can redesign some of the functions to take the list object instead of
fence objects.

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: M Henning <drawoc@darkrefraction.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10752>

nouveau: use PUSH_KICK instead of nouveau_pushbuf_kick

This makes it easier to insert locking code around libdrm.

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: M Henning <drawoc@darkrefraction.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10752>

nouveau: wrap nouveau_pushbuf_refn

This makes it easier to insert locking code around libdrm.

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: M Henning <drawoc@darkrefraction.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10752>

nouveau: wrap nouveau_pushbuf_validate

This makes it easier to insert locking code around libdrm.

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: M Henning <drawoc@darkrefraction.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10752>

nouveau: wrap all nouveau_pushbuf_space calls

This makes it easier to insert locking code around libdrm.

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: M Henning <drawoc@darkrefraction.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10752>

nouveau: wrap nouveau_bo_wait

This makes it easier to insert locking code around libdrm.

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: M Henning <drawoc@darkrefraction.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10752>

nouveau: wrap nouveau_bo_map

This makes it easier to insert locking code around libdrm.

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: M Henning <drawoc@darkrefraction.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10752>

nouveau: move nouveau_context initialization to common code

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10752>

nouveau/fence: rework nouveau_fence_emit so we can call it on emitted fences

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10752>

nouveau/buffer: simplify uses of nouveau_fence_work

nouveau_fence_work already checks the state, so we can just call it
directly.

Strictly speaking, this isn't functional equivalent, but practically it
doesn't matter when we get the callback called.

Main reason for doing this is, that this makes locking way easier as we
can simply lock within nouveau_fence_* functions and callers don't have to
take locks themselves.

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: M Henning <drawoc@darkrefraction.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10752>

nouveau/mm: make code thread safe

With this helgrind doesn't report any races in this code.

Cc: mesa-stable
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Pierre Moreau <dev@pmoreau.org>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10752>

nouveau: use the contexts pushbuf and client where possible

At the moment the context uses the screens object, but in order to fix
threading without having to lock too much we will have to allocate those
objects per context later.

Cc: mesa-stable
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10752>

mesa/st: Drop st_texture_image_transfer::map

It's only used in the scope of a single function, st_UnmapTextureImage. Use
a local variable instead.

Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18257>

mesa/st: Fix indentation in st_MapTextureImage

Dedent an if-else block and setup the else braces according to the coding
style.

Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18257>

mesa/st: Don't map compressed texture fallbacks for reads

We don't need to map the compressed texture's fallback resource for operations
which will only read from the texture (e.g., glGetCompressedTexImage2D and
glCopyImageSubData).

For compressed textures using a fallback, delay the map until
st_UnmapTextureImage and only do the map if it's being written to.

To help identify what's being changed, the expected indentation changes are
split out into the next patch.

Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18257>

mesa/st: Add and use st_texture_image_transfer::box

Use this field to determine which parts of the compressed texture fallback
resource needs updating. Drops a dependency on the
st_texture_image_transfer::transfer pointer.

Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18257>

mesa/st: Add and use st_texture_image_insert_transfer

Split out the transfer array reallocation and insertion functionality from
st_texture_image_map.

Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18257>

mesa/st: Set PIPE_MAP_DISCARD_RANGE in fallback_copy_image

Enables some drivers to avoid reading from the destination when mapping it.

Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18257>

v3dv: Use the common physical device enumeration

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17750>

panvk: Use the common physical device enumeration

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17750>

lvp: Use the common physical device enumeration

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17750>

turnip: Use the common physical device enumeration

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17750>

anv: Use the common physical device enumeration

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17750>

radv: Use the common physical device enumeration

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17750>

vulkan: Add common physical device management

Add common entrypoints for enumerating physical devices, based on the RADV implementation.

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17750>

radv: enable 64-bit vertex formats

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6956
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17894>

radv,aco: implement 64-bit vertex inputs

Note that, from 22.4.1. Vertex Input Extraction of Vulkan spec:
The input variable in the shader must be declared as a 64-bit data type if
and only if format is a 64-bit data type.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17894>

radv: enable R8G8B8, B8G8R8 and R16G16B16 vertex formats

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Gitlab: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6956
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17894>

radv: remove radv_translate_vertex_format

This is not used anymore.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17894>

radv,aco: use pipe_format for dynamic vertex input state

Also prepare for 64-bit and R8G8B8/R16G16B16 with the addition of
radv_vs_input_state::nontrivial_formats.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5021
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17894>

radv,aco: use pipe_format for static vertex input state

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17894>

radv: add radv_is_vertex_buffer_format_supported

We will want to support formats which cannot used for texel buffers.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17894>

ac: add ac_vtx_format_info

This will be used by RADV and ACO.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17894>

util/format,vulkan/format: update p_format for Vulkan vertex buffers

We need to move some earlier (so they are <= 255 like the comment says)
and add a few 64-bit ones.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17894>

nir: use a GC context for instructions

Gives an roughly -15% change in compile-time for RADV/ACO.

Memory usage increase seems to be 5-6%.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5034
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12910>

nir: adjust nir_src_copy signature to take a nir_instr *

This is almost always a nir_instr and updating the src of a nir_if will
have to work slightly differently in the future.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Acked-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12910>

Revert "nir: Drop the unused instr arg for src/dest copy functions."

This reverts commit c3a01841184ee8303c0c5ebe58491301622c5ad6.

Acked-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12910>

nir/serialize: remove unused parameter from read_src()

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12910>

util: add freelist allocator with mark/sweep

Based on the allocator https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/524,
but modified a bit (in particular, it's now separate from ralloc).

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12910>

util/ralloc: add HEADER_ALIGN macro

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12910>

nir: Free instructions more often

Soon we'll be allocating instructions out of a per-shader pool, which
means that if we don't free too many instructions during the main
optimization loop, the final nir_sweep() call will create holes which
can't be filled. By freeing instructions more aggressively, we can
allocate more instructions from the freelist which will reduce the final
memory usage.

Modified from Connor Abbott's original patch to rebase on top of
refactored DCE and so that the use-after-free in nir_algebraic_impl() is
fixed.

Co-authored-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12910>

gallium/u_threaded: buffer sharedness tracking

This fixes TC's buffer invalidation code for buffers that are shared
between contexts.

TC is unable to notify other TCs in case it replaces a shared buffer's
underlying storage when invalidating, causing those other TCs to use
wrong buffer IDs for busyness tracking, which leads to corruption due
to invalidation fast-paths being triggered when they shouldn't be.

This patch addresses this issue by tracking if a buffer is shared, and
if it is, disabling buffer storage replacement for the affected buffer.
This is achieved by tracking which TC instance first accessed a certain
buffer. If a second instance then accesses it as well, it will realize
that it isn't the only one working on the buffer and mark the buffer
accordingly.

If TC needs to invalidate a buffer for the correctness of an operation
at any point, it will fall back to doing the operation in a synchronous
fashion with this patch if the buffer is shared and currently busy.

It might be possible to later detect that a buffer has become un-shared;
however, this is outside of the scope of this bugfix patch.

v2: Do not disable buffer busyness tracking for shared buffers.

Fixes: e9c41b32 ("gallium/u_threaded: add buffer lists - tracking of buffers referenced by tc")

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17338>