intel/fs: Disable sample mask predication for scratch stores

Scratch stores are being lowered to the instructions with side-effects,
however they should be enabled in fs helper invocations, since they
are produced from operations which don't imply side-effects.

To fix this - we move the decision of whether the sample mask predication
is enable to the point where logical brw instructions are created.

GLSL example of the issue:

 int tmp[1024];
 ...
 do {
   // changes to tmp
 } while (some_condition(tmp))

If `tmp` is lowered to scrach memory, `some_condition` would be
undefined if scratch write is predicated on sample mask, making
possible for the while loop to become infinite and hang the GPU.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3256
Fixes: 53bfcdeecf4c9632e09ee641d2ca02dd9ec25e34
Signed-off-by: Danylo Piliaiev <danylo.piliaiev@globallogic.com>
Reviewed-by: Matt Turner <mattst88@gmail.com>
Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6056>

zink: only set stencil-ref for back if two-sided

Otherwise, we want to set both front and back to the same state.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6853>

zink: store base-object of DSA-state

This is useful in the next commit, where we need to inspect the
base-state.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6853>

zink: correct typo in stencil-setup

Without this, we end up using two-sided stencil when one-sided stencil
should be used. Whoops.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6853>

radeonsi: remove redundant info.uses_fbfetch

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6782>

radeonsi: remove redundant variables from struct si_compute

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6782>

radeonsi: assume that constant load_local_group_size has been optimized out

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6782>

radeonsi: run NIR optimizations that glsl_to_nir runs but other places might not

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6782>

radeonsi: get input/output usage flags from shader_info directly

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6782>

radeonsi: get information about FS color outputs from shader_info directly

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6782>

radeonsi: use info.system_values_read

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6782>

radeonsi: call nir_shader_gather_info after lowering and optimizing NIR

to get more accurate info

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6782>

radeonsi: fix indirect dispatches with variable block sizes

The block size input was uninitialized.

Fixes: 77c81164bc1c "radeonsi: support ARB_compute_variable_group_size"

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6782>

etnaviv: simplify linear stride implementation

As documented in the galcore kernel driver "only LOD0 is valid
for this register". This makes sense, as NTE's LINEAR_STRIDE is
only capable to store one linear stride value per sampler.
This fixes linear textures in sampler slot != 0.

Fixes: 34458c1cf6c ("etnaviv: add linear sampling support")
CC: <mesa-stable@lists.freedesktop.org>
Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Michael Tretter <m.tretter@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3285>

ci: Extend meson-clang coverage by compiling all gallium drivers

Some compile-time regressions go unnoticed because not all gallium
drivers are compiled as part of the meson-clang job. Let's pass an
explicit list of drivers to compile instead of setting GALLIUM_DRIVERS
to "auto" to increase CI coverage.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6811>

panfrost: gen_pack: Fix gnu-empty-initializer errors

Fixes: 1b27817f1755 ("panfrost: gen_pack: Allow empty structs")
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Tested-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6811>

nir: gather information about fbfetch and dual source color

Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6758>

nir: fix input/output info gathering for lowered IO

Ooops.

Fixes: 17af07024df - nir: gather all IO info from IO intrinsics

Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6758>

nir: gather fs.uses_sample_qualifier from lowered IO

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6758>

nir: set system_values_read for all intrinsics

Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6758>

nir: gather tess.tcs_cross_invocation info from lowered IO intrinsics

Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6758>

nir: gather indirect info from lowered IO intrinsics

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6758>

spirv: vtn_fail with a nice message on unsupported rounding modes

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6845>

gallium/dri2: Move image->texture assignment after image NULL check.

Fix defect reported by Coverity Scan.

Dereference before null check (REVERSE_INULL)
check_after_deref: Null-checking image suggests that it may be
null, but it has already been dereferenced on all paths leading to
the check.

Fixes: ad609bf55a87 ("frontend/dri: Implement mapping individual planes.")
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6807>

r600/sfn: Initialize GPRValue member m_pin_to_channel.

Fix defects reported by Coverity Scan.

Uninitialized scalar field (UNINIT_CTOR)
uninit_member: Non-static class member m_pin_to_channel is not
initialized in this constructor nor in any functions that it calls.

Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6830>

gallium/drm: Make the pipe loader handle the driconf merging.

We can pretty easily handle merging the driver's driconf with the common
driverconf right there, rather than pushing that to each driver.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6751>

gallium/drm: Define the DRM entrypoints in drm_helper.h

This cuts even more code duplication.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6751>

gallium/drm: Refactor the stub screen create functions.

This gives me one place to document why it works this way.  This does make
the debug message a little less helpful ("etna" instead of "etnaviv" and
"vmwgfx" instead of "svga", but you should only be able to reach this when
doing something like trying the radeon/nouveau vdpau target on the wrong
DRM device for example.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6751>

gallium/drm: Deduplicate screen creation for the dynamic (clover) pipe loader.

We can just reuse drm_helper.h, which has either the real code or the stub
for all pipe_screens based on the GALLIUM_* driver defines, and the
dynamic pipe loader's .c build will only define one GALLIUM_* driver
define.  The remaining stubs should get GCed by the linker.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6751>

radeonsi: Only call nir_lower_var_copies at the end of the opt loop

In 283ad85944b5d, radeonsi started using nir_find_var_copies.  However,
it was also calling nir_lower_var_copies in the optimization loop and
the two can end up fighting.  The simple solution is to wait to lower
copies until the end of the optimization loop.

Fixes: 283ad85944b5d
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3550

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6841>

zink: reorder create_stream_output_target to fix failure case leak

the previous version of this leaked a reference to the streamout buffer here

thanks to deltragon on my blog for pointing this out!

Fixes: 37778fcd9a35 ("zink: implement transform feedback support to finish off opengl 3.0")

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6457>

zink: apply viewport count when creating pipelines

without VK_EXT_extended_dynamic_state, we need to pass this directly or
else only the first viewport will work

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6349>

zink: correctly set up fb-sized scissors for each viewport

each viewport must have a corresponding scissor, so we need to generate
an array of these to ensure our states match up

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6349>

zink: set multiviewport cap in ntv when gl_ViewportIndex is a written output

this cap varies based on the shader stage, but it must be enabled any time this
output is written

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6349>

zink: move viewport count to zink_gfx_pipeline_state

this is part of the pipeline state (there's an extension for setting it
dynamically but we don't supprot that atm

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6349>

zink: implement ARB_instanced_arrays

this is just a simple case of connecting up the vertex state to the pipeline
state

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6270>

zink: verify that src and dst aspects are the same in resource_copy_region hook

this is required by spec

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6263>

zink: clamp min created fb size to 1x1

this is required by spec

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6263>

zink: use correct layer count when creating framebuffer

pipe_framebuffer_state::layers doesn't necessarily reflect the value we
need to use here, so we can use the util function to verify

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6263>

zink: use correct number of samples on framebuffer in set_framebuffer_state

state->samples doesn't necessarily reflect the correct value here, so we need
to use the util function

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6263>

zink: use correct value for color buffer sample count when creating renderpass

nr_samples is valid on the texture pipe_resource, not the surface

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6263>

Revert "nir: replace lower_ffma and fuse_ffma with has_ffma"

This reverts commit 939ddf3f67de2ed1700c093e60cf95c1b72ff20b.

Intel has a separate pass for fusing FFMAs selectively.  We split
these flags in commit 1b72c31e1f1947123d8c236b56e230f030f60cf9 and
the reasoning still stands.  The patch being reverted was just a
cleanup, so there should be no issue with reverting it.

Acked-by: Matt Turner <mattst88@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6849>

radv: use syncobj for wsi fence

Note: this also fixes vk_object_base_init() not being called for the wsi
fences, now that it goes through radv_CreateFence().

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6846>

radv: fix incorrect ResetFences path for WSI fence

Looks like this path is meant for RADV_FENCE_WINSYS type fences.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6846>

amd/tmz: move uses_secure_bos to radeon_winsys

This allows to inline radeon_uses_secure_bos calls and reduce CPU overhead.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6049>

radeonsi/tmz: add workaround for mpv/vaapi subtitles

Subtitles are rendering with an upload through a staging texture.
So the sequence is:
  1. draw video (with a secure cs)
  2. copy staging texture to the real texture (via si_resource_copy_region) in
    a non-secure cs.
  3. draw video (with a secure cs)

Step 2 and 3 both generates a flush with RADEON_FLUSH_TOGGLE_SECURE_SUBMISSION.
These flushes are executed quite late: right before doing the draw/dispatch,
so maybe the issue here is the handling of dependencies.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6049>

radeonsi/tmz: add a tmz variant for sctx::eop_bug_scratch

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6049>

radeonsi: disable primitive discard if tmz is in use

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6049>

radeonsi/tmz: add tmz variant for sctx::tess_rings

tess_rings must be encrypted when used in a secure job so this commit
introduces a tess_rings_tmz resource.

The cs_preamble_state doesn't contain the tess_rings address anymore since
it can change. The tess_rings related registers go in a separate preamble.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6049>

radeonsi/tmz: add tmz variant of sctx::wait_mem_scratch

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6049>

radeonsi/tmz: fail si_texture_transfer_map if tex is encrypted

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6049>

amd/winsys: add RADEON_FLUSH_TOGGLE_SECURE_SUBMISSION

Instead of exposing a cs_set_secure() callback that always needs a call
to si_flush_gfx_cs before a switch, this commit introduces a new
flag to switch between secure and non-secure on submissions.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6049>

radeonsi/tmz: allow secure job if the app made a tmz allocation

This commit makes TMZ always allowed instead of being either off or forced-on
with AMD_DEBUG=tmz.

With this change:
- secure job can be used as soon as the application made a tmz allocation. Driver
  internal allocations are not enough to enable secure jobs (if tmz is supported
  and enabled by the kernel)
- AMD_DEBUG=tmz forces all scanout/depth/stencil buffers to be allocated as TMZ.
  This is useful to test app thats don't explicitely support protected content.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6049>

ac/gpu_info: add detection of TMZ support

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6049>

amd: add AMDGPU_IDS_FLAGS_TMZ definition to amdgpu_drm.h

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6049>

radeonsi: introduce SI_RESOURCE_FLAG_INTERNAL / RADEON_FLAG_DRIVER_INTERNAL

Tag allocations as driver internal.
Some of these allocations will need to be doubled to handle TMZ (one secure bo,
one normal bo) but these allocations shouldn't switch the winsys in "the app
is using TMZ".

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6049>

radeonsi/tmz: allocate depth/stencil buffers as encrypted

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6049>

radeonsi/tmz: add safety assert when tmz is enabled

This commit adds asserts verifying the following conditions when
using a secure job:
- fb textures are encrypted (both colors and depth/stencil buffers)
- all writeable bo are encrypted

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6049>

radeonsi/tmz: use secure job if using an encrypted z/s buffer

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6049>

radeonsi/tmz: use secure job if framebuffer has dcc

Fixes a hang with DCC.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6049>

turnip: rework fences to use syncobjs

Fences are now just a syncobj, which makes our life easier.

The next step will be to fill out ImportFenceFdKHR()/GetFenceFdKHR().

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6683>

turnip: require syncobj support

Note: this means turnip requires kernel 5.8 (or older with syncobj patch).

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6683>

turnip: add a fd field to tu_device

Avoid the extra indirect for this commonly used field.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6683>

turnip: delete unused tu_fence_signal function

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6683>

mesa: handle GL_FRONT after translating to it

Without this, we end up throwing errors on code along these lines when
rendering using single-buffering:

GLint att;
glGetIntegerv(GL_READ_BUFFER, &att);
glGetFramebufferAttachmentParameteriv(GL_READ_FRAMEBUFFER, att, ...);

This is because we internally translate GL_BACK (which is what
glGetIntegerv returned) to GL_FRONT, which we don't handle in the
Desktop GL case. So let's start handling it.

This fixes the GLTF-GL33.gtf21.GL2FixedTests.buffer_color.blend_color
test for me.

Fixes: e6ca6e587e7 ("mesa: Handle pbuffers in desktop GL framebuffer attachment queries")

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6815>

ci: adjust RadeonSI rules

src/amd/vulkan is specific to RADV and shouldn't trigger RadeonSI CI.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6839>

vulkan/wsi/display: add option for display fence to signal syncobj

To avoid having a separate "wsi_fence" path in the driver, make it so wsi
fences can signal a syncobj.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Acked-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6707>

radeonsi: always inline draw-related functions that have only one use

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6786>

radeonsi: lift the conditional for skipping si_upload_vertex_buffer_descriptors

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6786>

radeonsi: add unlikely statements into si_draw_vbo

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6786>

radeonsi: move si_upload_vertex_buffer_descriptors into si_state_draw.c

It will be inlined there.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6786>

radeonsi: reorganize the code around the gfx9 scissor bug

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6786>

radeonsi: don't call emit_cache_flush after uploading bindless descriptors

The next draw call will do it.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6786>

radeonsi: move a displaced comment in si_draw_vbo

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6786>

radeonsi: call si_upload_graphics_shader_descriptors before the big conditional

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6786>

radeonsi: use optimal order of operations when setting up a compute dispatch

The order is:
- set registers
- flush caches
- set render condition
- prefetch the shader
- set registers that may be read from memory (indirect draw)
- dispatch

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6786>

panfrost: Remove redundant casts of viewport position

The position is already converted to int when clamping to framebuffer
size.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6840>

panfrost: Handle non-positive viewport positions

It's possible for viewport positions to be negative, so add a lower
bound of zero and avoid wraparound when maximum values are zero.

Fixes the menu blur effect in the OpenGL 3.3 render of GZDoom.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6840>

panfrost: Clamp uniform buffer size

Issue (57) for the ARB_uniform_buffer_object spec states:
    "The uniform buffer could be larger than the amount of uniform
    block(s) data inside it."

This means we need to clamp the uniform buffer size in case it is
bigger than what hardware supports.

Fixes the OpenGL 3.3 renderer of GZDoom.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6835>

nir: replace lower_ffma and fuse_ffma with has_ffma

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6756>

radeonsi: fuse or lower ffma optimally on all chips

LLVM is going to support the legacy instructions soon.

This change switches FMA to MAD for gfx10.

54793 shaders in 33659 tests
Totals:
SGPRS: 2632554 -> 2629570 (-0.11 %)
VGPRS: 1536364 -> 1535312 (-0.07 %)
Spilled SGPRs: 3602 -> 3562 (-1.11 %)
Spilled VGPRs: 44 -> 40 (-9.09 %)
Private memory VGPRs: 256 -> 256 (0.00 %)
Scratch size: 312 -> 308 (-1.28 %) dwords per thread
Code Size: 55422660 -> 55345408 (-0.14 %) bytes
Max Waves: 963983 -> 964200 (0.02 %)

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6756>

nir: split lower_ffma into lower_ffma16/32/64

AMD wants different behavior for each bit size

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6756>

nir: split fuse_ffma into fuse_ffma16/32/64

AMD wants different behavior for each bit size

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6756>

radeonsi: force linear for textures with height=1 (gfx6-8)

addrlib will force it only for level = 0, so force it when we create
a height=1/multi-level texture.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2250
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6775>

radeonsi: add a tweak for PS wave CU utilization for gfx10.3

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6822>

radeonsi: break a binning batch on a new PS if bins can use multiple state sets

ported from PAL

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6822>

radeonsi: move binning parameters into si_screen

it will be used in the next commit

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6822>

radeonsi: set KEEP_TOGETHER_ENABLE if needed

ported from PAL

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6822>

pan/bi: Remove old register mode definitions

Now replaced by canonical equivalents.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6793>

pan/bi: Use canonical register packing

This better matches the hardware behaviour and will allow us to
implement write masks down the line.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6793>

pan/bi: Rename port -> slot

To match the canonical naming convention.

$ sed -i -e 's/\([_ \.>"!]\)[pP]ort\([ ,'"'"'_0123s\(\[]\)/\1slot\2/g' *.c *.h

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6793>

pan/bi: Decode all 32-bit register modes

There's actually more than 16 of them, disambiguated by `r2 == r3` and
`first?` as conditions for another "fun" encoding. The extra space
allows for writing half-registers.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6793>

pan/bi: Pass 'first' through disassembler

Required to decode the registers of the first instruction of a clause
correctly.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6793>

pan/bi: Add bifrost_reg_mode enum

This generalizes the previous control field, adding in the full set of
enums required for 16-bit writemasks.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6793>

pan/bi: Disassemble PC-relative addresses

Apply whatever modifier we computed and symbolically work out the
behaviour of the hardware. This involves some pretty gnarly primitives
(e.g. 28-bit sign extensions) but seems to works ok.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6793>

pan/bi: Decode M values in disasm

This is rather complicated and mostly unused in real world code but
correct handling is required to disassemble branchy code.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6793>

pan/bi: Track M values of disassembled constants

We'll want to route these values from the clause itself to the source
dump in order to disassemble modified embedded constants.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6793>

pan/bi: Inline dump_instr

Tuple dumping is trivial now that we autogenerate most of it.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6793>

pan/bi: Annotate disassemble with format names

It's hard enough to keep this all straight as it is.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6793>

pan/bi: Annotate stop bit (canonically "Z-bit")

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6793>