ci/docs: Always expose the built docs.

When testing my docs changes outside of an MR, there was no way to look at
the result.

Acked-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19733>

intel: Fix crashes for importing drm buffer

image_aspect_to_binding() converts aspect to index by subrracting
VK_IMAGE_ASPECT_MEMORY_PLANE_0_BIT_EXT, however these enum values
are bitfields, not consecutive numbers, so comparing and subtracting
them won't work.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20269>

hasvk: pipelineStageCreationFeedbackCount is allowed to be 0

Fixes: 6601e5d6fc6 ("anv: implement VK_EXT_pipeline_creation_feedback")

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20216>

vdpau: allow building vdpau st with virgl only

Fixes: 6b5aecb1955 ("virgl: add support for hardware video acceleration")

Signed-off-by: Jan Palus <jpalus@fastmail.com>
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20108>

glsl: fix function inlining for images

Here we skip replacing parameters with their actual values for
images as glsl_to_nir() expects them to be copied to temps first.
Tree grafting has a similiar rule to avoid this happening also.

Fixes: 8d10a6835f32 ("glsl: dont create temps for builtin function inputs")

Tested-by: Martin Roukala <martin.roukala@mupuf.org>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20274>

radv: Improved export conflict bug workaround.

Based on f129db911bd ("radeonsi/gfx11: use a better workaround for the export conflict bug")
which claims better performance.

I couldn't be bothered to do the refactor to check the sample count with
dynamic sample counts, so this is just conservative there.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20273>

mesa: move the _mesa_set_varying_vp_inputs call to where the state changes

This removes the _mesa_update_vao_state() call from all glDraw* functions
where it was executed unconditionally.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19953>

mesa: move _DrawVAOEnabledAttribs determination into st_update_array

This can just be computed where it's used.

Now the non-glDraw paths like glRasterPos, glBegin/End, and the slow
display list path have to save and restore _VPModeInputFilter, which
is the only field that's different from the glDraw* path.

_VPModeInputFilter is a bitmask of VP inputs that might have to be bound.
The only difference with glBegin/End and the slow display list path is
that they also add VERT_BIT_MAT_ALL to the bitmask, whereas the glDraw* path
doesn't have that.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19953>

mesa: inline _mesa_draw_array_bits & _mesa_draw_current_bits

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19953>

mesa: don't AND with VERT_BIT_ALL because it's ~0u

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19953>

mesa: simplify VBO state flagging in _mesa_update_vao_state

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19953>

mesa: remove VAO::NewVertexBuffers/Elements, set the dirty flags directly

These intermediate dirty flags are unnecessary now.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19953>

mesa: move calling _mesa_update_vao_derived_arrays into st_update_array

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19953>

mesa: remove the NewVAO flag, set directly what we need to set

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19953>

st/mesa: move st_atom.c contents into st_context.c

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19953>

st/mesa: inline st_validate_state and remove redundant checking in callers

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19953>

st/mesa: remove the switch from st_validate_state by passing state mask directly

Instead of passing the enum that represents which states should be updated
to st_validate_state, pass the state bitmask to st_validate_state directly.
This removes the switch statement over the enums.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19953>

st/mesa: remove st_context::dirty, use gl_context::NewDirtyState instead

We stored the same state in 2 different variables, and then we OR'd them.

This changes the st_validate_state logic slightly, but should be identical.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19953>

mesa: remove per_vertex_edgeflags parameter from DrawGalliumVertexState

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19953>

mesa: completely rewrite the edge flag state update logic

This removes the edge flag logic from st_validate_state, which was always
executed and added overhead there. Now it's done only when needed.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19953>

mesa: don't call _mesa_set_varying_vp_inputs in set_vertex_processing_mode

It's redundant because it's always called in _mesa_update_draw_vao.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19953>

mesa: don't call _mesa_update_vao_derived_arrays for dynamic VAOs

This just moves the exit into the caller. There is nothing else in
_mesa_update_vao_derived_arrays that dynamic VAOs care about.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19953>

mesa: merge 2 switch statements into one in _mesa_get_derived_vao_masks

_mesa_vao_enable_to_vp_inputs has a switch statement and we call it twice.
Let's inline it.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19953>

mesa: remove derived _EffEnabled* fields and compute them at their only use

We were passing the fields to their only use and used them only once,
so let's just compute them there.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19953>

mesa: merge _mesa_draw_(user_array,nonzero_divisor)_bits into one helper

preparation for the next change

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19953>

mesa: remove unused inline functions from arrayobj.h

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19953>

mesa: keep the current VAO also set as _DrawVAO

This removes setting _DrawVAO from all glDraw* functions that draw
primitives because now _DrawVAO is always equal to the current VAO.

glEnd, glRasterPos, and the slow display list path now have to save
_DrawVAO before changing it, and restore it after they are done.

This also removes _EmptyVAO, which becomes unused because we no longer
bind "no VAO" into _DrawVAO.

Rules:
- only glBindVertexArray and non-glDraw* paths should call
  _mesa_set_draw_vao, other places shouldn't
- only draws should call _mesa_update_vao_state, other places shouldn't

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19953>

mesa: split _mesa_set_draw_vao into set and update functions

This is the beginning of splitting _mesa_set_draw_vao and moving pieces
of it out of draw functions.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19953>

radv: Handle NULL miss shaders

Fixes reflections in DOOM Eternal.

Fixes: 85580faa ("radv: Add ray traversal loop.")
Closes: #6210
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20270>

radv: Change shaderGroupBaseAlignment to RADV_RT_HANDLE_SIZE

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20270>

radv: Force non uniform texture sampling with NV Remix

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20243>

radv: Add a driconf option to force non uniform texture sampling

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20243>

spirv: Add a debug option to force non uniform texture sampling

Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20243>

radv/rt: Hash the pipeline key

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20243>

radv/rt: Propagate radv_pipeline_key

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20243>

radv/rt: Work around incorrect NOT_DEFERRED handling in Portal RTX

Allows Portal RTX to reach the main menu.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20243>

tu: Implement VK_EXT_descriptor_buffer

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19849>

vk/descriptor_set_layout: Add optional destructor

Drivers implementing descriptor buffers will want to allocate and free
descriptors with the layout for embedded samplers, so we need a hook to
allow them to free any GPU buffers.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19849>

tu: Fall back to ldg for variable-size inline uniform blocks

With descriptor buffers, we have no way to know how big the descriptor
set actually is, so we have no idea how many constants we can safely
push. If we use a UBO then it will still get pushed, because we normally
assume that we can freely access UBOs without any fear of faults due to
the range checking. This does the easiest thing of using raw pointer
loads, although performance will fall off a cliff, because we don't have
many better options.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19849>

ir3/a6xx: Fix immediate offset stg/ldg path

The immediate offset is in units of bytes, whereas the register offset
is in dwords. We need to compensate for that.

Also, fix an off-by-one when checking the range - the offset field is 13
bits, but the sign bit means we can only represent up to 1 << 12 in
bytes or 1 << 10 in dwords.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19849>

microsoft/compiler: Fix codegen when a loop ends in a jump

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7792
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20255>

microsoft/compiler: Handle holes in driver_location when adding sysvals

All of the full runtime+compiler stacks reassign these driver_location
values to compact them and sort between shader stages, but for the
spirv2dxil tool, we leave the original shader's "location" intact. That
means that there can be holes in the driver_location space, and simply
counting how many inputs there are can lead to collisions. So instead
place all sysvals after the last-used driver_location.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7811
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20253>

ci: Bump virglrenderer version

Update virglrenderer to the latest version on time.

Signed-off-by: Corentin Noël <corentin.noel@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20277>

ir3/freedreno: Find regs for FS inputs when printing info

FS inputs are not directly loaded into regs, but require additional
instruction to do so. So in order to print in which reg the input
is loaded we have to scan the shader for the instruction
which loads the input.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20247>

radv: fix assertion on gpu hang detection

fixes assert in RADV_DECL_PIPELINE_DOWNCAST when bound pipline is a compute
pipeline

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20276>

intel/compiler: Remove leftover declarations of old NIR passes

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19805>

intel/nir/rt: fixup primitive id

There is a delta index value in the hit structure, we forgot to add it
to the base value.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 046571479028 ("intel/nir/rt: add more helpers for ray queries")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7565
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19346>

radv: fix hashing descriptor set layout

Shouldn't have pointers.

Fixes: 19f8d338761 ("radv: Use vk_descriptor_set_layout")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20250>

nir/lower_shader_calls: Remove phis after dead control flow

This potentially gets rid of some more phis without sources.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19960>

nir: Do not consider phis with incompatible dests equal

CSE tries to collapse equal instructions, and collapsing two phis with incompatible dests is illegal.

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Fixes: 6bdce55c ("nir: Add a basic CSE pass")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19960>

docs/release-calendar: add 22.3.x dates

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20131>

zink: Don't set dynamic color attachment state for 0 attachments.

Fixes some validation failures like:

VUID-vkCmdSetColorBlendEquationEXT-attachmentCount-arraylength(ERROR / SPEC): msgNum: -175001922 - Validation Error: [ VUID-vkCmdSetColorBlendEquationEXT-attachmentCount-arraylength ] Object 0: handle = 0xaaaae7632fa0, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0xf591aebe | vkCmdSetColorBlendEquationEXT: parameter attachmentCount must be greater than 0. The Vulkan spec states: attachmentCount must be greater than 0 (https://www.khronos.org/registry/vulkan/specs/1.3-extensions/html/vkspec.html#VUID-vkCmdSetColorBlendEquationEXT-attachmentCount-arraylength)

However, we still have some around dynamic color attachment state:

    Objects: 1
        [0] 0xaaaafcab4150, type: 6, name: NULL
VUID_Undefined(ERROR / SPEC): msgNum: 2044605652 - Validation Error: [ VUID_Undefined ] Object 0: handle = 0xaaaafcab4150, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x79de34d4 | VkCommandBuffer 0xaaaafcab4150[]: Dynamic color blend enable state not set for this command buffer.
    Objects: 1
        [0] 0xaaaafcab4150, type: 6, name: NULL
VUID_Undefined(ERROR / SPEC): msgNum: 2044605652 - Validation Error: [ VUID_Undefined ] Object 0: handle = 0xaaaafcab4150, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x79de34d4 | VkCommandBuffer 0xaaaafcab4150[]: Dynamic color blend equation state not set for this command buffer.
    Objects: 1
        [0] 0xaaaafcab4150, type: 6, name: NULL
VUID_Undefined(ERROR / SPEC): msgNum: 2044605652 - Validation Error: [ VUID_Undefined ] Object 0: handle = 0xaaaafcab4150, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x79de34d4 | VkCommandBuffer 0xaaaafcab4150[]: Dynamic color write mask state not set for this command buffer.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20186>

radv: Use correct watermark for early loop exit.

The previous check assumed the stack starts at offset=0, which isn't
necessarily true for ray queries.

Note that this didn't cause correctness issues, just made an optimization
not apply. Found when I accidentally made this load-bearing in a
refactor.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20265>

radv: Rename stack_base to stack_low_watermark.

Better covers the purpose.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20265>

Revert "r600/sfn: Make use of variable length DOT"

This reverts commit fcafe1ffc85573b6a0df2d64a438f1bac2bae3a2.

Variable length DOT products are not supported for pre EG
cards, and the read port evaluation is not correctly
checked, so that scheduling might fail. Revert for now to
fix the issues below and get gack with a better implementation
later.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7876
Related: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7878

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20268>

cso: remove cso_draw_vbo from all draws, call the driver or u_vbuf directly

Instead of calling like this:
    st_draw_gallium -> cso_draw_vbo -> driver_draw_vbo

Do it like this:
    st_draw_gallium -> driver_draw_vbo
OR
    st_draw_gallium -> u_vbuf_draw_vbo

It's accomplished by adding a draw_vbo function pointer into cso_context.
The pointer is equal to pipe_context::draw_vbo when needed, so there is
no call overhead from this if cso's draw_vbo callback is indeed equal to
driver_draw_vbo. We just call cso_context_base::draw_vbo to jump into
the driver directly, or u_vbuf if needed.

The cso function with the indirect function call is inlined, so draws
don't actually visit any cso_context function.

Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20025>

cso: add a base class cso_context_base holding pipe_context*

We'll add more stuff there. The first change is that we need pipe_context*
there.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20025>

cso: unify cso_draw_vbo and cso_multi_draw

This is going to be inlined.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20025>

gallium/u_vbuf: change u_vbuf_draw_vbo to accept pipe_context as first param

This makes the parameters equal to pipe_context::draw_vbo.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20025>

gallium: add the u_vbuf pointer into pipe_context

This will allow removing the draw_vbo wrapping in cso_context.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20025>

asahi: Implement occlusion queries

While the hardware supports both counter and boolean occlusion queries, the
programming model is quite different from OpenGL. In AGX (and in Metal),
there is a single "visibility result buffer" associated with the render pass.
Each draw that uses occlusion queries writes into this render pass global
visibility result buffer at a particular index. By contrast, the OpenGL
occlusion query model supposes that each query has independent state that can be
mixed and matched within a render pass. We can't simply allocate backing memory
for a query and write to it from a job. We can't allocate visibility result
buffers for each batch up front and statically assign OpenGL queries to indices,
because the OpenGL query can span multiple batches. Finally we can't use a
global visibility result buffer without introducing additional synchronization,
given that we now support multiple batches in-flight at once.

In this patch, I've elected to use a simple solution: allocate visibility result
buffers and indices on the fly as needed, and accumulate the results on the CPU
at the end of the render pass. When we have proper synchronization we'll want to
revisit this, but as everything is stalling at submit time now, I'm not inclined
to "optimize" something I can't test.

Passes dEQP-GLES3.functional.occlusion_query.* and the relevant piglit tests.
The piglits are considerably more thorough, checking lots of "we hate tilers"
conditions that dEQP skips over.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Move query functions to agx_query.c

New file. They're just stubs now but will get nontrivial in a moment.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Don't upload samplers for clears/stores

Unlikely to help but makes the traces neater.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Avoid reloads with staging blits

Noticed by inspection. Not likely to matter unless these staging blits are in a
hot path, but it's an easy win.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Don't reload uninitialized surfaces

Pointless. This should save some bandwidth in some cases (possibly mipmap
generation?)

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Fix Z32S8 harder

Fixes dEQP-GLES3.functional.texture.format.sized.2d.depth32f_stencil8_pot after
stencil texturing broke it.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Don't shadow idle resources

Pointless allocation+memcpy.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Model alignment of occlusion query indices

8-byte offsets.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Identify some more fields used with layered

These values depend on the framebuffer width/height and maybe other
stuff. Maybe strides.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Identify XML for anisotropic filtering

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Check-box implement rasterizer discard

Passes dEQP-GLES3.functional.rasterizer_discard.*

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Wire in 1D (array) textures

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Dirty track VS/FS key updates

drawoverhead 1 score doubled to 7668.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Separate VS/FS shader keys

First remove agx_shader_key from asahi_shader_key. It's trivial. agx_shader_key
is going to go away soon now that we lower everything in NIR.

Then everything else is mutually exclusive between stages. That means much
less to hash. drawoverhead test 1 from 2331 to 3443.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Implement invalidate_resource

From Panfrost. This lets us avoid storing depth/stencil attachments at the end
of the frame in GLES.

On my 4K monitor, glmark2 -btexture at fullscreen goes from 705fps to 1150fps. I
assume gains on real workloads will be smaller.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Track batch masks on ZS/blend CSO

Adapted from panfrost, with the work happening at CSO create time instead of
draw time allowing us to do more sophisticated analysis. We'll use these for
accurate masks in a moment.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Dirty track blend state

We'll want this to reduce variant lookups eventually.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

asahi: Enable dirty tracking

Whoops. drawoverhead test 1 score from 496 -> 2377.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

ail: Save level_offsets_compressed_B

So we can bind specific mip levels for rendering into compressed Z/S.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20123>

panfrost: implement clear_depth_stencil

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20238>

nine: enable on panfrost

Also, enable required kmsro dependencies.

Tested-by: Aleksey Komarov <q4arus@ya.ru>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20238>

panfrost: implement clear_render_target

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Co-authored-by: Aleksey Komarov <q4arus@ya.ru>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Aleksey Komarov <q4arus@ya.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20238>

panfrost: Handle resources without depth in batch_to_fb_info

Prevent preloading data from resources which doesn't exist.

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Aleksey Komarov <q4arus@ya.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20238>

panfrost: Implement GL_EXT_clip_control

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Co-authored-by: Aleksey Komarov <q4arus@ya.ru>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Aleksey Komarov <q4arus@ya.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20238>

intel: add devinfo->has_64bit_float_via_math_pipe

Unusual hardware features that require special hanlding usually get a
devinfo field, so do this for MTL's unordered DF types. This will
guarantee that any platform based on MTL (thus inheriting from
MTL_FEATURES) will automatically be handled in these special cases.

v2: s/has_unordered_64bit_float/has_64bit_float_via_math_pipe/ (Curro).

Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20072>

intel/compiler: fix intel_swsb_decode for newer platforms

In the previous patch we adjusted the scoreboard pass to take into
consideration a new case of unordered operations for TGL. Fix the
decoding as well.

v2: use intel_device_info_is_mtl()  (Curro, Jordan)
v3: the part where we export num_sources_from_inst() is now a separate patch
    (Curro).
v4: Work around false positive maybe-unitialized warning since Marge
    uses -Werror=maybe-uninitialized (Marge).

Reviewed-by: Francisco Jerez <currojerez@riseup.net> (v3)
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20072>

intel/compiler: export brw_num_sources_from_inst

We want to call this from brw_disasm.c, so move it out to brw_eu.c
since it's about to become more of a shared utility function than
something specific to the EU validator.

Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20072>

intel/compiler: avoid 64bit SEL_EXEC on MTL

On MTL, instructions with DF type are unordered, executed in the math
pipe. This means that they require different SWSB dependency handling,
and also that in some cases such as MOVs it's generally faster to
simply use 2 smaller ordered moves than a single unordered MOV.

One problem we have with the current code is that generate_code() is
not setting the proper SWSB dependencies for the generated DF MOVs,
causing some tests to fail.

One solution would be to fix generate_code() by making it set the
appropriate dependencies. This was the first patch I wrote. Another
solution to this problem, pointed to us by Curro, is to change
required_exec_type() so we use UD instructions instead of DF, just
like we do with platforms that don't have 64 bit instructions, which
means there won't be anything to fix in generate_code(). The second
solution is what this patch implements.

This fixes at least:
 - dEQP-VK.subgroups.arithmetic.framebuffer.subgroupmin_double_vertex

Thanks to Francisco Jerez for all the major help provided with this
problem.

Credits-to: Francisco Jerez <currojerez@riseup.net>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20072>

intel/compiler: avoid (RegDist, SBID) on DF instructions on MTL

When we use this form there's no way to specify which pipe RegDist
refers to, so there are a few rules to figure this out, which is what
inferred_sync_pipe() implements. But for MTL there's no long pipe and
the documentation does not explicitly explain what should be the
inferred type for its long (DF) instructions - which are out-of-order,
by the way.  One way to interpret this is that such case should be
avoided.  So add the extra check to entirely avoid this case.

Notice that this is not actually fixing any bug, since returning
TGL_PIPE_LONG (what we do today) will actually make these DF
instructions incompatible with every in-order instruction, so we'll
never opt to use the (RegDist, SBID) form anyway. But still, it's
better to have this case explicitly documented instead of having it
covered by a semi coincidence.

v2: use intel_device_info_is_mtl()  (Curro, Jordan)

Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20072>

intel/compiler: on MTL, DF instructions run in the math pipe

Adjust the scoreboard code to take that into account.

Fixes at least:
  - dEQP-VK.glsl.builtin.precision_double.refract.compute.vec3
  - dEQP-VK.glsl.builtin.precision_double.matrixcompmult.compute.mat4

v2: use intel_device_info_is_mtl()  (Curro, Jordan)

Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20072>

intel/fs: Make the result of is_unordered() dependent on devinfo.

Reviewed-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20072>

Revert "anv: compile anv_acceleration_structure.c"

This reverts commit 74d0be27ae9eb666df948874a20a3b4464df7db1.

Also remove anv_acceleration_structure.c, it was meant to be removed
earlier. There was probably a rebase issue somewhere.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20248>

freedreno/a6xx: fix blend all_mrt_write_mask

Fix all_mrt_write_mask when independent_blend_enable is false.
Otherwise, lrz write is always diabled with MRT when
independent_blend_enable is false.  This fixes a 2% perf regression for
multiple gfxbench benchmarks.

Fixes: 0132c22de75 ("freedreno/a6xx: Don't disable LRZ for invalid channels")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20254>

iris: Improve direct CPU map heuristics

We were promoting reads with a valid primary to direct CPU maps even
if the mmap mode was IRIS_MMAP_WC, which would mean uncached reads from
VRAM.  In that case, GPU blits are in fact useful!

We were also only checking for !DISCARD_RANGE rather than MAP_READ,
which isn't a great idea for image maps, given the discussion in the
previous commit about image map semantics.

The original code was also just confusingly structured.  Make a helper
function with clearly defined cases where we want to bail on CPU maps.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19209>

iris: Only copy existing data into staging images with PIPE_MAP_READ

When performing transfer maps on images that require staging buffers
(say, for presenting a linear view of tiled memory), we were reading
the existing contents of the buffer into the staging resource on map
unless PIPE_MAP_DISCARD_RANGE was set.

The thinking was to support partial writes.  If you map a subrectangle
of an image, but then only write selective pixels - should it preserve
the existing contents of the mapped region?  I believed that it should,
unless you pass PIPE_MAP_DISCARD_RANGE to explicitly say that that it's
okay to invalidate the destination region.

However, that does not appear to be the interpretation favored by other
Mesa developers (in particular Michel Dänzer and Marek Olšák).  The
radeonsi driver does not do this readback from the destination region
to the staging buffer unless you pass PIPE_MAP_READ.  If you want to
do a partial write and preserve contents, you need to pass both flags:
(PIPE_MAP_READ | PIPE_MAP_WRITE).  Passing READ is expected to come
with an associated cost.

OpenGL defines GL_MAP_INVALIDATE_RANGE_BIT for mapping buffer objects,
which is translated to PIPE_MAP_DISCARD_RANGE.  However, unextended
OpenGL doesn't define mapping textures.  There are two main sources of
image maps: our internal MapTextureImage() hook, and gbm_bo_map().

I've audited our internal MapTextureImage() calls, and while some do
pass PIPE_MAP_DISCARD_RANGE, almost all of them wholly overwrite the
mapped region, and those that care about combining with existing image
contents all pass PIPE_MAP_READ.  So this should work there.

GBM defines three flags: GBM_BO_TRANSFER_READ, WRITE, and READ_WRITE.
There is no defined "invalidate range" bit.  In issue #6020, Matthias
Treydte notes that this extra readback can cause performance problems,
and with iris's current interpretation, there's no way to avoid it.
During that discussion, Michel and Matthias both argued that
GBM_BO_TRANSFER_WRITE should invalidate the destination contents and
avoid the readback, while GBM_BO_TRANSFER_READ_WRITE would preserve it.

This patch makes iris follow that model for image mappings, removing
readback on staging maps for both detiling and stall avoidance, unless
PIPE_MAP_READ is passed.  I believe we can change this with impunity.

For buffer objects, Ian Romanick and I both agree that partial writes
should be supported, and GL_MAP_INVALIDATE_RANGE_BIT exists precisely
to indicate that you should spend effort preserving existing contents.
So we continue doing readback for buffers unless PIPE_MAP_DISCARD_RANGE
is flagged, for now.  While I think this is work, it also seems to be
undertested in the CTS and Piglit.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6020
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19209>

iris: Return idle status from iris_invalidate_buffer, skip busy checks

If we successfully replace the backing storage for a buffer, we know
that it's idle, and the transfer map code can mark it unsynchronized
right away, letting us skip redundant resource_is_busy() checks.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19209>

iris: Don't replace backing storage for exported buffers.

We already gave out the old BO...or acquired it from somewhere which
may be affecting it.  We simply can't replace the backing store.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19209>

iris: Promote DISCARD_RANGE to DISCARD_WHOLE_RESOURCE where possible

This allows us to replace the backing storage for a buffer, which
means we'd have an idle buffer and thus could do an unsynchronized
mapping where we otherwise wouldn't.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19209>

iris: Use persistent mappings for pinned memory (userptr)

This is a port of Nicolai's b52721e3b693e113aa537d163c8a855169e7b75d
from radeonsi.  Because GL_AMD_pinned_memory guarantees that mappings
will refer to the same underlying page, we need to avoid using staging
maps.  Using a persistent map is a reasonable way to accomplish this.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19209>

iris: Delete map->dest_had_defined_contents

Dead since commit 6cc09699cd77286f7601a1afab60dc9fa0bafd48.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19209>