ac/llvm: add a callback to ac_cull_triangle to generate code in inner-most block

This will reduce jumps in culling code.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11486>

radeonsi: fix multi draws for the prim discard CS

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11486>

radeonsi: fix incorrect counting of compute_num_verts_rejected

Both cases should subtract to undo what is done in the conditional.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11486>

radeonsi: use ac_build_bit_count instead of opencoding it

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11486>

radeonsi: fix compile failures with SI_PRIM_DISCARD_DEBUG enabled

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11486>

egl: fix in expected type

Function mincore expects a pointer of type char* but we use an unsigned
char* instead generating signedness related warnings.

v2: Made the fix FreeBSD specific because the type is unsigned char* for
Linux and char* for FreeBSD. (Adam Jackson)

v3: We'd rather cast the param to (void*) to avoid warnings in all
systems (Adam Jackson)

Signed-off-by: Eleni Maria Stea <elene.mst@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11298>

crocus: Drop extra_aux support

This exists for combined MCS+CCS or HiZ+CCS which was introduced on
Tigerlake.  Crocus will never support hardware that has these features
so there's no point carrying the dead copied+pasted code from iris.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11483>

radv: Enable VK_KHR_acceleration_structure with RADV_PERFTEST=rt.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11078>

radv: Add rt perftest flag.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11078>

radv: Expose formats for acceleration structure.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11078>

radv: Implement load_vulkan_descriptor for acceleration structures.

It always uses nir_address_format_64bit_global.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11078>

radv: Convert lower_intrinsics to a switch statement

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11078>

radv: Add acceleration structure descriptor set support.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11078>

radv: Implement device-side BVH building.

Same naive algorithm as the host build.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11078>

radv: Add initial CPU BVH building.

The algorithm used for the BVH:

1) first create 1 leaf per primitive (triangle/aabb/instance)
2) Then create internal layers from the bottom up until we are left with
   1 node in the top layer. Node i in the layer will have children
   (i*4+0) ... (i*4+3) in the previous layer.

This results in a very naive algorithm but it is also very simple to implement.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11078>

radv: Use the global BO list for acceleration structures.

We have nested structures so tracking this from the descriptor
set is going to be a mess.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11078>

util: Move the 4x4 matrix inverse function to u_math

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11078>

freedreno/ci: Increase # of jobs for CI runners

The idea is that the tests will spend *some* time stalling waiting to
read back results from the GPU.  So use a # of jobs that is slightly
more than the # of CPUs to keep the CPUs more busy.

Locally this is dropping a bit more than a minute off a parallel
deqp-gles31 run, so turn it on across the board for a6xx.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11477>

freedreno/ci: Start longest traces first

Shave off a bit of runtime on the CI job by starting the longer traces
first.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11477>

crocus/gen6: fix depth blit blorp regression.

The tesseract fix broke depth blits using blorp as depth blits
on gen6 are done using the color engine. Just disable aux
up front on the destination for this case.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11456>

freedreno/ir3: Move NIR printing to mesa_log.

Now we can get some NIR debug on Android.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9262>

nir: Add an interface for logging shaders with mesa_log*.

For debug on Android, it's useful to be able to print shaders to the
android log interface, since you don't usually have stdout/stderr.

Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9262>

freedreno/ir3: Move the native code output to mesa_log as well.

I didn't feel like rewriting ir3_shader_disasm() off of FILE *s, so use
the same trick as the disasm_info path above to write to memory and then
hand the multi-line blob off to mesa_log.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9262>

freedreno/ir3: Use mesa_log_stream() for ir3 disassembly.

This means you can get dumps on android, and output on Linux goes to
stderr.  However, this does mean that on Linux the output goes from
looking like:

AFTER: ir3_legalize:
block3276208368 {
0000:0001:002:  cov.u32s16 hr2.x, c2.x
0000:0002:002:  mov.u32u32 r0.x, c0.x
[...]

to:

MESA: info: AFTER: ir3_legalize:
MESA: info: block3405271904 {
MESA: info:  0000:0001:002:  cov.u32s16 hr2.x, c2.x
MESA: info:  0000:0002:002:  mov.u32u32 r0.x, c0.x
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9262>

util/log: Add a streaming printf interface.

Often disassemblers and things in our drivers want to be able to
incrementally printf together a line, but that gets in the way of
Android's logging that wants to see a whole line all at once.  Make a
little wrapper to do the ralloc_asprintf_rewrite_tail() and flushing lines
as they appear.

Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9262>

freedreno/ir3: Move the assert output to mesa_loge().

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9262>

freedreno: Move some driver debug printfs to mesa_logd.

This means the logging will work on Android, and won't get mixed up in
application stdout on Linux.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9262>

i915g: Add triangle provoking vertex support.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11470>

docs: update calendar and link releases notes for 21.1.3

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11472>

docs: add release notes for 21.1.3

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11472>

nine: add zink to the build target

Reviewed-by: Axel Davy <davyaxel0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11467>

nine: only enable tgsi disk cache if the driver supports it

this crashes otherwise

Reviewed-by: Axel Davy <davyaxel0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11471>

radv: fix dynamic culling and depth/stencil related dynamic states

To avoid overwriting previous dynamic state with default state from
the pipeline.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4926
Cc: 21.1 mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11375>

radv: move pipe_misaligned and l2_coherent image checks to flags set on init

this should save 4-5% cpu in some cases

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11462>

radeon/vcn: allocate non-tmz context buffer for VCN2+

By design, context buffer should be allocated as TMZ buffer for secure playback
for VCN 1 only. For VCN 2&2+, context buffer should be moved out of TMZ.

Signed-off-by: Boyuan Zhang <boyuan.zhang@amd.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11388>

radeon/vcn: move calc_dpb_size into create_decoder

Dpb buffer size calculation should based on the values provided in player's
decoder creation call. db_alignmet should be decided in decoder creation
call as well. Therefore, move db_alignment and dpb buffer size calculation
from rvcn_dec_message_decode to radeon_create_decoder function.

Signed-off-by: Boyuan Zhang <boyuan.zhang@amd.com>
Reviewed-by: Leo Liu <leo.liu@amd.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11378>

docs/isl: Add detailed documentation about CCS compression

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11366>

docs/isl: Add detailed documentation about tiling on Intel GPUs

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11366>

docs/isl: Add detailed documentation about isl formats

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11366>

docs/isl: Document ISL's units

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11366>

isl: Document more members of isl_surf

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11366>

docs: Begin documenting ISL

This commit mostly just adds the framework required to scrape
documentation out of the ISL sources and headers.  The method chosen
here is a combination of doxygen and breathe (a sphinx extension for
doxygen integration).  I'll freely admit that doxygen is pretty terrible
but it seems like the best option we have available to us today.

Acked-by: Emma Anholt <emma@anholt.net>
Acked-by: Daniel Stone <daniels@collabora.com>
Acked-by: Erik Faye-Lund <erik.faye-lund@collabora.com>.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11366>

zink: remove inlinable_uniforms_dirty_mask

this should've always just been flagging the shaders dirty directly

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11405>

zink: remove duplicated bitflag filtering for inline uniforms

'bits' already does this

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11405>

radv: create only one pipeline for decompressing depth/stencil images

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11263>

radv: always decompress both aspects of a depth/stencil image

If compressed rendering is only used for the depth aspect of a
depth/stencil image, stencil might also be compressed and it needs
to be decompressed. This only happens for non-TC compatible images.

As long as the driver needs to decompress the depth aspect, I don't
think that decompressing the stencil aspect introduces extra cost.

Fixes dEQP-VK.renderpass*late_fragment_tests*.d32_sfloat_s8_uint for
chips that don't support TC-compat HTILE.

Cc: 21.1 mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11263>

radv: reject binding buffer/image when the device memory is too small

From the Vulkan spec 1.2.181:
    "The difference of the size of memory and memoryOffset must be
     greater than or equal to the size member of the
     VkMemoryRequirements structure returned from a call to
     vkGetImageMemoryRequirements with the same image"

This is invalid usage but adding a check in the driver is safe and
might avoid spurious failures.

This is a workaround for the inventory GPU hang with Cyberpunk 2077
which is actually a game bug. Luckily the game handles this error
gracefully.

Since the addrlib change from March, addrlib now selects a better
swizzle mode (4KB instead of 64KB) which reduces image size. Though,
the game assumes that an image with 2 mips is always smaller than the
same image but with 6 mips. This is not always true if the swizzle mode
is different. Then, it creates a D312 heap that is too small for the 2
mips image and the GPU hang with a memory violation, ugh...

Note that next vkd3d-proton release should also reject this but
fixing both sides is fine.

Cc: 21.1 mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4823
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4593
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11448>

radeonsi: skip instance_count==0 draws on <= GFX9

This changes seems to prevent a hang, at least on Renoir chips.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4866
Cc: mesa-stable
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11356>

nvc0/ir: Initialize Limits members in constructor.

Fix defect reported by Coverity Scan.

Uninitialized scalar field (UNINIT_CTOR)
uninit_member: Non-static class member min is not initialized in this constructor nor in any functions that it calls.
uninit_member: Non-static class member max is not initialized in this constructor nor in any functions that it calls.

Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11351>

intel/vec4: Add missing break statement.

Fix defect reported by Coverity Scan.

Missing break in switch (MISSING_BREAK)
unterminated_case: The case for value
VEC4_OPCODE_ZERO_OOB_PUSH_REGS is not terminated by a break
statement.

Fixes: 89fd196f6b5 ("intel/vec4: Add support for masking pushed data")
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11347>

i915g: Switch to using nir-to-tgsi.

This fixes ~10% of the GLES2 failures thanks to having a better compiler,
though in some cases we get some new compile fails due to instr count or
uniform count.  We still have to do NIR-to-TGSI because the NIR gallivm
draw path isn't ready for non-native-integer NIR code, and st/mesa treats
native-integer as a screen property instead of a stage property.

Other than the noted regressions in the xfails, for
dEQP-GLES2.functional.uniform_api.random.74: compile fail changes reasons
triggering an aassertion instead of drawing magenta.

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11329>

i915g: Handle fragment depth being in OUT[1] not OUT[0].

Prevents regressions when switching to nir-to-tgsi which orders the
outputs differently.

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11329>

nir_to_tgsi: Support integer sysvals on !CAP_INTEGERS hardware.

glsl_to_tgsi does the same thing, needed for the draw path on i915g.

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11329>

nir_to_tgsi: Fix internal handling of NIR uints for !CAP_INTEGERS

If we called nir_lower_int_to_float(), then ALU-consumed ints got turned
into floats and we have to interpret them that way.

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11329>

nir: Do peephole select on other instructions if the limit is ~0.

limit==0 is the signal for "don't peephole anything but a move that will
be optimized aways."  limit > 0 is "up to N alu instructions may be moved
out."  nir-to-tgsi uses ~0 as the indicator of "No, we really need to
eliminate all if instructions" on hardware like i915 that doesn't have
control flow.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11329>

nir/lower_int_to_float: Make sure the cursor is in the right spot.

We need to make get it updated after we may have nir_instr_remove()d an
instruction, and when we cross blocks.  This didn't really matter before
because the only builder usage was idiv, which other users of
lower_int_to_float were probably never hitting.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11329>

i915g: Allow fragment coord conventions TGSI properties to be set.

The frontend lowering handles normalizing the conventions to the only
model we support, we just need to ignore the property in the TGSI.

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11329>

i915: Drop assertion failure about seeing each const decled once.

nir_to_tgsi sometimes emits multiple decls currently, but we don't
actually care because we're just checking which ones are live.

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11329>

i915: Disable vertex texturing and delete the code.

It's not a required feature of the GL2.1 or GLES2, and you really don't
want to be doing SW VS access of the write-combined texture data.  Also,
avoids memory corruption in deqp:

Test case 'dEQP-GLES2.functional.texture.vertex.cube.filtering.linear_mipmap_nearest_linear_repeat'..
Mesa: User error: GL_INVALID_ENUM in glGetIntegerv(pname=GL_MAJOR_VERSION)
  Fail (Image comparison failed)

Test case 'dEQP-GLES2.functional.fragment_ops.depth_stencil.stencil_depth_funcs.stencil_equal_depth_always'..
==559181== Invalid read of size 4
==559181==    at 0x641E8D0: i915_drm_buffer_unmap (i915_drm_buffer.c:204)
==559181==    by 0x64151EB: i915_cleanup_vertex_sampling (i915_state.c:449)
==559181==    by 0x640AEA7: i915_draw_vbo (i915_context.c:134)
==559181==    by 0x640AEA7: i915_draw_vbo (i915_context.c:55)
==559181==    by 0x61367B1: cso_draw_vbo (cso_context.c:1524)
[...]

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11329>

anv: enable multi-planar support for drm format modifier

This patch only enables the below VkFormat:
- VK_FORMAT_G8_B8R8_2PLANE_420_UNORM

This patch ensures the proper behavior of the below APIs:
- vkGetPhysicalDeviceFormatProperties2
- vkGetPhysicalDeviceImageFormatProperties2
- vkCreateImage
- vkGetImageSubresourceLayout
- vkGetImageDrmFormatModifierPropertiesEXT
- vkGetImageMemoryRequirements
- vkGetImageMemoryRequirements2

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Chad Versace <chad@kiwitree.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11281>

anv: support multi-planar format in add_all_surfaces_explicit_layout

Add initial multi-planar format support on the images with modifiers:
- With aux usage,
  - Format plane count must be 1.
  - Memory plane count must be 2.
- Without aux usage,
  - Each format plane must map to a distinct memory plane.

For the other cases, currently there is no way to properly map memory
planes to format planes and aux planes due to the lack of defined ABI
for external multi-planar images.

This patch doesn't include some potentially supported cases like all
format planes mapping to a single memory plane, additional refactoring
is needed to workaround explicit base offset + ANV_OFFSET_IMPLICIT.

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Chad Versace <chad@kiwitree.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11281>

anv: fix some log formats

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Chad Versace <chad@kiwitree.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11281>

freedreno: Add some cheza flakes from the last week.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11453>

freedreno: Skip staging blits from uninitialized resources.

When storing depth- or stencil-only texture data that has been packed into
a depth/stencil texture, the tex store gets PIPE_MAP_READ added onto it
since the other channel will get ORed into the incoming data, but
sometimes we know that the other component is undefined because the whole
texture is either fresh or just invalidated.

Cleans up a confusing extra blit in a dEQP case I've been debugging, and
should be less work for dEQP CI.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11452>

freedreno: Add more detailed blit debug in FD_MESA_DEBUG=msgs.

For debugging the batch cache, it really helps to see the blits that
happen, and which are staging blits in the transfer map process.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11452>

freedreno/fdl: Give the tiling mode a nice name in debug dumps.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11452>

freedreno: Move FD_MESA_DEBUG=msgs output to mesa_logi.

It didn't work unless you had a debug build, and I regularly want to use
it on non-debug builds.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11452>

freedreno: Add perf_debug() for our software conditional rendering.

We could do it in hardware, and turnip does, but it hasn't bubbled up our
priorities yet.  At least make it more discoverable when you stumble over
it.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11452>

freedreno: Defer freeing batch->key

We use the same key in autotune to track historical data about a given
framebuffer state, to inform the decision about using gmem vs sysmem
rendering.  Which means we need the key to stick around during the
flush, even if the batch is removed from the batch-cache before the
flush.

Fixes: 507f701d9e8 ("freedreno: Fix batch flush race condition")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11450>

spirv: Fix handling of OpBranchConditional with same THEN and ELSE

When an OpBranchConditional that had two equal branches was parsed, we
were treating it as a regular OpBranch.  However this doesn't work
well when there's an associated OpSelectionMerge.  We ended up
skipping marking the merge block as such, and depending on what was
inside the construct we would end up trying to process the block
twice.

Fix this by keeping the vtn_if around, but when emitting NIR identify
the two equal branch case.

Fixes: 9c2a11430e1 ("spirv: Rewrite CFG construction")
Closes: #3786, #4580
Reviewed-by: Yevhenii Kolesnikov <yevhenii.kolesnikov@globallogic.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9297>

pan/mdg: Fix reading a spilt register in the bundle it's written

Read directly from the instruction getting spilt. Otherwise a fill
will be inserted before the spill writing the value, so the
instruction reading the spilt value gets garbage data.

Use the bundle_id to check if the instructions are in the same bundle.

Insert a move instruction, as the spill needs the value in a LD/ST
register such as AL0, while the ALU instruction reading the value
needs it in a work register such as R0.

Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4857
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11212>

pan/mdg: Fill from TLS before spilling non-SSA nodes

Otherwise the data already written to the node will get overwritten.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11212>

pan/mdg: Reorder some code in mir_spill_register

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11212>

pan/mdg: Add a bundle ID to instructions

So that it is possible to check if two instructions were scheduled
into the same bundle.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11212>

freedreno/a6xx: Skip nv_copy_image tests

These look pretty redundant with arb_copy_image, so skip to keep CI
runtime reasonable.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11402>

freedreno/a6xx: Flip on copy_image

Now that we have the rest of format "casting" sharp edges sorted, flip
on copy_image and gles32.

Unfortunately it adds back to piglit xfails (but at least that is more
than offset by my previous round of piglit fixes, and these are pretty
much all things we know had issues based on corresponding nv_copy_image
tests).

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11402>

freedreno: Fix for multi-draw blits

We have some logic to detect when u_blitter generated draws overwrite
the entire render-target, so we know we can discard anything previous.
But some blits (like multi-sample) do multiple draws.  We don't want to
discard the earlier draws from the same blit.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11402>

freedreno/a6xx: Handle u/snorm vs u/sint validation

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11402>

freedreno/a6xx: Use UNORM for SNORM copy blits

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11402>

freedreno/blitter: Flush before self-blits

In paths where we are handling blits on the 3d pipe, if src==dst we need
to flush to ensure what gets sampled by the blit shader reflects the
results of any previous blits.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11402>

freedreno: Fix flushes with NULL batch

Sequences that pctx->set_framebuffer_state() before pctx->flush() will
see ctx->batch being NULL.. but they still need to call fd_bc_flush(ctx)
to ensure pending batches associated with the context are flushed.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11402>

intel/nir: Fix txs for null surfaces

Closes: #4860
Fixes: 05a37e24220 ("intel/nir: Set lower txs with non-zero LOD")

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11435>

freedreno: Flush batches upon destroying the ctx.

The invalidate would take it out of the bc tracking, so you could go
allocate a new batch->idx matching this one, while this one is still in
the bc using that idx.

You can't generate any new rendering with the ctx's old batches at this
point, anyway, so just flush for simplicity.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11439>

freedreno: Remove broken back_blit optimization.

It wasn't checking that the transfer map would definitely overwrite all of
the data being initialized by the back blit, and if we knew that it
would then the caller would have provided PIPE_MAP_DISCARD_WHOLE_RESOURCE.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11439>

freedreno: Move the !MAP_WRITE write batch refcounting to the branch.

For MAP_WRITE, we flush all the batches referencing the BO, so the write
batch will get flushed anyway.  No need to take an extra ref.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11439>

freedreno: Fix batch reference handling in flush_resource().

We take references under the lock, but then accessed the lock-requiring
batch_cache structure without holding the lock.  The batches wouldn't get
freed and removed from their slots until the last ref goes away so it was
safe (other than the assert at the end), but writing the simple code is
shorter and requires fewer assumptions.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11439>

v3dv: implement VK_EXT_index_type_uint8

Relevant CTS tests:
dEQP-VK.pipeline.input_assembly.*.index_type_uint8.*

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11444>

v3dv: enable ASTC formats

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11336>

v3dv: add ASTC formats to get_compatible_tlb_format

CTS doesn't seem to hit this, but they're all 128bit formats so this
should be right

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11336>

v3dv: divide by block size in copy_image_blit

This handles compressed formats with non-4x4 blocks, like ASTC.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11336>

v3dv: add the unswizzled RGBA4444 format

If we're supporting the R/B swapped one we might as well support the one that
isn't.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11336>

r600/sfn: Clean up some ALU lowering and move code

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11348>

r600/sfn: Don't read return values of atomic ops that are not used

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11348>

r600/sfn: Drop method for emit_atomic_add, it is handled in generic code

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11348>

r600/sfn: don't read back unused image atomic result values

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11348>

r600/sfn: don't designates initializers, since they are c++20

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11348>

iris: Avoid abort() if kernel can't allocate memory

When the system doesn't have enough memory, GNOME Shell may be crashed
by iris:
gnome-shell[1161]: iris: Failed to submit batchbuffer: Cannot allocate memory
gnome-shell[1161]: GNOME Shell crashed with signal 6

So don't abort() when kernel can't allocate memory to avoid crashing the
entire desktop.

Acked-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11178>

v3dv: expose VK_KHR_shader_non_semantic_info

This is entirely implemented in the SPIR-V frontend.

Relevant CTS tests:
dEQP-VK.spirv_assembly.instruction.compute.non_semantic_info.*

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11440>

disk_cache: use UTIL_QUEUE_INIT_SCALE_THREADS

Instead of spawning 4 threads when the cache is created,
spawn 1 and let u_queue grow the number of threads if
needed.

I wrote this patch because when running piglit's quick_shader
profile I had lots of samples in disk cache threads - mostly
in native_queued_spin_lock_slowpath kernel function.

Since these tests shouldn't really stress the cache, I assumed
it was caused only by thread creations.

After writing the patch and redoing the measurement, I got an
improvement but I still more hits in the same function for
shader_runner:$disk0 thread so something was wrong.

After digging more, I found out that my shader cache index was
corrupted: the on-disk size was 29MB but the index reported it
was way more than 1GB. So each disk cache thread was spending
a lot of time trying to evict files. Given that my cache had
a really low count of files, the LRU method based on randomly
generating subfolder names failed, so evicting was very slow.

Now that my cache index is fixed, the disk cache threads are
mostly idle but I still think it makes sense to grow the
number of threads instead of spawning 4 at the program start.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11296>

util/u_queue: add UTIL_QUEUE_INIT_SCALE_THREADS flag

This flag allow to create a single thread initially, but set
max_thread to the request thread count.

If the queue is full and num_threads is lower than max_threads,
we spawn a new thread to help process the queue faster.

This avoid creating N threads at queue creation time.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11296>