ir3: Rewrite register allocation

Switch to the new SSA-based register allocator.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9842>

ir3: Expose occupancy calculation functions

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9842>

ir3: Add pass to lower arrays to SSA

This will be run right after nir->ir3. Even though we have SSA coming
out of NIR, we still need it for NIR registers, even though we keep the
original array around to insert false dependencies.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9842>

ir3: Add dominance infrastructure

Mostly lifted from nir.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9842>

ir3: Remove unused check_src_cond()

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9842>

ir3/postsched: Don't use SSA source information

This was only used for calculating if a source is a tex or SFU
instruction, which is easily replacable. It's going away with the new
RA.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9842>

ir3/delay: Delete pre-RA repeat handling

It looks likely that any implementation of (rptN) in ir3 will have to
actually create (rptN) instructions after RA, which means that this can
be dropped.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9842>

ir3: Rewrite delay calculation

The old delay calculation relied on the SSA information staying around,
and wouldn't work once we start introducing phi nodes and making
"normal" values defined in multiple blocks not array regs anymore.
What's worse is that properly inserting phi nodes when splitting live
ranges would make that code even more complicated, and this was the last
place post-RA that actually needed that information.

The new version only compares the physical registers of sources and
destinations. It works by going backwards up to a maximum number of
cycles, so it might be slightly slower when the definition is closer but
should be faster when it is farther away.

To avoid complicating the new method, the old method is kept around, but
only for pre-RA scheduling and it can therefore be drastically
simplified as the array case can be dropped.

ir3_delay_calc() is split into a few variants to avoid an explosion of
boolean arguments in users, especially now that merged_regs now has to
be passed to it.

The new method is a little more complicated when it comes to handling
(rptN), because both the assigner and consumer may be (rptN). This adds
some unit tests for those cases, in addition to dropping the to-SSA code
in the test harness since it's no longer needed.

Finally, ir3_legalize has to be switched to using physical registers for
the branch condition. This was the one place where IR3_REG_SSA remained
after RA.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9842>

ir3: Make branch conditions non-SSA

In particular, make sure they have a physreg assigned. This was the last
place after RA where SSA registers were created, which won't work with
the new post-RA delay calculation that relies on the physreg.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9842>

ir3: Add reg_elems(), reg_elem_size(), and reg_size()

For working with registers in units of half-regs in the new RA.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9842>

ir3/delay: Fix full->half and half->full delay

The current compiler never does this, but the new compiler will start to
in mergeregs mode. There is an extra penalty for this.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9842>

ir3: Add ir3_register::array.base

There were two different approaches I saw in the post-RA code for
figuring out what regiser range a relative access touched:

1. Use reg->array.offset and reg->array.size. This is wrong in case
   reg->array.offset was non-zero before RA, because array.size is
   the size of the whole array and array.offset has the const offset
   within the array baked in.
2. Lookup the array from the array ID and use the base + range there.
   This is correct, but won't work with the new RA, where an array might
   not always be assigned to the same register.

This replaces both methods with a new ir3_register::array.base field,
and switches all the users I could find to it.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9842>

ir3: Improve register printing for SSA

Print the ssa name for array destinations, and handle printing undef SSA
sources.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9842>

ir3: Prepare for instructions with multiple destinations

To simplify the pre-RA merge set code and express the result live-range
splitting in RA, we need to add support for parallel copy instructions,
and for the merge set code these parallel copies need to be in SSA form.
Parallel copies have multiple destinations by necessity, but there was
no way to express this in the existing IR. In particular there was no
support for marking a register as being a destination, and no support
for indicating which destination register out of several an SSA source
refers to. This replaces ir3_register::instr with ir3_register::def and
re-purposes ir3_register::instr. I haven't propagated this into common
helpers, like ssa(), because that would vastly increase the amount of
churn and the number of places that produce such instructions should be
limited -- only RA will create parallel copies and they will be
destroyed right after RA. In the future swz will have multiple
destinations too, but it will only be created after RA via parallel copy
lowering.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9842>

ir3: Readd support for translating NIR phi nodes

This is roughly based on the support removed a while ago, but it handles
sources better by associating each source with a predecessor block.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9842>

ir3: Add ir3_start_block()

Name based on nir_start_block(). A number of places were already
open-coding this, convert them.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9842>

ir3: Introduce phi and parallelcopy instructions

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9842>

docs/panfrost: Update API versions

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

docs/features: Mark GLES3.1 as done on Panfrost

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost/ci: Do fractional dEQP-GLES31 run on Midgard

Drop the skip list and correspondingly populate the fails list.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost/ci: Don't skip SSBO tests on G52

These were blocked on failing RA, but that's been resolved now.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost/ci: Blank G52 flakes file

Haven't seen these tests flake, and we don't even run dEQP-GLES2 on G52
in CI anymore. (I still do local runs, and I don't see them flake
there.)

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/decode: Handle cache flush jobs

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/decode: Fix image attribute counting

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Advertise GLES3.1

We have CI, we're just a few tests away from conformance on v7, and
Midgard is just a few hundred tests behind. Given the branch point isn't
for another month, I think this is a good time to flip the switch.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Add "Cache Flush" job XML

Likely useful for efficient memory_barrier and texture_barrier
operations.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Set vertex job_barrier

Fixes KHR-GLES31.core.vertex_attrib_binding.advanced-iterations which
pingpongs XFB/attributes

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Flush before compute jobs

Suboptimal but fixes KHR-GLES31.core.compute_shader.pipeline-post-xfb,
which is stubbornly still broken with memory barriers implemented and
cache flush jobs inserted. More investigation needed but probably not
right now.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Flush everything for glMemoryBarrier

This is inefficient but so far I see the DDK doing the same thing. Fixes
KHR-GLES31.core.shader_storage_buffer_object.advanced-usage-sync-vsfs

In the future we should look into cache flush jobs.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Clean up vertex/instance ID on Midgard

Use the proper XML.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Add XML for vertex/instance ID records

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Set valid_buffer_range for GPU writes

Transform feedback, SSBO writes, and image writes in particular can
affect this and have bad interactions. Fixes
KHR-GLES31.core.shader_atomic_counters.basic-usage-vs

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Remove pan_image_state

Instead just group the fields about validity into a simpler structure in
panfrost_resource. Panvk can do the same. Common code shouldn't be
thinking in terms of this 'larger' structure anyway.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Make data_valid a bitset

More compact and will allow simpler code.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Don't clobber indirect dispatch fields

These should be kept as zero so they can be packed correctly. Fixes a
number of KHR-GLES31 fails.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Use direct dispatch with shared memory

This would require memory allocations we don't handle.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/indirect_dispatch: Use extracted values

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/indirect_dispatch: Expand split expressions

Careful algebraic transforms makes these much simpler.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/indirect_dispatch: Distinguish minus-1 defs

This makes the logic clearer and allows the original values to be
accessed.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/indirect_dispatch: Simplify empty command case

Job type is alone with bitsize in the bottom byte of the addressed
worse, so if we use an 8-bit store we avoid the RMW complexity.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/indirect_dispatch: Indent NIR blocks

Easier to visualize the control flow this way.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Reduce pan_image_state indirection

In actuality, this just shadows the crc_valid for pan_cs... the
data_valid checks are contained in the caller and just add noise.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Don't CRC mipmapped textures

CRC is intended for final render targets and especially for UI, not the
kind of things you'd mipmap. Meanwhile CRC only works for a single
level, meaning at any given point, half the CRC buffer would be wasted
for a full miptree.

"Arm Mali Best Practices Guide" tells developers that the DDK only
enables CRC for non-mipmapped resources (at least the Vulkan DDK), so
let's do the same, save some memory, and simplify our code.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Drop todo on PIPE_COMPUTE_CAP_IMAGES_SUPPORTED

They work fine.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Set PIPE_COMPUTE_CAP_MAX_THREADS_PER_BLOCK

Fixes KHR-GLES31.core.gpu_shader5.images_array_indexing

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Set PIPE_COMPUTE_CAP_SUBGROUP_SIZE

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Lower max compute size

Match the DDK's limit (Mali G52), I think there's undocumented errata
here. Fixes
KHR-GLES31.core.texture_buffer.texture_buffer_operations_image_store

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Make image buffers robust

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Fix BUFFER image handling

Fixes:

   KHR-GLES31.core.shader_image_load_store.advanced-allMips-cs
   KHR-GLES31.core.shader_image_load_store.advanced-allMips-fs

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Allocate XFB buffers per-instance

Somehow XFB gets so little use we never noticed. Fixes:

   KHR-GLES31.core.vertex_attrib_binding.basic-input-case9
   KHR-GLES31.core.vertex_attrib_binding.basic-input-case11
   KHR-GLES31.core.vertex_attrib_binding.basic-inputI-case2

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Don't set a blend shader for no_colour

It's pointless and confuses the hardware. Fixes (on Bifrost)

KHR-GLES31.core.draw_buffers_indexed.color_masks

Yes, this is a silly edge case. Yes, we still have to handle it
correctly.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Remove scissor_culls_everything

Based on a misunderstanding of how the scissor test works, and in
particular breaks transform feedback and SSBO writes from vertex
shaders.

Replace it with a moral equivalent to rasterizer_discard so vertex
shaders still run.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Add some missing BGRA formats

Fixes:

KHR-GLES3.copy_tex_image_conversions.forbidden.*
KHR-GLES3.packed_pixels.pbo_rectangle.rgb5_a1
KHR-GLES3.packed_pixels.pbo_rectangle.rgba
KHR-GLES3.packed_pixels.pbo_rectangle.rgba4
KHR-GLES3.packed_pixels.pbo_rectangle.rgba8
KHR-GLES3.packed_pixels.rectangle.rgb5_a1
KHR-GLES3.packed_pixels.rectangle.rgba
KHR-GLES3.packed_pixels.rectangle.rgba4
KHR-GLES3.packed_pixels.rectangle.rgba8
KHR-GLES3.packed_pixels.varied_rectangle.rgb5_a1
KHR-GLES3.packed_pixels.varied_rectangle.rgba
KHR-GLES3.packed_pixels.varied_rectangle.rgba4
KHR-GLES3.packed_pixels.varied_rectangle.rgba8

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Emulate indirect draws on Midgard

I can't really justify spending time on this right now, even to myself.
So take the perf hit and get out checkbox.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Fix dirty state emission

If we have per-draw state (vertex ID stuff), there's an ordering
mismatch. Fixes
dEQP-GLES31.functional.draw_base_vertex.draw_elements_instanced_base_vertex.builtin_variable.vertex_id
on Midgard, and I'm not sure why it was passing on Bifrost before. Also
should fix (on both architectures) DRAWID issues.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/mdg: Insert moves to load/store registers

Ensures a valid schedule/regalloc is possible when vectors are used in
funny ways, as occurs in dEQP-GLES31 resulting in a scheduler hang (or
with prior patches, an RA failure).

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/mdg: Assert scheduled instructions are reasonable

Would've got a scheduler hang.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/mdg: Don't skip unit-based checks in choose_instruction

If an explicit unit isn't specified, we still should check.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/mdg: Use more accurate ld/st reg estimates

And assert that we got them right.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/mdg: Lower away gl_VertexID offset

Technically we can stick the offset in the vertex ID attribute record,
but this is a faster way to get the test passing and Midgard perf?
what's that?

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/mdg: Wire in PAN_SYSVAL_VERTEX_INSTANCE_OFFSETS

If we're going to advertise the CAP, better not crash..

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Don't allocate WLS when not needed

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Mark 16/32_UNORM as non-renderable (v5)

You'd just get a blend shader anyway, and since they're not spec
requirements, let's not worry about backporting the Midgard lowerings.

Takes dEQP-GLES31.functional.fbo.color.tex2d.* on Midgard from crashing
to not supported.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/mdg: Fix incorrect rewrite in Midgard scheduler

Fixes on Midgard
dEQP-GLES31.functional.shaders.builtin_functions.uniform.findLSBMinusOne.highp_fragment

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/mdg: Update r1.w comment

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/mdg: Handle {i,u}{add,sub}_sat

As SATADD with different modifiers.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/mdg: Fix units for SUBSAT

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Respect early-Z force on Midgard

Fixes dEQP-GLES31.functional.image_load_store.early_fragment_tests.* on
Midgard.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Don't force early-z with occlusion query

..even if there is no z/s enabled. Fixes
dEQP-GLES31.functional.fbo.no_attachments.* on Midgard.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Simplify Midgard blend disable

Probably a bit faster too.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Clarify how fs_sidefx works with oq

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/mdg: Stub memory_barrier{_image}

Same as we do for Bifrost.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/mdg: Make -Wswitch happy

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/mdg: Use consistent casing in midgard_print

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Assert alignment of indirect records

Continuation records need alignment, this shows they already have it.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Make instancing code more obvious

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Fix src_offset data type

We treat it as signed but had it marked as unsigned. It can be negative
in obscure cases.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Align NPOT divisor records

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Add util_draw_indirect() debug path

Useful for finding problems with the GPU indirect path.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Zero r_dimension for buffer textures

Instead of reading wrong side of the union (undefined behaviour). Fixes
a GenXML assertion failure in
KHR-GLES31.core.texture_buffer.texture_buffer_texture_buffer_range

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Fix crc_valid condition

Fixes fails in dEQP-GLES31.functional.texture.border_clamp.* when run in
parallel with certain other tests.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Simplify compute_checksum_size formula

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Fix vertex image attribute overrun

Images take a continuation record, don't scribble zeroes over.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Fixes: dc85f65e059 ("panfrost: emit shader image attribute descriptors")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/bi: Force u32 for flat varyings

Since the GLSL compilers will pack together flat varyings with no regard
to type, under the assumption the backend can deal with it. I guess we
can deal with it then... Fixes fails in
dEQP-GLES31.functional.separate_shader.random.*

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Use varying format from frag shader

Needed to fix up flat varyings to u32 due to TGSI brokenness. If we wack
TGSI, we can drop this.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Correctly size varyings

The same slot could be specified multiple times with different
location_frac out of order, so we use two passes.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/indirect_draw: Fix 1 instance, nonzero divisor

Instead of doing a complicated hack with the POT divisor, just zero the
stride of the linear attribute buffer like we do on the CPU.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/indirect_draw: Use unsigned comparisons

Instead of signed -- get the types right.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/indirect: Factor out is_power_of_two_or_zero

The function is complicated enough as it is -- hide the bit twiddling
behind a helper function.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Default indirect attributes to 1D type

Avoids some complexity in the indirect draw happy path.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Use util_last_bit for images

Probbaly more correct for hols in image_mask.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Be explicit in image modifier handling

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Separate image attribute and buffer emit

Trying to disentangle attributes and attribute buffers, so here's
a leaf node for that change.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Don't duplicate attribute buffers

If the (vbi, divisor) tuple matches, we can save an attribute buffer
descriptor. We do the linking at CSO create time. This should be a bit
more cache friendly.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Disable AFBC on v7

Broken in several ways. Hide it until we can get this sorted, and have a
test plan to keep it sorted.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Cc: mesa-stable
Acked-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Add missing 'Reverse issue order flag'

Should fix an issue I'm seeing. Spoiler alert, it does not.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Acked-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

panfrost: Remove AFBC format fixups

It's too complicated and probably for no actual benefit. The main reason
we have BGR formats is for display, but that's export and doesn't get
hit by this path. Internal BGRA textures are possible with a Mesa
extension but sufficiently rare that I regret suggesting this as a
possible optimization. My apologies, and thanks for the fish.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Acked-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/bi: Don't allocate past the end of the reg file

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/bi: Track words instead of bytes in RA

Reduces RA memory footprint by 4x, fixing an OOM in the following dEQP
test that otherwise would allocate 8GB of memory...

   dEQP-GLES31.functional.ssbo.layout.random.all_shared_buffer.36

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>

pan/bi: Simplify spill code

Now allow spilling all nodes. Fixes failed spilling in

dEQP-GLES31.functional.ssbo.layout.random.all_shared_buffer.21

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11123>