pan/bi: Add dummy carry/borrow argument for iadd/isub

On FMA, a carry/borrow is required for iaddc/isubb (whereas the ADD
counterparts don't support carrying/borrowing). The trick is to model
this with an extra dummy (ZERO) argument which is free to encode on FMA,
and in the scheduler, "demote" to the non-carried versions if we want to
schedule to ADD.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6749>

diff --git a/src/panfrost/bifrost/bifrost_compile.c b/src/panfrost/bifrost/bifrost_compile.c
index 413c8d3..d4acad7 100644 (file)
--- a/src/panfrost/bifrost/bifrost_compile.c
+++ b/src/panfrost/bifrost/bifrost_compile.c
@@ -805,9 +805,13 @@ emit_alu(bi_context *ctx, nir_alu_instr *instr)
                 break;
         case nir_op_iadd:
                 alu.op.imath = BI_IMATH_ADD;
+                /* Carry */
+                alu.src[2] = BIR_INDEX_ZERO;
                 break;
         case nir_op_isub:
                 alu.op.imath = BI_IMATH_SUB;
+                /* Borrow */
+                alu.src[2] = BIR_INDEX_ZERO;
                 break;
         case nir_op_iabs:
                 alu.op.special = BI_SPECIAL_IABS;

diff --git a/src/panfrost/bifrost/test/bi_test_pack.c b/src/panfrost/bifrost/test/bi_test_pack.c
index 9a9f991..a84eed4 100644 (file)
--- a/src/panfrost/bifrost/test/bi_test_pack.c
+++ b/src/panfrost/bifrost/test/bi_test_pack.c
@@ -557,6 +557,7 @@ bit_imath_helper(struct panfrost_device *dev, uint32_t *input, unsigned size, en
 {
         bi_instruction ins = bit_ins(BI_IMATH, 2, nir_type_uint, size);
         bit_swizzle_identity(&ins, 2, size);
+        ins.src[2] = BIR_INDEX_ZERO; /* carry/borrow for FMA */
 
         for (unsigned op = BI_IMATH_ADD; op <= BI_IMATH_SUB; ++op) {
                 ins.op.imath = op;