Fix for infinite-loop test
authorSøren Sandmann Pedersen <ssp@redhat.com>
Sat, 15 Sep 2012 07:13:09 +0000 (03:13 -0400)
committerSøren Sandmann Pedersen <ssp@redhat.com>
Mon, 24 Sep 2012 22:43:31 +0000 (18:43 -0400)
The infinite loop detected by "affine-test 212944861" is caused by an
overflow in this expression:

    max_x = pixman_fixed_to_int (vx + (width - 1) * unit_x) + 1;

where (width - 1) * unit_x doesn't fit in a signed int. This causes
max_x to be too small so that this:

    src_width = 0

    while (src_width < REPEAT_NORMAL_MIN_WIDTH && src_width <= max_x)
        src_width += src_image->bits.width;

results in src_width being 0. Later on when src_width is used for
repeat calculations, we get the infinite loop.

By casting unit_x to int64_t, the expression no longer overflows and
affine-test 212944861 and infinite-loop no longer loop forever.

pixman/pixman-inlines.h

index 5517de5..3a3c658 100644 (file)
@@ -859,7 +859,7 @@ fast_composite_scaled_bilinear ## scale_func_name (pixman_implementation_t *imp,
     {                                                                                          \
        vx = v.vector[0];                                                                       \
        repeat (PIXMAN_REPEAT_NORMAL, &vx, pixman_int_to_fixed(src_image->bits.width));         \
-       max_x = pixman_fixed_to_int (vx + (width - 1) * unit_x) + 1;                            \
+       max_x = pixman_fixed_to_int (vx + (width - 1) * (int64_t)unit_x) + 1;                   \
                                                                                                \
        if (src_image->bits.width < REPEAT_NORMAL_MIN_WIDTH)                                    \
        {                                                                                       \