gallium/util: fix returning empty box for rectangle intersection
authorRoland Scheidegger <sroland@vmware.com>
Tue, 1 Sep 2015 22:29:17 +0000 (16:29 -0600)
committerBrian Paul <brianp@vmware.com>
Tue, 1 Sep 2015 22:29:17 +0000 (16:29 -0600)
These functions deal with inclusive coordinates, hence a 0/0/0/0 rect
returned when there's no intersection doesn't actually represent an empty
rectangle. Hence return 0/-1/0/-1 instead.
This fixes some problems in llvmpipe with empty scissor rects (which up
to now didn't really matter because while the intersect test returned the
wrong result all pixels were scissored away later anyway).

src/gallium/auxiliary/util/u_rect.h

index dea1e1e..221d918 100644 (file)
@@ -87,7 +87,12 @@ u_rect_possible_intersection(const struct u_rect *a,
       u_rect_find_intersection(a,b);
    }
    else {
-      b->x0 = b->x1 = b->y0 = b->y1 = 0;
+      /*
+       * Note the u_rect_xx tests deal with inclusive coordinates
+       * hence all-zero would not be an empty box.
+       */
+      b->x0 = b->y0 = 0;
+      b->x1 = b->y1 = -1;
    }
 }