utils.c: Set DEVIATION to 0.0128

Consider a HARD_LIGHT operation with the following pixels:

- source:           15      (6 bits)
- source alpha:     255     (8 bits)
- mask alpha:       223     (8 bits)
- dest              255     (8 bits)
- dest alpha:       0       (8 bits)

Since 2 times the source is less than source alpha, the first branch
of the hard light blend mode is taken:

        (1 - sa) * d + (1 - da) * s + 2 * s * d

Since da is 0 and d is 1, this degenerates to:

        (1 - sa) + 3 * s

Taking (src IN mask) into account along with the fact that sa is 1,
this becomes:

        (1 - ma) + 3 * s * ma

      = (1 - 223/255.0) + 3 * (15/63.0) * (223/255.0)

      = 0.7501400560224089

When computed with the source converted by bit replication to eight
bits, and additionally with the (src IN mask) part rounded to eight
bits, we get:

        ma = 223/255.0

        s * ma = (60 / 255.0) * (223/255.0) which rounds to 52 / 255

and the result is

        (1 - ma) + 3 * s * ma

      = (1 - 223/255.0) + 3 * 52/255.0

      = 0.7372549019607844

so now we have an error of 0.012885.

Without making changes to the way pixman does integer
rounding/arithmetic, this error must then be considered
acceptable. Due to conservative computations in the test suite we can
however get away with 0.0128 as the acceptable deviation.

This fixes the remaining failures in pixel-test.

diff --git a/test/utils.c b/test/utils.c
index d182710378980a5eb7039a8bb36fbde3054109e7..188841783e54c59ea9ac46b45a9179c3903ec485 100644 (file)
--- a/test/utils.c
+++ b/test/utils.c
@@ -1754,7 +1754,7 @@ get_limits (const pixel_checker_t *checker, double limit,
 
 /* The acceptable deviation in units of [0.0, 1.0]
  */
-#define DEVIATION (0.0064)
+#define DEVIATION (0.0128)
 
 void
 pixel_checker_get_max (const pixel_checker_t *checker, color_t *color,