math-emu: fix floating-point to integer unsigned saturation
authorJoseph Myers <joseph@codesourcery.com>
Mon, 4 Nov 2013 16:53:50 +0000 (16:53 +0000)
committerScott Wood <scottwood@freescale.com>
Wed, 8 Jan 2014 00:34:47 +0000 (18:34 -0600)
The math-emu macros _FP_TO_INT and _FP_TO_INT_ROUND are supposed to
saturate their results for out-of-range arguments, except in the case
rsigned == 2 (when instead the low bits of the result are taken).
However, in the case rsigned == 0 (converting to unsigned integers),
they mistakenly produce 0 for positive results and the maximum
unsigned integer for negative results, the opposite of correct
unsigned saturation.  This patch fixes the logic.

Signed-off-by: Joseph Myers <joseph@codesourcery.com>
Signed-off-by: Scott Wood <scottwood@freescale.com>
include/math-emu/op-common.h

index 9696a5e..70fe5e9 100644 (file)
@@ -685,7 +685,7 @@ do {                                                                        \
            else                                                                \
              {                                                                 \
                r = 0;                                                          \
-               if (X##_s)                                                      \
+               if (!X##_s)                                                     \
                  r = ~r;                                                       \
              }                                                                 \
            FP_SET_EXCEPTION(FP_EX_INVALID);                                    \
@@ -762,7 +762,7 @@ do {                                                                        \
            if (!rsigned)                                                       \
              {                                                                 \
                r = 0;                                                          \
-               if (X##_s)                                                      \
+               if (!X##_s)                                                     \
                  r = ~r;                                                       \
              }                                                                 \
            else if (rsigned != 2)                                              \