tgsi_exec: Fix NaN behavior of saturate

Modern shader APIs, like DX10 and GLSL 1.30, want saturate or
clamp(..., 0.0, 1.0) to "cleanse" NaN.  If the source is NaN, the
result should be zero.

There are many cases where TGSI is generate from NIR, and many
optimizations in NIR expect this behavior.  Not meeting these
expectations can lead to unexpected results.

Reviewed-by: Eric Anholt <eric@anholt.net>
Fixes: 56c30bf17b9 ("tgsi: Saturate modifier obeys ExecMask. Implement NVIDIA [-1;+1] saturate mode.")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10419>

diff --git a/src/gallium/auxiliary/tgsi/tgsi_exec.c b/src/gallium/auxiliary/tgsi/tgsi_exec.c
index 008d8c0..b4c6f60 100644 (file)
--- a/src/gallium/auxiliary/tgsi/tgsi_exec.c
+++ b/src/gallium/auxiliary/tgsi/tgsi_exec.c
@@ -1938,7 +1938,7 @@ store_dest(struct tgsi_exec_machine *mach,
    else {
       for (i = 0; i < TGSI_QUAD_SIZE; i++)
          if (execmask & (1 << i)) {
-            if (chan->f[i] < 0.0f)
+            if (chan->f[i] < 0.0f || isnan(chan->f[i]))
                dst->f[i] = 0.0f;
             else if (chan->f[i] > 1.0f)
                dst->f[i] = 1.0f;
@@ -3621,7 +3621,7 @@ store_double_channel(struct tgsi_exec_machine *mach,
    else {
       for (i = 0; i < TGSI_QUAD_SIZE; i++)
          if (execmask & (1 << i)) {
-            if (chan->d[i] < 0.0)
+            if (chan->d[i] < 0.0 || isnan(chan->d[i]))
                temp.d[i] = 0.0;
             else if (chan->d[i] > 1.0)
                temp.d[i] = 1.0;