swrast: fix readpix clamping

Broken with e5c6a92a12b5cd7db205d72039f58d302b0be9d5. (ARB_color_buffer_float)

Clamping should occur if type != float, otherwise the MSBs of the resulting
pixels are killed off. For example, reading back LUMINANCE = R+G+B can be
greater than 0xff, but the result is naturally masked by 0xff
for UNSIGNED_BYTE, leading to bogus results.

The following bug report seems to want clamping to occur if type == half_float
too. Not sure what's correct.

Bug: [bisected pineview] oglc case pxconv-read failed
https://bugs.freedesktop.org/show_bug.cgi?id=35852

Tested by: Fang Xun <xunx.fang@intel.com>
Reviewed-and-tested-by: Ian Romanick <ian.d.romanick@intel.com>

diff --git a/src/mesa/swrast/s_readpix.c b/src/mesa/swrast/s_readpix.c
index a201a63..214f2ea 100644 (file)
--- a/src/mesa/swrast/s_readpix.c
+++ b/src/mesa/swrast/s_readpix.c
@@ -318,12 +318,12 @@ read_rgba_pixels( struct gl_context *ctx,
    if (!rb)
       return;
 
-   if (type == GL_FLOAT && ((ctx->Color.ClampReadColor == GL_TRUE) ||
-                            (ctx->Color.ClampReadColor == GL_FIXED_ONLY_ARB &&
-                             rb->DataType != GL_FLOAT)))
+   if ((ctx->Color._ClampReadColor == GL_TRUE || type != GL_FLOAT) &&
+       !_mesa_is_integer_format(format)) {
       transferOps |= IMAGE_CLAMP_BIT;
+   }
 
-   /* Try optimized path first */
+   /* Try the optimized path first. */
    if (fast_read_rgba_pixels(ctx, x, y, width, height,
                              format, type, pixels, packing, transferOps)) {
       return; /* done! */