c-family: Fix up -Wsign-compare BIT_NOT_EXPR handling [PR107465]

The following patch fixes multiple bugs in warn_for_sign_compare related to
the BIT_NOT_EXPR related warnings.
My understanding is that what those 3 warnings are meant to warn (since 1995
apparently) is the case where we have BIT_NOT_EXPR of a zero-extended
value, so in result_type the value is something like:
0b11111111XXXXXXXX (e.g. ~ of a 8->16 bit zero extension)
0b000000000000000011111111XXXXXXXX (e.g. ~ of a 8->16 bit zero extension
then zero extended to 32 bits)
0b111111111111111111111111XXXXXXXX (e.g. ~ of a 8->16 bit zero extension
then sign extended to 32 bits)
and the intention of the warning is to warn when this is compared against
something that has some 0 bits at the place where the above has guaranteed
1 bits, either ensured through comparison against constant where we know
the bits exactly, or through zero extension from some narrower type where
again we know at least some upper bits are zero extended.
The bugs in the warning code are:
1) misunderstanding of the {,c_common_}get_narrower APIs - the unsignedp
   it sets is only meaningful if the function actually returns something
   narrower (in that case it says whether the narrower value is then
   sign (0) or zero (1) extended to the originally passed value.
   Though op0 or op1 at this point might be already narrower than
   result_type, and if the function doesn't return anything narrower,
   it all depends on whether the passed in op{0,1} had TYPE_UNSIGNED
   type or not
2) the code didn't check at all whether the BIT_NOT_EXPR operand
   was actually zero extended (i.e. that it was narrower and unsignedp
   was set to 1 for it), all it did is check that unsignedp from the
   call was 1.  But that isn't well defined thing, if the argument
   is returned as is, the function sets unsignedp to 0, but if there
   is e.g. a useless cast to the same or compatible type in between,
   it can return 1 if the cast is unsigned; now, if BIT_NOT_EXPR
   operand is not zero extended, we know nothing at all about any bits
   in the operand containing BIT_NOT_EXPR, so there is nothing to warn
   about
3) the code was actually testing both operands after calling
   c_common_get_narrower on them and on the one with BIT_NOT_EXPR
   again for constants; I think that is just wrong in case the BIT_NOT_EXPR
   operand wouldn't be fully folded, the warning makes sense only if the
   other operand not having BIT_NOT_EXPR in it is constant
4) as can be seen from the above bit pattern examples, the upper bits above
   (in the patch arg0) aren't always all 1s, there could be some zero extension
   above it and from it one would have 0s, so that needs to be taken into
   account for the choice which constant bits to test for being always set
   otherwise warning is emitted, or for the zero extension guaranteed zero
   bits
5) the patch also simplifies the handling, we only do it if one but not
   both operands are BIT_NOT_EXPR after first {,c_common_}get_narrower,
   so we can just use std::swap to ensure it is the first one
6) the code compared bits against HOST_BITS_PER_LONG, which made sense
   back in 1995 when the values were stored into long, but now that they
   are HOST_WIDE_INT should test HOST_BITS_PER_WIDE_INT (or we could rewrite
   the stuff to wide_int, not done in the patch)

2023-03-04  Jakub Jelinek  <jakub@redhat.com>

	PR c/107465
	* c-warn.cc (warn_for_sign_compare): If c_common_get_narrower
	doesn't return a narrower result, use TYPE_UNSIGNED to set unsignedp0
	and unsignedp1.  For the one BIT_NOT_EXPR case vs. one without,
	only check for constant in the non-BIT_NOT_EXPR operand, use std::swap
	to simplify the code, only warn if BIT_NOT_EXPR operand is extended
	from narrower unsigned, fix up computation of mask for the constant
	cases and for unsigned other operand case handle differently
	BIT_NOT_EXPR result being sign vs. zero extended.

	* c-c++-common/Wsign-compare-2.c: New test.
	* c-c++-common/pr107465.c: New test.

diff --git a/gcc/c-family/c-warn.cc b/gcc/c-family/c-warn.cc
index 2941cf4c2f9d31f1704411d229273f26dae0f095..70d9bb20097fc5758ad5a3bccedabe986cf874b3 100644 (file)
--- a/gcc/c-family/c-warn.cc
+++ b/gcc/c-family/c-warn.cc
@@ -2345,42 +2345,50 @@ warn_for_sign_compare (location_t location,
      have all bits set that are set in the ~ operand when it is
      extended.  */
 
-  op0 = c_common_get_narrower (op0, &unsignedp0);
-  op1 = c_common_get_narrower (op1, &unsignedp1);
+  tree arg0 = c_common_get_narrower (op0, &unsignedp0);
+  if (TYPE_PRECISION (TREE_TYPE (arg0)) == TYPE_PRECISION (TREE_TYPE (op0)))
+    unsignedp0 = TYPE_UNSIGNED (TREE_TYPE (op0));
+  op0 = arg0;
+  tree arg1 = c_common_get_narrower (op1, &unsignedp1);
+  if (TYPE_PRECISION (TREE_TYPE (arg1)) == TYPE_PRECISION (TREE_TYPE (op1)))
+    unsignedp1 = TYPE_UNSIGNED (TREE_TYPE (op1));
+  op1 = arg1;
 
   if ((TREE_CODE (op0) == BIT_NOT_EXPR)
       ^ (TREE_CODE (op1) == BIT_NOT_EXPR))
     {
-      if (TREE_CODE (op0) == BIT_NOT_EXPR)
-       op0 = c_common_get_narrower (TREE_OPERAND (op0, 0), &unsignedp0);
       if (TREE_CODE (op1) == BIT_NOT_EXPR)
-       op1 = c_common_get_narrower (TREE_OPERAND (op1, 0), &unsignedp1);
-
-      if (tree_fits_shwi_p (op0) || tree_fits_shwi_p (op1))
        {
-         tree primop;
-         HOST_WIDE_INT constant, mask;
-         int unsignedp;
-         unsigned int bits;
+         std::swap (op0, op1);
+         std::swap (unsignedp0, unsignedp1);
+       }
 
-         if (tree_fits_shwi_p (op0))
-           {
-             primop = op1;
-             unsignedp = unsignedp1;
-             constant = tree_to_shwi (op0);
-           }
-         else
-           {
-             primop = op0;
-             unsignedp = unsignedp0;
-             constant = tree_to_shwi (op1);
-           }
+      int unsignedp;
+      arg0 = c_common_get_narrower (TREE_OPERAND (op0, 0), &unsignedp);
 
-         bits = TYPE_PRECISION (TREE_TYPE (primop));
-         if (bits < TYPE_PRECISION (result_type)
-             && bits < HOST_BITS_PER_LONG && unsignedp)
+      /* For these warnings, we need BIT_NOT_EXPR operand to be
+        zero extended from narrower type to BIT_NOT_EXPR's type.
+        In that case, all those bits above the narrower's type
+        are after BIT_NOT_EXPR set to 1.  */
+      if (tree_fits_shwi_p (op1))
+       {
+         HOST_WIDE_INT constant = tree_to_shwi (op1);
+         unsigned int bits = TYPE_PRECISION (TREE_TYPE (arg0));
+         if (unsignedp
+             && bits < TYPE_PRECISION (TREE_TYPE (op0))
+             && bits < HOST_BITS_PER_WIDE_INT)
            {
-             mask = HOST_WIDE_INT_M1U << bits;
+             HOST_WIDE_INT mask = HOST_WIDE_INT_M1U << bits;
+             if (unsignedp0)
+               {
+                 bits = TYPE_PRECISION (TREE_TYPE (op0));
+                 if (bits < TYPE_PRECISION (result_type)
+                     && bits < HOST_BITS_PER_WIDE_INT)
+                   mask &= ~(HOST_WIDE_INT_M1U << bits);
+               }
+             bits = TYPE_PRECISION (result_type);
+             if (bits < HOST_BITS_PER_WIDE_INT)
+               mask &= ~(HOST_WIDE_INT_M1U << bits);
              if ((mask & constant) != mask)
                {
                  if (constant == 0)
@@ -2394,11 +2402,28 @@ warn_for_sign_compare (location_t location,
                }
            }
        }
-      else if (unsignedp0 && unsignedp1
-              && (TYPE_PRECISION (TREE_TYPE (op0))
-                  < TYPE_PRECISION (result_type))
+      else if ((TYPE_PRECISION (TREE_TYPE (arg0))
+               < TYPE_PRECISION (TREE_TYPE (op0)))
+              && unsignedp
+              && unsignedp1
+              /* If unsignedp0, the BIT_NOT_EXPR result is
+                 zero extended, so say if op0 is unsigned char
+                 variable, BIT_NOT_EXPR is unsigned short and
+                 result type int and op0 has value 0x55, the
+                 int value will be 0xffaa, or for op0 0xaa it
+                 will be 0xff55.  In these cases, warn if
+                 op1 is unsigned and narrower than unsigned short.
+                 While if unsignedp0 is false, the BIT_NOT_EXPR
+                 result is sign extended and because of the
+                 above TYPE_PRECISION comparison we know the
+                 MSB of BIT_NOT_EXPR is set (perhaps with some
+                 further bits below it).  The sign extension will
+                 then ensure all bits above BIT_NOT_EXPR up to
+                 result_type's precision are set.  */
               && (TYPE_PRECISION (TREE_TYPE (op1))
-                  < TYPE_PRECISION (result_type)))
+                  < TYPE_PRECISION (unsignedp0
+                                    ? TREE_TYPE (op0)
+                                    : result_type)))
        warning_at (location, OPT_Wsign_compare,
                    "comparison of promoted bitwise complement "
                    "of an unsigned value with unsigned");

diff --git a/gcc/testsuite/c-c++-common/Wsign-compare-2.c b/gcc/testsuite/c-c++-common/Wsign-compare-2.c
new file mode 100644 (file)
index 0000000..23933d0
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/Wsign-compare-2.c
@@ -0,0 +1,106 @@
+/* { dg-do compile { target { ilp32 || lp64 } } } */
+/* { dg-options "-Wsign-compare" } */
+
+int
+f1 (unsigned char x)
+{
+  return (unsigned short) (~(unsigned short) x) == 0;          /* { dg-warning "promoted bitwise complement of an unsigned value is always nonzero" "" { target c } } */
+}
+
+int
+f2 (unsigned char x)
+{
+  return (unsigned short) (~(unsigned short) x) == 5;          /* { dg-warning "comparison of promoted bitwise complement of an unsigned value with constant" "" { target c }  } */
+}
+
+int
+f3 (unsigned char x)
+{
+  return (unsigned int) (~(unsigned short) x) == 0xffff0005U;  /* { dg-warning "comparison of promoted bitwise complement of an unsigned value with constant" } */
+}
+
+int
+f4 (unsigned char x)
+{
+  return (unsigned int) (~(unsigned short) x) == 0xffff0005ULL;        /* { dg-warning "comparison of promoted bitwise complement of an unsigned value with constant" } */
+}
+
+int
+f5 (unsigned char x)
+{
+  return (unsigned int) (~(unsigned short) x) == 0xffffff05U;  /* { dg-bogus "comparison of promoted bitwise complement of an unsigned value with constant" } */
+}
+
+int
+f6 (unsigned char x)
+{
+  return (unsigned int) (~(unsigned short) x) == 0xffffff05ULL;        /* { dg-bogus "comparison of promoted bitwise complement of an unsigned value with constant" } */
+}
+
+int
+f7 (unsigned char x)
+{
+  return (unsigned long long) (~(unsigned short) x) == 0xffffffffffffff05ULL;  /* { dg-bogus "comparison of promoted bitwise complement of an unsigned value with constant" } */
+}
+
+typedef unsigned short T;
+
+int
+f8 (T x)
+{
+  return (unsigned short) (~(unsigned short) x) == 0;          /* { dg-bogus "promoted bitwise complement of an unsigned value is always nonzero" } */
+}
+
+int
+f9 (T x)
+{
+  return (unsigned short) (~(unsigned short) x) == 5;          /* { dg-bogus "comparison of promoted bitwise complement of an unsigned value with constant" } */
+}
+
+int
+f10 (T x, unsigned char y)
+{
+  return (unsigned short) (~(unsigned short) x) == y;          /* { dg-bogus "comparison of promoted bitwise complement of an unsigned value with unsigned" } */
+}
+
+int
+f11 (T x, unsigned char y)
+{
+  return (unsigned short) (~(unsigned short) x) == y;          /* { dg-bogus "comparison of promoted bitwise complement of an unsigned value with unsigned" } */
+}
+
+int
+f12 (unsigned char x, unsigned char y)
+{
+  return (unsigned short) (~(unsigned short) x) == y;          /* { dg-warning "comparison of promoted bitwise complement of an unsigned value with unsigned" "" { target c } } */
+}
+
+int
+f13 (unsigned char x, unsigned char y)
+{
+  return (unsigned short) (~(unsigned short) x) == y;          /* { dg-warning "comparison of promoted bitwise complement of an unsigned value with unsigned" "" { target c } } */
+}
+
+int
+f14 (unsigned char x, unsigned int y)
+{
+  return (unsigned long long) (~x) == y;                       /* { dg-warning "comparison of promoted bitwise complement of an unsigned value with unsigned" } */
+}
+
+int
+f15 (unsigned short x, unsigned int y)
+{
+  return (long long) (~x) == y;                                        /* { dg-warning "comparison of promoted bitwise complement of an unsigned value with unsigned" } */
+}
+
+int
+f16 (unsigned char x, unsigned short y)
+{
+  return (unsigned short) (~(unsigned short) x) == y;          /* { dg-bogus "comparison of promoted bitwise complement of an unsigned value with unsigned" } */
+}
+
+int
+f17 (unsigned char x, unsigned short y)
+{
+  return (unsigned short) (~(unsigned short) x) == y;          /* { dg-bogus "comparison of promoted bitwise complement of an unsigned value with unsigned" } */
+}

diff --git a/gcc/testsuite/c-c++-common/pr107465.c b/gcc/testsuite/c-c++-common/pr107465.c
new file mode 100644 (file)
index 0000000..7c9ff24
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/pr107465.c
@@ -0,0 +1,22 @@
+/* PR c/107465 */
+/* { dg-do compile } */
+/* { dg-options "-Wsign-compare" } */
+
+void baz (void);
+typedef unsigned short T;
+
+#if __SIZEOF_SHORT__ * __CHAR_BIT__ == 16
+void
+foo (unsigned short x)
+{
+  if (!(x ^ 0xFFFF))
+    baz ();
+}
+
+void
+bar (T x)
+{
+  if (!(x ^ 0xFFFF))   /* { dg-bogus "promoted bitwise complement of an unsigned value is always nonzero" } */
+    baz ();
+}
+#endif