From 95ac5635409606386259d2ff21fb61738858ca4a Mon Sep 17 00:00:00 2001 From: Jakub Jelinek Date: Tue, 1 Feb 2022 20:48:03 +0100 Subject: [PATCH] libcpp: Fix up padding handling in funlike_invocation_p [PR104147] As mentioned in the PR, in some cases we preprocess incorrectly when we encounter an identifier which is defined as function-like macro, followed by at least 2 CPP_PADDING tokens and then some other identifier. On the following testcase, the problem is in the 3rd funlike_invocation_p, the tokens are CPP_NAME Y, CPP_PADDING (the pfile->avoid_paste shared token), CPP_PADDING (one created with padding_token, val.source is non-NULL and val.source->flags & PREV_WHITE is non-zero) and then another CPP_NAME. funlike_invocation_p remembers there was a padding token, but remembers the first one because of its condition, then the next token is the CPP_NAME, which is not CPP_OPEN_PAREN, so the CPP_NAME token is backed up, but as we can't easily backup more tokens, it pushes into a new context the padding token (the pfile->avoid_paste one). The net effect is that when Y is not defined as fun-like macro, we read Y, avoid_paste, padding_token, Y, while if Y is fun-like macro, we read Y, avoid_paste, avoid_paste, Y (the second avoid_paste is because that is how we handle end of a context). Now, for stringify_arg that is unfortunately a significant difference, which handles CPP_PADDING tokens with: if (token->type == CPP_PADDING) { if (source == NULL || (!(source->flags & PREV_WHITE) && token->val.source == NULL)) source = token->val.source; continue; } and later on /* Leading white space? */ if (dest - 1 != BUFF_FRONT (pfile->u_buff)) { if (source == NULL) source = token; if (source->flags & PREV_WHITE) *dest++ = ' '; } source = NULL; (and c-ppoutput.cc has similar code). So, when Y is not fun-like macro, ' ' is added because padding_token's val.source->flags & PREV_WHITE is non-zero, while when it is fun-like macro, we don't add ' ' in between, because source is NULL and so used from the next token (CPP_NAME Y), which doesn't have PREV_WHITE set. Now, the funlike_invocation_p condition if (padding == NULL || (!(padding->flags & PREV_WHITE) && token->val.source == NULL)) padding = token; looks very similar to that in stringify_arg/c-ppoutput.cc, so I assume the intent was to prefer do the same thing and pick the right padding. But there are significant differences. Both stringify_arg and c-ppoutput.cc don't remember the CPP_PADDING token, but its val.source instead, while in funlike_invocation_p we want to remember the padding token that has the significant information for stringify_arg/c-ppoutput.cc. So, IMHO we want to overwrite padding if: 1) padding == NULL (remember that there was any padding at all) 2) padding->val.source == NULL (this matches the source == NULL case in stringify_arg) 3) !(padding->val.source->flags & PREV_WHITE) && token->val.source == NULL (this matches the !(source->flags & PREV_WHITE) && token->val.source == NULL case in stringify_arg) 2022-02-01 Jakub Jelinek PR preprocessor/104147 * macro.cc (funlike_invocation_p): For padding prefer a token with val.source non-NULL especially if it has PREV_WHITE set on val.source->flags. Add gcc_assert that CPP_PADDING tokens don't have PREV_WHITE set in flags. * c-c++-common/cpp/pr104147.c: New test. --- gcc/testsuite/c-c++-common/cpp/pr104147.c | 27 +++++++++++++++++++++++++++ libcpp/macro.cc | 5 ++++- 2 files changed, 31 insertions(+), 1 deletion(-) create mode 100644 gcc/testsuite/c-c++-common/cpp/pr104147.c diff --git a/gcc/testsuite/c-c++-common/cpp/pr104147.c b/gcc/testsuite/c-c++-common/cpp/pr104147.c new file mode 100644 index 0000000..4ee237e --- /dev/null +++ b/gcc/testsuite/c-c++-common/cpp/pr104147.c @@ -0,0 +1,27 @@ +/* PR preprocessor/104147 */ +/* { dg-do run } */ + +#define X(x,y) x y +#define STR_(x) #x +#define STR(x) STR_(x) +const char *str = +STR(X(Y,Y)) +#define Y() +STR(X(Y,Y)) +#undef Y +STR(X(Y,Y)) +#define Y() +STR(X(Y,Y)) +STR(X(Y, +Y)) +STR(X(Y +,Y)) +; + +int +main () +{ + if (__builtin_strcmp (str, "Y YY YY YY YY YY Y") != 0) + __builtin_abort (); + return 0; +} diff --git a/libcpp/macro.cc b/libcpp/macro.cc index 65b7a1c..8ebf360 100644 --- a/libcpp/macro.cc +++ b/libcpp/macro.cc @@ -1373,8 +1373,11 @@ funlike_invocation_p (cpp_reader *pfile, cpp_hashnode *node, token = cpp_get_token (pfile); if (token->type != CPP_PADDING) break; + gcc_assert ((token->flags & PREV_WHITE) == 0); if (padding == NULL - || (!(padding->flags & PREV_WHITE) && token->val.source == NULL)) + || padding->val.source == NULL + || (!(padding->val.source->flags & PREV_WHITE) + && token->val.source == NULL)) padding = token; } -- 2.7.4