review.tizen.org Git - platform/upstream/gcc.git/commit

Support extended characters in C/C++ identifiers (PR c/67224)

libcpp/ChangeLog
2019-09-19  Lewis Hyatt  <lhyatt@gmail.com>

PR c/67224
* charset.c (_cpp_valid_utf8): New function to help lex UTF-8 tokens.
* internal.h (_cpp_valid_utf8): Declare.
* lex.c (forms_identifier_p): Use it to recognize UTF-8 identifiers.
(_cpp_lex_direct): Handle UTF-8 in identifiers and CPP_OTHER tokens.
Do all work in "default" case to avoid slowing down typical code paths.
Also handle $ and UCN in the default case for consistency.

gcc/Changelog
2019-09-19  Lewis Hyatt  <lhyatt@gmail.com>

PR c/67224
* doc/cpp.texi: Document support for extended characters in
identifiers.
* doc/cppopts.texi: Likewise.

gcc/testsuite/ChangeLog
2019-09-19  Lewis Hyatt  <lhyatt@gmail.com>

PR c/67224
* c-c++-common/cpp/ucnid-2011-1-utf8.c: New test.
* g++.dg/cpp/ucnid-1-utf8.C: New test.
* g++.dg/cpp/ucnid-2-utf8.C: New test.
* g++.dg/cpp/ucnid-3-utf8.C: New test.
* g++.dg/cpp/ucnid-4-utf8.C: New test.
* g++.dg/other/ucnid-1-utf8.C: New test.
* gcc.dg/cpp/ucnid-1-utf8.c: New test.
* gcc.dg/cpp/ucnid-10-utf8.c: New test.
* gcc.dg/cpp/ucnid-11-utf8.c: New test.
* gcc.dg/cpp/ucnid-12-utf8.c: New test.
* gcc.dg/cpp/ucnid-13-utf8.c: New test.
* gcc.dg/cpp/ucnid-14-utf8.c: New test.
* gcc.dg/cpp/ucnid-15-utf8.c: New test.
* gcc.dg/cpp/ucnid-2-utf8.c: New test.
* gcc.dg/cpp/ucnid-3-utf8.c: New test.
* gcc.dg/cpp/ucnid-4-utf8.c: New test.
* gcc.dg/cpp/ucnid-6-utf8.c: New test.
* gcc.dg/cpp/ucnid-7-utf8.c: New test.
* gcc.dg/cpp/ucnid-9-utf8.c: New test.
* gcc.dg/ucnid-1-utf8.c: New test.
* gcc.dg/ucnid-10-utf8.c: New test.
* gcc.dg/ucnid-11-utf8.c: New test.
* gcc.dg/ucnid-12-utf8.c: New test.
* gcc.dg/ucnid-13-utf8.c: New test.
* gcc.dg/ucnid-14-utf8.c: New test.
* gcc.dg/ucnid-15-utf8.c: New test.
* gcc.dg/ucnid-16-utf8.c: New test.
* gcc.dg/ucnid-2-utf8.c: New test.
* gcc.dg/ucnid-3-utf8.c: New test.
* gcc.dg/ucnid-4-utf8.c: New test.
* gcc.dg/ucnid-5-utf8.c: New test.
* gcc.dg/ucnid-6-utf8.c: New test.
* gcc.dg/ucnid-7-utf8.c: New test.
* gcc.dg/ucnid-8-utf8.c: New test.
* gcc.dg/ucnid-9-utf8.c: New test.

From-SVN: r275979

author	Lewis Hyatt <lhyatt@gmail.com>
	Thu, 19 Sep 2019 19:56:11 +0000 (19:56 +0000)
committer	Joseph Myers <jsm28@gcc.gnu.org>
	Thu, 19 Sep 2019 19:56:11 +0000 (20:56 +0100)
commit	7d112d6670a0e0e662f8a7e64c33686e475832c8
tree	983eb23217b2572ff4fe5a7f7fe0e5c0c0b9a48d	tree \| snapshot
parent	e0710fcf7dc70054a9a20ab1b8d77f4fef26ef2c	commit \| diff

gcc/ChangeLog		diff \| blob \| history
gcc/doc/cpp.texi		diff \| blob \| history
gcc/doc/cppopts.texi		diff \| blob \| history
gcc/testsuite/ChangeLog		diff \| blob \| history
gcc/testsuite/c-c++-common/cpp/ucnid-2011-1-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/g++.dg/cpp/ucnid-1-utf8.C	[new file with mode: 0644]	blob
gcc/testsuite/g++.dg/cpp/ucnid-2-utf8.C	[new file with mode: 0644]	blob
gcc/testsuite/g++.dg/cpp/ucnid-3-utf8.C	[new file with mode: 0644]	blob
gcc/testsuite/g++.dg/cpp/ucnid-4-utf8.C	[new file with mode: 0644]	blob
gcc/testsuite/g++.dg/other/ucnid-1-utf8.C	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/cpp/ucnid-1-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/cpp/ucnid-10-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/cpp/ucnid-11-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/cpp/ucnid-12-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/cpp/ucnid-13-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/cpp/ucnid-14-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/cpp/ucnid-15-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/cpp/ucnid-2-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/cpp/ucnid-3-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/cpp/ucnid-4-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/cpp/ucnid-6-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/cpp/ucnid-7-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/cpp/ucnid-9-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/ucnid-1-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/ucnid-10-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/ucnid-11-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/ucnid-12-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/ucnid-13-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/ucnid-14-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/ucnid-15-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/ucnid-16-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/ucnid-2-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/ucnid-3-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/ucnid-4-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/ucnid-5-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/ucnid-6-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/ucnid-7-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/ucnid-8-utf8.c	[new file with mode: 0644]	blob
gcc/testsuite/gcc.dg/ucnid-9-utf8.c	[new file with mode: 0644]	blob
libcpp/ChangeLog		diff \| blob \| history
libcpp/charset.c		diff \| blob \| history
libcpp/internal.h		diff \| blob \| history
libcpp/lex.c		diff \| blob \| history