[libFuzzer] Use octal instead of hex escape sequences in PrintASCII
authorHans Wennborg <hans@chromium.org>
Fri, 1 Oct 2021 08:59:55 +0000 (10:59 +0200)
committerHans Wennborg <hans@chromium.org>
Mon, 4 Oct 2021 09:29:54 +0000 (11:29 +0200)
commitc7bd6435993ffbe0fbd1a06c2ba6fb55575edbe1
tree897d8b95d541fd7eb4369ada9bdf9795b0598ee4
parent4288b6520a8ed37a17fe3f41c3a686f3d3bf9f6e
[libFuzzer] Use octal instead of hex escape sequences in PrintASCII

Previously, PrintASCII would print the string "\ta" as "\x09a". However,
in C/C++ those strings are not the same: the trailing 'a' is part of the
escape sequence, which means it's equivalent to "\x9a". This is an
annoying quirk of the standard. (See
https://eel.is/c++draft/lex.ccon#nt:hexadecimal-escape-sequence)

To fix this, output three-digit octal escape sequences instead. Since
octal escapes are limited to max three digits, this avoids the problem
of subsequent characters unintentionally becoming part of the escape
sequence.

Dictionary files still use the non-C-compatible hex escapes, but I
believe we can't change the format since it comes from AFL, and
libfuzzer never writes such files, it only has to read them, so they're
not affected by this change.

Differential revision: https://reviews.llvm.org/D110920
compiler-rt/lib/fuzzer/FuzzerIO.cpp
compiler-rt/lib/fuzzer/FuzzerIO.h
compiler-rt/lib/fuzzer/FuzzerUtil.cpp
compiler-rt/lib/fuzzer/tests/FuzzerUnittest.cpp