Disable use-color if the output stream is not a terminal with color support.

Summary:
LLDB currently only checks the output terminal for color support by looking
at the `TERM` environment variable and comparing it to `"dumb"`. This causes that
when running LLDB on a CI node, the syntax highlighter will not be deactivated by
LLDB and the output log is filled with color codes (unless the terminal emulator
actually exposes itself as dumb).

This patch now relies on the LLVM code for detecting color support which is more
reliable. We now also correctly actually initialize the `m_supports_colors` variable in `File`.
`m_supports_colors` was so far uninitialized, but the code path that uses `m_supports_colors`
was also dead so the sanitizers didn't sound an alarm.

The old check that compares `TERM` is not removed by this patch as the new LLVM code
doesn't seem to handle this case (and it's a good thing to check for "dumb" terminals).

Reviewers: aprantl, javed.absar

Reviewed By: aprantl

Subscribers: kristof.beyls, abidh, lldb-commits

Differential Revision: https://reviews.llvm.org/D51243

llvm-svn: 340747

diff --git a/lldb/include/lldb/Host/File.h b/lldb/include/lldb/Host/File.h
index d240f81..8cc21ba 100644 (file)
--- a/lldb/include/lldb/Host/File.h
+++ b/lldb/include/lldb/Host/File.h
@@ -54,13 +54,15 @@ public:
       : IOObject(eFDTypeFile, false), m_descriptor(kInvalidDescriptor),
         m_stream(kInvalidStream), m_options(0), m_own_stream(false),
         m_is_interactive(eLazyBoolCalculate),
-        m_is_real_terminal(eLazyBoolCalculate) {}
+        m_is_real_terminal(eLazyBoolCalculate),
+        m_supports_colors(eLazyBoolCalculate) {}
 
   File(FILE *fh, bool transfer_ownership)
       : IOObject(eFDTypeFile, false), m_descriptor(kInvalidDescriptor),
         m_stream(fh), m_options(0), m_own_stream(transfer_ownership),
         m_is_interactive(eLazyBoolCalculate),
-        m_is_real_terminal(eLazyBoolCalculate) {}
+        m_is_real_terminal(eLazyBoolCalculate),
+        m_supports_colors(eLazyBoolCalculate) {}
 
   //------------------------------------------------------------------
   /// Constructor with path.

diff --git a/lldb/lit/Settings/TestDisableColor.test b/lldb/lit/Settings/TestDisableColor.test
new file mode 100644 (file)
index 0000000..659e207
--- /dev/null
+++ b/lldb/lit/Settings/TestDisableColor.test
@@ -0,0 +1,7 @@
+# RUN: %lldb -x -b -s %s | FileCheck %s
+settings show use-color
+q
+# This tests that LLDB turns off use-color if the output file is not an
+# interactive terminal. In this example, use-color should be off because LLDB
+# is run just by the non-interactive lit test runner.
+# CHECK: use-color (boolean) = false

diff --git a/lldb/lit/Settings/lit.local.cfg b/lldb/lit/Settings/lit.local.cfg
new file mode 100644 (file)
index 0000000..df9b335
--- /dev/null
+++ b/lldb/lit/Settings/lit.local.cfg
@@ -0,0 +1 @@
+config.suffixes = ['.test']

diff --git a/lldb/source/Core/Debugger.cpp b/lldb/source/Core/Debugger.cpp
index f5eab84..898cccb 100644 (file)
--- a/lldb/source/Core/Debugger.cpp
+++ b/lldb/source/Core/Debugger.cpp
@@ -804,6 +804,9 @@ Debugger::Debugger(lldb::LogOutputCallback log_callback, void *baton)
   const char *term = getenv("TERM");
   if (term && !strcmp(term, "dumb"))
     SetUseColor(false);
+  // Turn off use-color if we don't write to a terminal with color support.
+  if (!m_output_file_sp->GetFile().GetIsTerminalWithColors())
+    SetUseColor(false);
 }
 
 Debugger::~Debugger() { Clear(); }