buildman: properly translate strings for log and err files to ASCII

The build output can still produce unicode encoded output. But in
the buildman's log and err files we only want plain ASCII characters.

To handle all situations with unicode and non-unicode output, encode
the stdout and stderr strings to UTF-8 and afterwards to ASCII with
replacing all special characters.

Signed-off-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>

diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py
index acaf500..9e8ca80 100644 (file)
--- a/tools/buildman/builderthread.py
+++ b/tools/buildman/builderthread.py
@@ -280,13 +280,15 @@ class BuilderThread(threading.Thread):
         outfile = os.path.join(build_dir, 'log')
         with open(outfile, 'w') as fd:
             if result.stdout:
-                fd.write(result.stdout.encode('latin-1', 'ignore'))
+                # We don't want unicode characters in log files
+                fd.write(result.stdout.decode('UTF-8').encode('ASCII', 'replace'))
 
         errfile = self.builder.GetErrFile(result.commit_upto,
                 result.brd.target)
         if result.stderr:
             with open(errfile, 'w') as fd:
-                fd.write(result.stderr.encode('latin-1', 'ignore'))
+                # We don't want unicode characters in log files
+                fd.write(result.stderr.decode('UTF-8').encode('ASCII', 'replace'))
         elif os.path.exists(errfile):
             os.remove(errfile)