You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kudu.apache.org by to...@apache.org on 2016/02/25 23:11:07 UTC
[2/3] incubator-kudu git commit: test_result_server: ignore binary
data during diagnose
test_result_server: ignore binary data during diagnose
Kudu's log output sometimes includes binary data, which leads to diagnosing
errors like these:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 670, in respond
response.body = self.handler()
File "/usr/local/lib/python2.7/dist-packages/cherrypy/lib/encoding.py", line 217, in __call__
self.body = self.oldhandler(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cpdispatch.py", line 61, in __call__
return self.callable(*self.args, **self.kwargs)
File "/home/testserver/build-support/test_result_server.py", line 182, in diagnose
log_text = gzip.GzipFile(fileobj=StringIO(log_text_gz)).read().decode('utf-8')
File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe2 in position 82571: invalid continuation byte
Switching from the default ('strict') error handling behavior in favor of
'ignore' resolves the issue.
Change-Id: I44cae10bf1c02419e2c2152d0f1f97f1c22c3dd6
Reviewed-on: http://gerrit.cloudera.org:8080/2312
Tested-by: Kudu Jenkins
Reviewed-by: Todd Lipcon <to...@apache.org>
Project: http://git-wip-us.apache.org/repos/asf/incubator-kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-kudu/commit/cd37e8d1
Tree: http://git-wip-us.apache.org/repos/asf/incubator-kudu/tree/cd37e8d1
Diff: http://git-wip-us.apache.org/repos/asf/incubator-kudu/diff/cd37e8d1
Branch: refs/heads/master
Commit: cd37e8d1d40c6ec2e4e638259429f5024e309f2a
Parents: b48d9f6
Author: Adar Dembo <ad...@cloudera.com>
Authored: Wed Feb 24 17:43:06 2016 -0800
Committer: Todd Lipcon <to...@apache.org>
Committed: Thu Feb 25 22:09:11 2016 +0000
----------------------------------------------------------------------
build-support/test_result_server.py | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-kudu/blob/cd37e8d1/build-support/test_result_server.py
----------------------------------------------------------------------
diff --git a/build-support/test_result_server.py b/build-support/test_result_server.py
index 8570eb2..dde1839 100755
--- a/build-support/test_result_server.py
+++ b/build-support/test_result_server.py
@@ -179,7 +179,11 @@ class TRServer(object):
k = boto.s3.key.Key(self.s3_bucket)
k.key = key
log_text_gz = k.get_contents_as_string()
- log_text = gzip.GzipFile(fileobj=StringIO(log_text_gz)).read().decode('utf-8')
+ encoded_text = gzip.GzipFile(fileobj=StringIO(log_text_gz)).read()
+
+ # Ignore errors in decoding, as logs may contain binary data.
+ log_text = encoded_text.decode('utf-8', 'ignore')
+
summary = parse_test_failure.extract_failure_summary(log_text)
if not summary:
summary = "Unable to diagnose"