You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zeppelin.apache.org by mo...@apache.org on 2018/11/19 18:53:42 UTC
zeppelin git commit: [MINOR] Let jenkins print some build summary on
failure
Repository: zeppelin
Updated Branches:
refs/heads/master 6e97a527e -> 09dc9fcae
[MINOR] Let jenkins print some build summary on failure
### What is this PR for?
Currently actual CI build occurs in individual contributor's travis-ci account, and apache Jenkins poll the result from travis-ci and print build status on it's console output, like
```
Build https://travis-ci.org/test/zeppelin/builds/451036679
[1] OK https://travis-ci.org/test/zeppelin/jobs/451036680
[2] Running ... https://travis-ci.org/test/zeppelin/jobs/451036681
[3] Running ... https://travis-ci.org/test/zeppelin/jobs/451036682
[4] Not completed https://travis-ci.org/test/zeppelin/jobs/451036683
[5] Running ... https://travis-ci.org/test/zeppelin/jobs/451036684
[6] Running ... https://travis-ci.org/test/zeppelin/jobs/451036685
[7] Running ... https://travis-ci.org/test/zeppelin/jobs/451036686
[8] Not started https://travis-ci.org/test/zeppelin/jobs/451036687
[9] Not started https://travis-ci.org/test/zeppelin/jobs/451036688
[10] Not started https://travis-ci.org/test/zeppelin/jobs/451036689
[11] Not started https://travis-ci.org/test/zeppelin/jobs/451036690
1 job(s) failed, 9 job(s) running/pending
```
When there're failed job, developer has to open the travis-ci link and download the log and manually look through to see what was wrong.
This PR make Jenkins not only print build status but also short summary of failure, so developer quickly identify what went wrong. After this patch, Jenkins console output will look something like
```
[2018-11-18 09:11:20] Author: test, commit: ee6d89b
[2018-11-18 09:11:20] --------------------------------
[2018-11-18 09:11:25] Get build status ...
Build https://travis-ci.org/test/zeppelin/builds/451036679
[1] OK https://travis-ci.org/test/zeppelin/jobs/451036680
[2] OK https://travis-ci.org/test/zeppelin/jobs/451036681
[3] Error 1 https://travis-ci.org/test/zeppelin/jobs/451036682
Tests run: 38, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 153.857 sec <<< FAILURE! - in org.apache.zeppelin.notebook.NotebookTest
Please check full log at https://api.travis-ci.org/v3/job/451036682/log.txt
[4] Not completed https://travis-ci.org/test/zeppelin/jobs/451036683
Your build has been stopped.
Please check full log at https://api.travis-ci.org/v3/job/451036683/log.txt
[5] OK https://travis-ci.org/test/zeppelin/jobs/451036684
[6] Error 1 https://travis-ci.org/test/zeppelin/jobs/451036685
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 14.374 sec <<< FAILURE! - in org.apache.zeppelin.spark.SparkRInterpreterTest
Please check full log at https://api.travis-ci.org/v3/job/451036685/log.txt
[7] Error 1 https://travis-ci.org/test/zeppelin/jobs/451036686
Tests run: 15, Failures: 15, Errors: 0, Skipped: 0, Time elapsed: 216.32 sec <<< FAILURE! - in org.apache.zeppelin.interpreter.SparkIntegrationTest
Please check full log at https://api.travis-ci.org/v3/job/451036686/log.txt
[8] Error 1 https://travis-ci.org/test/zeppelin/jobs/451036687
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 17.021 sec <<< FAILURE! - in org.apache.zeppelin.spark.SparkRInterpreterTest
Please check full log at https://api.travis-ci.org/v3/job/451036687/log.txt
[9] Error 1 https://travis-ci.org/test/zeppelin/jobs/451036688
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 18.169 sec <<< FAILURE! - in org.apache.zeppelin.spark.SparkRInterpreterTest
Please check full log at https://api.travis-ci.org/v3/job/451036688/log.txt
[10] OK https://travis-ci.org/test/zeppelin/jobs/451036689
[11] Error 1 https://travis-ci.org/test/zeppelin/jobs/451036690
Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 353.585 sec <<< FAILURE! - in org.apache.zeppelin.livy.LivyInterpreterIT
Please check full log at https://api.travis-ci.org/v3/job/451036690/log.txt
7 job(s) failed, 0 job(s) running/pending
```
### What type of PR is it?
Improvement
### How should this be tested?
* try script manually against some build
e.g.
```
python ./travis_check.py [github username] [commit hash]
python ./travis_check.py NicolasRouquette ee6d89b
```
Author: Lee moon soo <le...@gmail.com>
Closes #3231 from Leemoonsoo/ci_summary and squashes the following commits:
9f42b3226 [Lee moon soo] print build log summary
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/09dc9fca
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/09dc9fca
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/09dc9fca
Branch: refs/heads/master
Commit: 09dc9fcae542815728758b09a32b19456134dd89
Parents: 6e97a52
Author: Lee moon soo <le...@gmail.com>
Authored: Sun Nov 18 09:12:30 2018 +0900
Committer: Lee moon soo <mo...@apache.org>
Committed: Tue Nov 20 03:53:38 2018 +0900
----------------------------------------------------------------------
travis_check.py | 52 +++++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 51 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/09dc9fca/travis_check.py
----------------------------------------------------------------------
diff --git a/travis_check.py b/travis_check.py
index ea5e37b..009ba9a 100644
--- a/travis_check.py
+++ b/travis_check.py
@@ -30,7 +30,10 @@
# # with custom check interval
# python travis_check.py Leemoonsoo 1f2549a 5,60,60
-import os, sys, getopt, traceback, json, requests, time
+import os, sys, getopt, traceback, json, requests, time, urllib3, re
+
+# disable SNIMissingWarning. see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
+urllib3.disable_warnings()
author = sys.argv[1]
commit = sys.argv[2]
@@ -70,6 +73,51 @@ def getBuildStatus(author, commit):
def status(index, msg, jobId):
return '{:20}'.format("[" + str(index+1) + "] " + msg) + "https://travis-ci.org/" + author + "/zeppelin/jobs/" + str(jobId)
+
+# load full build log and summarize
+def logSummary(url):
+ # test start pattern "Running org.apache.zeppelin.scheduler.ParallelSchedulerTest"
+ testStartPattern = re.compile("^Running[ ](.*)")
+ # test end pattern "Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.554 sec - in org.apache.zeppelin.scheduler.JobTest"
+ testEndPattern = re.compile("^Tests [^0-9]*([0-9]+)[^0-9]*([0-9]+)[^0-9]*([0-9]+)[^0-9]*([0-9]+)[^-]*[-][ ]in[ ](.*)")
+
+ tests = {}
+ resp = requests.get(url=url)
+ lines = resp.text.splitlines()
+ lastNonEmptyLine = ""
+ indent = '{:10}'.format("")
+
+ for line in lines:
+ if not len(line.strip()) == 0:
+ lastNonEmptyLine = line
+
+ mStart = testStartPattern.match(line)
+ if mStart:
+ testName = mStart.group(1)
+ tests[testName] = {
+ "start": mStart
+ }
+ continue
+
+ mEnd = testEndPattern.match(line)
+ if mEnd:
+ testName = mEnd.group(5)
+ tests[testName]["end"] = mEnd
+ continue
+
+ for testName, test in tests.items():
+ if not "end" in test:
+ print(indent + "Test " + testName + " never finished")
+ else:
+ failures = int(test["end"].group(2))
+ errors = int(test["end"].group(3))
+ if failures > 0 or errors > 0:
+ print(indent + test["end"].group(0))
+
+ if not lastNonEmptyLine.startswith("Done"):
+ print(indent + lastNonEmptyLine)
+ print(indent + "Please check full log at " + url)
+
def printBuildStatus(build):
failure = 0
running = 0
@@ -88,11 +136,13 @@ def printBuildStatus(build):
if result == None:
print(status(index, "Not completed", jobId))
failure = failure + 1
+ logSummary("https://api.travis-ci.org/v3/job/" + str(jobId) + "/log.txt")
elif result == 0:
print(status(index, "OK", jobId))
else:
print(status(index, "Error " + str(result), jobId))
failure = failure + 1
+ logSummary("https://api.travis-ci.org/v3/job/" + str(jobId) + "/log.txt")
else:
print(status(index, "Unknown state", jobId))
failure = failure + 1