[NEW] Debugging test failures on Jenkins

[Andrew Or <an...@databricks.com>]

Hi everyone,

As of today, the pull request tester on Jenkins includes not only the
standard test output, but also the log4j logs for all drivers / executors /
other processes started during the tests.

This is motivated by the fact that some test failure messages are not super
helpful, e.g. "futures timed out" or "subprocess exited with code 1".
Having log4j logs will help us track down flaky tests more effectively.

Follow the following steps (screenshots attached!) to view them:
1. From your pull request, go to Jenkins through "Test build X has finished"
2. You will be taken to the Console Output page. Click "Status" for your
build.
3. You will see a "Build Artifact" icon. Click into it.
4. Find the "unit-tests.log" file of interest

Since SPARK-7558, these log files mark the boundaries of tests within a
suite, so you should be able to find the logs for a particular test easily.
Note that this only applies to tests in branches 1.3 or newer.

Cheers,
-Andrew

[image: Images intégrées 1]

[image: Images intégrées 2]

[image: Images intégrées 3]

[image: Images intégrées 4]