[jira] [Created] (FLINK-9257) End-to-end tests prints "All tests PASS" even if individual test-script returns non-zero exit code

["Florian Schmidt (JIRA)" <ji...@apache.org>]

Florian Schmidt created FLINK-9257:
--------------------------------------

             Summary: End-to-end tests prints "All tests PASS" even if individual test-script returns non-zero exit code
                 Key: FLINK-9257
                 URL: https://issues.apache.org/jira/browse/FLINK-9257
             Project: Flink
          Issue Type: Bug
          Components: Tests
    Affects Versions: 1.5.0
            Reporter: Florian Schmidt
             Fix For: 1.5.0


In some cases the test-suite exits with non-zero exit code but still prints "All tests PASS" to stdout. This happens because how the test runner works, which is roughly as follows
 # Either run-nightly-tests.sh or run-precommit-tests.sh executes a suite of tests consisting of one multiple bash scripts.
 # As soon as one of those bash scripts exists with non-zero exit code, the tests won't continue to run and the test-suite will also exit with non-zero exit code.
 # *During the cleanup hook (trap cleanup EXIT in common.sh) it will be checked whether there are non-empty out files or log files with certain exceptions. If a tests fails with non-zero exit code, but does not have any exceptions or .out files, this will still print "All tests PASS" to stdout, even though they don't*

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)