You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by zentol <gi...@git.apache.org> on 2018/05/14 10:34:53 UTC
[GitHub] flink pull request #6008: [FLINK-9354][travis] Print execution times for nig...
GitHub user zentol opened a pull request:
https://github.com/apache/flink/pull/6008
[FLINK-9354][travis] Print execution times for nightly E2E tests
## What is the purpose of the change
With this PR we print the execution time for all end-to-end tests.
## Brief change log
* add `start_timer`/`end_timer` functions to `common.sh`
* cover all E2E invocations with timers
## Verifying this change
* observe travis output when execution e2e tests
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zentol/flink 9354
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/6008.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #6008
----
commit f3893e798fc4cbe93d6eb2b3a0a9ad771bc8595f
Author: zentol <ch...@...>
Date: 2018-05-14T10:27:34Z
[FLINK-9354][travis] Print execution times for nightly E2E tests
----
---
[GitHub] flink pull request #6008: [FLINK-9354][travis] Print execution times for nig...
Posted by tzulitai <gi...@git.apache.org>.
Github user tzulitai commented on a diff in the pull request:
https://github.com/apache/flink/pull/6008#discussion_r188507930
--- Diff: flink-end-to-end-tests/run-nightly-tests.sh ---
@@ -43,121 +43,151 @@ EXIT_CODE=0
# printf "\n==============================================================================\n"
# printf "Running my fancy nightly end-to-end test\n"
# printf "==============================================================================\n"
+# start_timer
# $END_TO_END_DIR/test-scripts/test_something_very_fancy.sh
# EXIT_CODE=$?
+# end_timer
# fi
if [ $EXIT_CODE == 0 ]; then
printf "\n==============================================================================\n"
printf "Running HA end-to-end test\n"
printf "==============================================================================\n"
+ start_timer
$END_TO_END_DIR/test-scripts/test_ha.sh
EXIT_CODE=$?
+ end_timer
fi
if [ $EXIT_CODE == 0 ]; then
printf "\n==============================================================================\n"
printf "Running Resuming Savepoint (file, async, no parallelism change) end-to-end test\n"
printf "==============================================================================\n"
+ start_timer
STATE_BACKEND_TYPE=file STATE_BACKEND_FILE_ASYNC=true $END_TO_END_DIR/test-scripts/test_resume_savepoint.sh 2 2
EXIT_CODE=$?
+ end_timer
fi
if [ $EXIT_CODE == 0 ]; then
printf "\n==============================================================================\n"
printf "Running Resuming Savepoint (file, sync, no parallelism change) end-to-end test\n"
printf "==============================================================================\n"
+ start_timer
STATE_BACKEND_TYPE=file STATE_BACKEND_FILE_ASYNC=false $END_TO_END_DIR/test-scripts/test_resume_savepoint.sh 2 2
EXIT_CODE=$?
+ end_timer
fi
if [ $EXIT_CODE == 0 ]; then
printf "\n==============================================================================\n"
printf "Running Resuming Savepoint (file, async, scale up) end-to-end test\n"
printf "==============================================================================\n"
+ start_timer
STATE_BACKEND_TYPE=file STATE_BACKEND_FILE_ASYNC=true $END_TO_END_DIR/test-scripts/test_resume_savepoint.sh 2 4
EXIT_CODE=$?
+ end_timer
fi
--- End diff --
Perhaps we can refactor this whole block into a common base script for the nightly / pre-commit hook scripts.
The end_timer / start_timer functions should also be located there.
---
[GitHub] flink pull request #6008: [FLINK-9354][travis] Print execution times for nig...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/6008
---
[GitHub] flink issue #6008: [FLINK-9354][travis] Print execution times for nightly E2...
Posted by tzulitai <gi...@git.apache.org>.
Github user tzulitai commented on the issue:
https://github.com/apache/flink/pull/6008
I see these in the Travis logs:
```
flink-end-to-end-tests/run-pre-commit-tests.sh: line 94: start_timer: command not found
...
flink-end-to-end-tests/run-pre-commit-tests.sh: line 97: end_timer: command not found
```
I think you'll have to include `common.sh` in the nightly / pre-commit hook scripts.
---
[GitHub] flink issue #6008: [FLINK-9354][travis] Print execution times for nightly E2...
Posted by tzulitai <gi...@git.apache.org>.
Github user tzulitai commented on the issue:
https://github.com/apache/flink/pull/6008
+1 LGTM
---
[GitHub] flink pull request #6008: [FLINK-9354][travis] Print execution times for nig...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/6008#discussion_r188550129
--- Diff: flink-end-to-end-tests/test-scripts/test_resume_savepoint.sh ---
@@ -33,8 +33,8 @@ else
NUM_SLOTS=$NEW_DOP
fi
-STATE_BACKEND_TYPE=${STATE_BACKEND_TYPE:-file}
-STATE_BACKEND_FILE_ASYNC=${STATE_BACKEND_FILE_ASYNC:-true}
+STATE_BACKEND_TYPE=${3:-file}
+STATE_BACKEND_FILE_ASYNC=${4:-true}
--- End diff --
will do 👍
---
[GitHub] flink pull request #6008: [FLINK-9354][travis] Print execution times for nig...
Posted by tzulitai <gi...@git.apache.org>.
Github user tzulitai commented on a diff in the pull request:
https://github.com/apache/flink/pull/6008#discussion_r188548653
--- Diff: flink-end-to-end-tests/test-scripts/test_resume_savepoint.sh ---
@@ -33,8 +33,8 @@ else
NUM_SLOTS=$NEW_DOP
fi
-STATE_BACKEND_TYPE=${STATE_BACKEND_TYPE:-file}
-STATE_BACKEND_FILE_ASYNC=${STATE_BACKEND_FILE_ASYNC:-true}
+STATE_BACKEND_TYPE=${3:-file}
+STATE_BACKEND_FILE_ASYNC=${4:-true}
--- End diff --
We should also update the usage message at the beginning of this file to reflect these extra parameters.
---
[GitHub] flink issue #6008: [FLINK-9354][travis] Print execution times for nightly E2...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/6008
nice catch @tzulitai, will rework this into a run_test method that takes a description and command to execute.
---