You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2020/01/02 10:15:12 UTC

[GitHub] [flink] KarmaGYZ opened a new pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

KarmaGYZ opened a new pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746
 
 
   …2e cluster
   
   <!--
   *Thank you very much for contributing to Apache Flink - we are happy that you want to help us improve Flink. To help the community review your contribution in the best possible way, please go through the checklist below, which will get the contribution into a shape in which it can be best reviewed.*
   
   *Please understand that we do not do this to make contributions to Flink a hassle. In order to uphold a high standard of quality for code contributions, while at the same time managing a large number of contributions, we need contributors to prepare the contributions well, and give reviewers enough contextual information for the review. Please also understand that contributions that do not follow this guide will take longer to review and thus typically be picked up with lower priority by the community.*
   
   ## Contribution Checklist
   
     - Make sure that the pull request corresponds to a [JIRA issue](https://issues.apache.org/jira/projects/FLINK/issues). Exceptions are made for typos in JavaDoc or documentation files, which need no JIRA issue.
     
     - Name the pull request in the form "[FLINK-XXXX] [component] Title of the pull request", where *FLINK-XXXX* should be replaced by the actual issue number. Skip *component* if you are unsure about which is the best component.
     Typo fixes that have no associated JIRA issue should be named following this pattern: `[hotfix] [docs] Fix typo in event time introduction` or `[hotfix] [javadocs] Expand JavaDoc for PuncuatedWatermarkGenerator`.
   
     - Fill out the template below to describe the changes contributed by the pull request. That will give reviewers the context they need to do the review.
     
     - Make sure that the change passes the automated tests, i.e., `mvn clean verify` passes. You can set up Travis CI to do that following [this guide](https://flink.apache.org/contributing/contribute-code.html#open-a-pull-request).
   
     - Each pull request should address only one issue, not mix up code from multiple issues.
     
     - Each commit in the pull request has a meaningful commit message (including the JIRA id)
   
     - Once all items of the checklist are addressed, remove the above text and this checklist, leaving only the filled out template below.
   
   
   **(The sections below can be removed for hotfixes of typos)**
   -->
   
   ## What is the purpose of the change
   
   There is a potential risk of permission problems when cleanup logs and output if we mount a directory to docker container. This PR fix that issue by remove the docker volume from docker-compose file.
   
   ## Brief change log
   
   - Remove the docker volume or mount when starting Mesos e2e cluster
   
   ## Verifying this change
   
   Trigger the mesos e2e test.
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): no
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
     - The serializers: no
     - The runtime per-record code paths (performance sensitive): no
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: no
     - The S3 file system connector: no
   
   ## Documentation
   
     - Does this pull request introduce a new feature? no
     - If yes, how is the feature documented? no
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] GJL commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
GJL commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-581440100
 
 
   > One minor concern, we may not redirect the error message of `rm -rf ${TEST_DATA_DIR}`. WDYT?
   
   Good point. IMO it should be an implementation error if the test data dir cannot be deleted. However, if we remove the redirection of stderr, I wonder if our nightly E2E tests would currently pass. Anyways, this can be done in a separate issue.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] GJL commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
GJL commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-580285665
 
 
   > The chmod -R ugo+rw looks good to me, but can you ensure this command will not meet permission issue? 
   
   Adding something like this
   ```
   docker exec mesos-master bash -c "chmod -R ogu+rw ${FLINK_DIR}/log/"
   docker exec mesos-master bash -c "chmod -R ogu+rw ${TEST_DATA_DIR}"
   ```
   to `cluster_shutdown()` should work (disclaimer: I haven't tested this on Linux). I don't see a permission issue here because by default these commands will be run by root (we do not use the `USER` command when building the docker image). 
   
   
   
   
   
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] wangyang0918 commented on a change in pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
wangyang0918 commented on a change in pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#discussion_r363719247
 
 

 ##########
 File path: flink-end-to-end-tests/test-scripts/common_mesos_docker.sh
 ##########
 @@ -52,11 +54,19 @@ function start_flink_cluster_with_mesos() {
     set_config_key "jobmanager.rpc.address" "mesos-master"
     set_config_key "rest.address" "mesos-master"
 
-    docker exec -itd mesos-master bash -c "${FLINK_DIR}/bin/mesos-appmaster.sh -Dmesos.master=mesos-master:5050"
+    docker cp ${FLINK_DIR} mesos-master:$MESOS_FLINK_DIR
+    docker cp ${END_TO_END_DIR}/test-scripts mesos-master:$MESOS_END_TO_END_DIR
+    docker cp ${END_TO_END_DIR}/test-scripts mesos-slave:$MESOS_END_TO_END_DIR
+
+    docker exec -itd mesos-master bash -c "${MESOS_FLINK_DIR}/bin/mesos-appmaster.sh -Dmesos.master=mesos-master:5050"
     wait_rest_endpoint_up "http://${NODENAME}:8081/taskmanagers" "Dispatcher" "\{\"taskmanagers\":\[.*\]\}"
     return 0
 }
 
+function copy_logs_from_container () {
 
 Review comment:
   Copy the logs to `$TEST_DATA_DIR/logs/` and cat them. The logs will be cleaned up automatically. Print the logs in stdout when failed will make debugging more easier.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-570173870
 
 
   <!--
   Meta data
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/142858913 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:FAILURE URL:https://travis-ci.com/flink-ci/flink/builds/143336203 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   Hash:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/143556983 TriggerType:PUSH TriggerID:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea
   Hash:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4191 TriggerType:PUSH TriggerID:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea
   Hash:edc9ea10e99dfc96f91ded6702a9032d801fafa9 Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/147164007 TriggerType:PUSH TriggerID:edc9ea10e99dfc96f91ded6702a9032d801fafa9
   Hash:edc9ea10e99dfc96f91ded6702a9032d801fafa9 Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4767 TriggerType:PUSH TriggerID:edc9ea10e99dfc96f91ded6702a9032d801fafa9
   Hash:09b4361d6279d3c1f03dcdd2a1ae6a3c59ac2058 Status:UNKNOWN URL:TBD TriggerType:PUSH TriggerID:09b4361d6279d3c1f03dcdd2a1ae6a3c59ac2058
   -->
   ## CI report:
   
   * 964b4874eaa647c908289dd841b660bc966fdb61 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142858913) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045) 
   * c324abe0418f58ce00aa3bb881400bea18a34d2b Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/143336203) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145) 
   * 76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143556983) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4191) 
   * edc9ea10e99dfc96f91ded6702a9032d801fafa9 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/147164007) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4767) 
   * 09b4361d6279d3c1f03dcdd2a1ae6a3c59ac2058 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-570173870
 
 
   <!--
   Meta data
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/142858913 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:FAILURE URL:https://travis-ci.com/flink-ci/flink/builds/143336203 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   Hash:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/143556983 TriggerType:PUSH TriggerID:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea
   Hash:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4191 TriggerType:PUSH TriggerID:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea
   Hash:edc9ea10e99dfc96f91ded6702a9032d801fafa9 Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/147164007 TriggerType:PUSH TriggerID:edc9ea10e99dfc96f91ded6702a9032d801fafa9
   Hash:edc9ea10e99dfc96f91ded6702a9032d801fafa9 Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4767 TriggerType:PUSH TriggerID:edc9ea10e99dfc96f91ded6702a9032d801fafa9
   -->
   ## CI report:
   
   * 964b4874eaa647c908289dd841b660bc966fdb61 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142858913) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045) 
   * c324abe0418f58ce00aa3bb881400bea18a34d2b Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/143336203) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145) 
   * 76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143556983) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4191) 
   * edc9ea10e99dfc96f91ded6702a9032d801fafa9 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/147164007) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4767) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] wangyang0918 commented on a change in pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
wangyang0918 commented on a change in pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#discussion_r363720131
 
 

 ##########
 File path: flink-end-to-end-tests/test-scripts/test_mesos_wordcount.sh
 ##########
 @@ -23,14 +23,16 @@ source "$(dirname "$0")"/common.sh
 source "$(dirname "$0")"/common_mesos_docker.sh
 
 RESULT_HASH="72a690412be8928ba239c2da967328a5"
-INPUT_ARGS="--input ${TEST_INFRA_DIR}/test-data/words"
-OUTPUT_LOCATION="${TEST_DATA_DIR}/out/wc_out_mesos"
-TEST_PROGRAM_JAR=${FLINK_DIR}/examples/batch/WordCount.jar
+INPUT_LOCATION="${MESOS_END_TO_END_DIR}/test-data/words"
+OUTPUT_LOCATION="${MESOS_END_TO_END_DIR}/test-data/out/wc_out_mesos"
+TEST_PROGRAM_JAR="${MESOS_FLINK_DIR}/examples/batch/WordCount.jar"
 
 mkdir -p "${TEST_DATA_DIR}"
 
 start_flink_cluster_with_mesos
 
-docker exec -it mesos-master nohup bash -c "${FLINK_DIR}/bin/flink run -p 1 ${TEST_PROGRAM_JAR} ${INPUT_ARGS} --output ${OUTPUT_LOCATION}"
+docker exec -it mesos-master nohup bash -c "${MESOS_FLINK_DIR}/bin/flink run -p 1 ${TEST_PROGRAM_JAR} --input ${INPUT_LOCATION} --output ${OUTPUT_LOCATION}"
+docker cp mesos-slave:"${MESOS_END_TO_END_DIR}/test-data/out" "${TEST_DATA_DIR}/out"
 
-check_result_hash "Mesos WordCount test" "${OUTPUT_LOCATION}" "${RESULT_HASH}"
+check_result_hash "Mesos WordCount test" "${TEST_DATA_DIR}/out/wc_out_mesos" "${RESULT_HASH}"
+copy_logs_from_container
 
 Review comment:
   Copy and print logs when failed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-570173870
 
 
   <!--
   Meta data
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/142858913 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:FAILURE URL:https://travis-ci.com/flink-ci/flink/builds/143336203 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:PENDING URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   -->
   ## CI report:
   
   * 964b4874eaa647c908289dd841b660bc966fdb61 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142858913) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045) 
   * c324abe0418f58ce00aa3bb881400bea18a34d2b Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/143336203) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] KarmaGYZ commented on a change in pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on a change in pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#discussion_r366138517
 
 

 ##########
 File path: flink-end-to-end-tests/test-scripts/common_mesos_docker.sh
 ##########
 @@ -53,11 +55,22 @@ function start_flink_cluster_with_mesos() {
     set_config_key "jobmanager.rpc.address" "mesos-master"
     set_config_key "rest.address" "mesos-master"
 
-    docker exec -itd mesos-master bash -c "${FLINK_DIR}/bin/mesos-appmaster.sh -Dmesos.master=mesos-master:5050"
+    docker cp ${FLINK_DIR} mesos-master:$MESOS_FLINK_DIR
 
 Review comment:
   > all writes should go to a volume or bind mount
   
   Regarding this concern, I think that the two approaches do not have significant difference. Since We'll always write the log to the container's storage. If you have a strong point here, I'm ok to bundle Flink and the test data inside the image.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-570173870
 
 
   <!--
   Meta data
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/142858913 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:FAILURE URL:https://travis-ci.com/flink-ci/flink/builds/143336203 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   Hash:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/143556983 TriggerType:PUSH TriggerID:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea
   Hash:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4191 TriggerType:PUSH TriggerID:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea
   Hash:edc9ea10e99dfc96f91ded6702a9032d801fafa9 Status:PENDING URL:https://travis-ci.com/flink-ci/flink/builds/147164007 TriggerType:PUSH TriggerID:edc9ea10e99dfc96f91ded6702a9032d801fafa9
   Hash:edc9ea10e99dfc96f91ded6702a9032d801fafa9 Status:PENDING URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4767 TriggerType:PUSH TriggerID:edc9ea10e99dfc96f91ded6702a9032d801fafa9
   -->
   ## CI report:
   
   * 964b4874eaa647c908289dd841b660bc966fdb61 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142858913) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045) 
   * c324abe0418f58ce00aa3bb881400bea18a34d2b Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/143336203) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145) 
   * 76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143556983) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4191) 
   * edc9ea10e99dfc96f91ded6702a9032d801fafa9 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/147164007) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4767) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
flinkbot commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-570166569
 
 
   Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
   to review your pull request. We will use this comment to track the progress of the review.
   
   
   ## Automated Checks
   Last check on commit 964b4874eaa647c908289dd841b660bc966fdb61 (Thu Jan 02 10:18:21 UTC 2020)
   
   **Warnings:**
    * No documentation files were touched! Remember to keep the Flink docs up to date!
   
   
   <sub>Mention the bot in a comment to re-run the automated checks.</sub>
   ## Review Progress
   
   * ❓ 1. The [description] looks good.
   * ❓ 2. There is [consensus] that the contribution should go into to Flink.
   * ❓ 3. Needs [attention] from.
   * ❓ 4. The change fits into the overall [architecture].
   * ❓ 5. Overall code [quality] is good.
   
   Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
    The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
    - `@flinkbot approve all` to approve all aspects
    - `@flinkbot approve-until architecture` to approve everything until `architecture`
    - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
    - `@flinkbot disapprove architecture` to remove an approval you gave earlier
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-570173870
 
 
   <!--
   Meta data
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/142858913 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:FAILURE URL:https://travis-ci.com/flink-ci/flink/builds/143336203 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   Hash:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/143556983 TriggerType:PUSH TriggerID:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea
   Hash:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Status:PENDING URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4191 TriggerType:PUSH TriggerID:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea
   -->
   ## CI report:
   
   * 964b4874eaa647c908289dd841b660bc966fdb61 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142858913) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045) 
   * c324abe0418f58ce00aa3bb881400bea18a34d2b Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/143336203) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145) 
   * 76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143556983) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4191) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] KarmaGYZ commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-581717626
 
 
   Thanks for the feedback @GJL . PR updated.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-570173870
 
 
   <!--
   Meta data
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/142858913 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:FAILURE URL:https://travis-ci.com/flink-ci/flink/builds/143336203 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   Hash:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/143556983 TriggerType:PUSH TriggerID:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea
   Hash:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4191 TriggerType:PUSH TriggerID:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea
   Hash:edc9ea10e99dfc96f91ded6702a9032d801fafa9 Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/147164007 TriggerType:PUSH TriggerID:edc9ea10e99dfc96f91ded6702a9032d801fafa9
   Hash:edc9ea10e99dfc96f91ded6702a9032d801fafa9 Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4767 TriggerType:PUSH TriggerID:edc9ea10e99dfc96f91ded6702a9032d801fafa9
   Hash:09b4361d6279d3c1f03dcdd2a1ae6a3c59ac2058 Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/147316217 TriggerType:PUSH TriggerID:09b4361d6279d3c1f03dcdd2a1ae6a3c59ac2058
   Hash:09b4361d6279d3c1f03dcdd2a1ae6a3c59ac2058 Status:PENDING URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4802 TriggerType:PUSH TriggerID:09b4361d6279d3c1f03dcdd2a1ae6a3c59ac2058
   -->
   ## CI report:
   
   * 964b4874eaa647c908289dd841b660bc966fdb61 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142858913) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045) 
   * c324abe0418f58ce00aa3bb881400bea18a34d2b Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/143336203) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145) 
   * 76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143556983) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4191) 
   * edc9ea10e99dfc96f91ded6702a9032d801fafa9 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/147164007) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4767) 
   * 09b4361d6279d3c1f03dcdd2a1ae6a3c59ac2058 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/147316217) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4802) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] GJL closed pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
GJL closed pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] wangyang0918 commented on a change in pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
wangyang0918 commented on a change in pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#discussion_r363718014
 
 

 ##########
 File path: flink-end-to-end-tests/test-scripts/common_mesos_docker.sh
 ##########
 @@ -25,6 +25,8 @@ source "$(dirname "$0")"/common_docker.sh
 MAX_RETRY_SECONDS=120
 IMAGE_BUILD_RETRIES=5
 NODENAME=${NODENAME:-`hostname -f`}
+export MESOS_FLINK_DIR="/flink"
 
 Review comment:
   Do not need to export. `common_mesos_docker.sh` will be `source` in the mesos related tests.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] KarmaGYZ edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
KarmaGYZ edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-573974153
 
 
   > will we always run into permission issues
   
   Sorry, I'm not sure about this. At least so far, I could not figure out what happens in [FLINK-15377](https://issues.apache.org/jira/browse/FLINK-15377) cause the permission issue. It always runs well on my local environment(MacOS). But I think this PR could prevent that happening again.
   
   > Another approach I have found is described here
   
   It sounds good, I give a try on this approach. However, I meet `addgroup: The GID '20' is already in use` issue. It seems this approach does not run well with MacOS. I'm not sure about that.
   https://github.com/pyro-ppl/pyro/issues/700#issuecomment-359984295
   https://github.com/docker/docker-py/issues/1714
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] KarmaGYZ commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-581315998
 
 
   Thanks for your comments. That's really helpful. I've updated the PR and test it on Ubuntu 18 and MacOS. It works well in both systems.
   
   One minor concern, we may not redirect the error message of `rm -rf ${TEST_DATA_DIR}`. WDYT?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-570173870
 
 
   <!--
   Meta data
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/142858913 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:FAILURE URL:https://travis-ci.com/flink-ci/flink/builds/143336203 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   -->
   ## CI report:
   
   * 964b4874eaa647c908289dd841b660bc966fdb61 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142858913) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045) 
   * c324abe0418f58ce00aa3bb881400bea18a34d2b Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/143336203) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-570173870
 
 
   <!--
   Meta data
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/142858913 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:FAILURE URL:https://travis-ci.com/flink-ci/flink/builds/143336203 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   Hash:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/143556983 TriggerType:PUSH TriggerID:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea
   Hash:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4191 TriggerType:PUSH TriggerID:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea
   Hash:edc9ea10e99dfc96f91ded6702a9032d801fafa9 Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/147164007 TriggerType:PUSH TriggerID:edc9ea10e99dfc96f91ded6702a9032d801fafa9
   Hash:edc9ea10e99dfc96f91ded6702a9032d801fafa9 Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4767 TriggerType:PUSH TriggerID:edc9ea10e99dfc96f91ded6702a9032d801fafa9
   Hash:09b4361d6279d3c1f03dcdd2a1ae6a3c59ac2058 Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/147316217 TriggerType:PUSH TriggerID:09b4361d6279d3c1f03dcdd2a1ae6a3c59ac2058
   Hash:09b4361d6279d3c1f03dcdd2a1ae6a3c59ac2058 Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4802 TriggerType:PUSH TriggerID:09b4361d6279d3c1f03dcdd2a1ae6a3c59ac2058
   -->
   ## CI report:
   
   * 964b4874eaa647c908289dd841b660bc966fdb61 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142858913) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045) 
   * c324abe0418f58ce00aa3bb881400bea18a34d2b Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/143336203) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145) 
   * 76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143556983) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4191) 
   * edc9ea10e99dfc96f91ded6702a9032d801fafa9 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/147164007) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4767) 
   * 09b4361d6279d3c1f03dcdd2a1ae6a3c59ac2058 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/147316217) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4802) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] GJL commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
GJL commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-580211791
 
 
   Sorry, for the delay. I will take another look at the PR/your comments today.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-570173870
 
 
   <!--
   Meta data
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/142858913 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:FAILURE URL:https://travis-ci.com/flink-ci/flink/builds/143336203 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   Hash:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Status:UNKNOWN URL:TBD TriggerType:PUSH TriggerID:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea
   -->
   ## CI report:
   
   * 964b4874eaa647c908289dd841b660bc966fdb61 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142858913) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045) 
   * c324abe0418f58ce00aa3bb881400bea18a34d2b Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/143336203) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145) 
   * 76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] GJL edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
GJL edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-580285665
 
 
   > The chmod -R ugo+rw looks good to me, but can you ensure this command will not meet permission issue? 
   
   Adding something like this
   ```
   docker exec mesos-master bash -c "chmod -R ogu+rw ${FLINK_DIR}/log/ ${TEST_DATA_DIR}"
   ```
   to `cluster_shutdown()` should work (disclaimer: I haven't tested this on Linux but I can do it later or tomorrow). I don't see a permission issue here because by default these commands will be run by root (we do not use the `USER` command when building the docker image). 
   
   
   
   
   
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] GJL commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
GJL commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-574229434
 
 
   When we opened FLINK-15377, we discovered that that the mesos logs cannot be deleted due to permission problems (logs written in the container have a different owner than the host's current user id). Since we are also deleting the job's output that is from within the container, I was wondering why the test is currently passing on Travis at all. It turns out that we also fail to remove the job's output (`${TEST_DATA_DIR}/out/wc_out_mesos`). However, because the clean up code is executed as a `trap`, and we redirect `stderr` to `/dev/null` [1], the error is never visible on Travis. 
   
   Your PR works around the permission problem by copying the data from the container. The issues I see with this approach are:
   - `wait_job_terminal_state_mesos` uses a new strategy to poll whether the job has terminated (compared to the standalone mode)
   - `copy_logs_from_container` will not work if the container dies unexpectedly, making it hard to debug a certain class of bugs (mesos exiting prematurely)
   
   The approach described [here](https://vsupalov.com/docker-shared-permissions/) doesn't work on OS X due to the docker daemon running on a hypervisor. However, on OS X we apparently do not suffer from the permission issue.
   
   A simple way out of this might be to just create the directories in advance with the right permissions. For example, if we ran
   
   ```
   mkdir ${TEST_DATA_DIR}/out/ # run on the host, not in the container
   ```
   
   prior to submitting the job, we will be able to delete the `/out` directory later from the host (unless nested directories are created within the container). Another option I see is to run `chmod -R ugo+rw` at the end of the test from within the container against all files/directories that we need to delete later from the host.
   
   Let me know what you think.
   
   [1] https://github.com/apache/flink/blob/6f6fb43ca2f8413e81a1b19e77c5cf3101b7e61d/flink-end-to-end-tests/test-scripts/test-runner-common.sh#L107

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-570173870
 
 
   <!--
   Meta data
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:PENDING URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:PENDING URL:https://travis-ci.com/flink-ci/flink/builds/142858913 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   -->
   ## CI report:
   
   * 964b4874eaa647c908289dd841b660bc966fdb61 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/142858913) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] KarmaGYZ commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-584024373
 
 
   @GJL Hi, would you like to merge it?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] KarmaGYZ commented on a change in pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on a change in pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#discussion_r374446383
 
 

 ##########
 File path: flink-end-to-end-tests/test-scripts/common_mesos_docker.sh
 ##########
 @@ -72,7 +73,25 @@ function build_image() {
 function wait_job_terminal_state_mesos {
   local job=$1
   local expected_terminal_state=$2
-  wait_job_terminal_state $1 $2 "mesos-appmaster"
+  local successful_response_regex="\{.*\"application-status\":\"${expected_terminal_state}\".*\}"
+  local query_url="${REST_PROTOCOL}://${NODENAME}:8081/jobs/${job}/execution-result"
+  local TIMEOUT=120
+
+  echo "Waiting for job ($job) to reach terminal state $expected_terminal_state ..."
+
+  for i in $(seq 1 ${TIMEOUT}); do
+    QUERY_RESULT=$(curl "$query_url" 2> /dev/null || true)
+    # ensure the response adapts with the suceessful regex
 
 Review comment:
   Yes, it is. I create FLINK-15870 to track this. FYI.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] wangyang0918 commented on a change in pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
wangyang0918 commented on a change in pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#discussion_r363719407
 
 

 ##########
 File path: flink-end-to-end-tests/test-scripts/common_mesos_docker.sh
 ##########
 @@ -52,11 +54,19 @@ function start_flink_cluster_with_mesos() {
     set_config_key "jobmanager.rpc.address" "mesos-master"
     set_config_key "rest.address" "mesos-master"
 
-    docker exec -itd mesos-master bash -c "${FLINK_DIR}/bin/mesos-appmaster.sh -Dmesos.master=mesos-master:5050"
+    docker cp ${FLINK_DIR} mesos-master:$MESOS_FLINK_DIR
+    docker cp ${END_TO_END_DIR}/test-scripts mesos-master:$MESOS_END_TO_END_DIR
 
 Review comment:
   Why do we need to copy the `MESOS_END_TO_END_DIR` directory to docker container? If you want to use the data, just copy them.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] GJL edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
GJL edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-580285665
 
 
   > The chmod -R ugo+rw looks good to me, but can you ensure this command will not meet permission issue? 
   
   Adding something like this
   ```
   docker exec mesos-master bash -c "chmod -R ogu+rw ${FLINK_DIR}/log/"
   docker exec mesos-master bash -c "chmod -R ogu+rw ${TEST_DATA_DIR}"
   ```
   to `cluster_shutdown()` should work (disclaimer: I haven't tested this on Linux but I can do it later or tomorrow). I don't see a permission issue here because by default these commands will be run by root (we do not use the `USER` command when building the docker image). 
   
   
   
   
   
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] GJL commented on a change in pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
GJL commented on a change in pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#discussion_r372993579
 
 

 ##########
 File path: flink-end-to-end-tests/test-scripts/common_mesos_docker.sh
 ##########
 @@ -53,11 +55,22 @@ function start_flink_cluster_with_mesos() {
     set_config_key "jobmanager.rpc.address" "mesos-master"
     set_config_key "rest.address" "mesos-master"
 
-    docker exec -itd mesos-master bash -c "${FLINK_DIR}/bin/mesos-appmaster.sh -Dmesos.master=mesos-master:5050"
+    docker cp ${FLINK_DIR} mesos-master:$MESOS_FLINK_DIR
 
 Review comment:
   Anyways, if we can use _bind mounts_ again, this isn't needed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
flinkbot commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-570173870
 
 
   <!--
   Meta data
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:UNKNOWN URL:TBD TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   -->
   ## CI report:
   
   * 964b4874eaa647c908289dd841b660bc966fdb61 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-570173870
 
 
   <!--
   Meta data
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/142858913 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:FAILURE URL:https://travis-ci.com/flink-ci/flink/builds/143336203 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   Hash:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/143556983 TriggerType:PUSH TriggerID:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea
   Hash:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4191 TriggerType:PUSH TriggerID:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea
   Hash:edc9ea10e99dfc96f91ded6702a9032d801fafa9 Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/147164007 TriggerType:PUSH TriggerID:edc9ea10e99dfc96f91ded6702a9032d801fafa9
   Hash:edc9ea10e99dfc96f91ded6702a9032d801fafa9 Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4767 TriggerType:PUSH TriggerID:edc9ea10e99dfc96f91ded6702a9032d801fafa9
   Hash:09b4361d6279d3c1f03dcdd2a1ae6a3c59ac2058 Status:PENDING URL:https://travis-ci.com/flink-ci/flink/builds/147316217 TriggerType:PUSH TriggerID:09b4361d6279d3c1f03dcdd2a1ae6a3c59ac2058
   Hash:09b4361d6279d3c1f03dcdd2a1ae6a3c59ac2058 Status:PENDING URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4802 TriggerType:PUSH TriggerID:09b4361d6279d3c1f03dcdd2a1ae6a3c59ac2058
   -->
   ## CI report:
   
   * 964b4874eaa647c908289dd841b660bc966fdb61 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142858913) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045) 
   * c324abe0418f58ce00aa3bb881400bea18a34d2b Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/143336203) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145) 
   * 76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143556983) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4191) 
   * edc9ea10e99dfc96f91ded6702a9032d801fafa9 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/147164007) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4767) 
   * 09b4361d6279d3c1f03dcdd2a1ae6a3c59ac2058 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/147316217) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4802) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] KarmaGYZ edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
KarmaGYZ edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-574460501
 
 
   - Regarding the `wait_job_terminal_state_mesos`, I still prefer the REST way. It will free our test from the fabricated contract with LOG.
   - Regarding the `copy_logs_from_container`, yes, you're right.
   
   The `chmod -R ugo+rw` looks good to me, but can you ensure this command will not meet permission issue? Maybe we need the `sudo` privilege here. If that makes sense to you, I'll update the PR asap.
   
   @GJL 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-570173870
 
 
   <!--
   Meta data
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/142858913 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:FAILURE URL:https://travis-ci.com/flink-ci/flink/builds/143336203 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   Hash:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/143556983 TriggerType:PUSH TriggerID:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea
   Hash:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4191 TriggerType:PUSH TriggerID:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea
   Hash:edc9ea10e99dfc96f91ded6702a9032d801fafa9 Status:UNKNOWN URL:TBD TriggerType:PUSH TriggerID:edc9ea10e99dfc96f91ded6702a9032d801fafa9
   -->
   ## CI report:
   
   * 964b4874eaa647c908289dd841b660bc966fdb61 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142858913) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045) 
   * c324abe0418f58ce00aa3bb881400bea18a34d2b Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/143336203) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145) 
   * 76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143556983) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4191) 
   * edc9ea10e99dfc96f91ded6702a9032d801fafa9 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] KarmaGYZ commented on a change in pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on a change in pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#discussion_r363723277
 
 

 ##########
 File path: flink-end-to-end-tests/test-scripts/common_mesos_docker.sh
 ##########
 @@ -52,11 +54,19 @@ function start_flink_cluster_with_mesos() {
     set_config_key "jobmanager.rpc.address" "mesos-master"
     set_config_key "rest.address" "mesos-master"
 
-    docker exec -itd mesos-master bash -c "${FLINK_DIR}/bin/mesos-appmaster.sh -Dmesos.master=mesos-master:5050"
+    docker cp ${FLINK_DIR} mesos-master:$MESOS_FLINK_DIR
+    docker cp ${END_TO_END_DIR}/test-scripts mesos-master:$MESOS_END_TO_END_DIR
+    docker cp ${END_TO_END_DIR}/test-scripts mesos-slave:$MESOS_END_TO_END_DIR
+
+    docker exec -itd mesos-master bash -c "${MESOS_FLINK_DIR}/bin/mesos-appmaster.sh -Dmesos.master=mesos-master:5050"
     wait_rest_endpoint_up "http://${NODENAME}:8081/taskmanagers" "Dispatcher" "\{\"taskmanagers\":\[.*\]\}"
     return 0
 }
 
+function copy_logs_from_container () {
 
 Review comment:
   If we copy logs to `${FLINK_DIR}/log`, the existing mechanism will help us to clean up too. BTW, `post_test_validation` will check the error in it after the test passed or failed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-570173870
 
 
   <!--
   Meta data
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/142858913 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:UNKNOWN URL:TBD TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   -->
   ## CI report:
   
   * 964b4874eaa647c908289dd841b660bc966fdb61 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142858913) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045) 
   * c324abe0418f58ce00aa3bb881400bea18a34d2b UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] GJL commented on a change in pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
GJL commented on a change in pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#discussion_r374127425
 
 

 ##########
 File path: flink-end-to-end-tests/test-scripts/common_mesos_docker.sh
 ##########
 @@ -72,7 +73,25 @@ function build_image() {
 function wait_job_terminal_state_mesos {
   local job=$1
   local expected_terminal_state=$2
-  wait_job_terminal_state $1 $2 "mesos-appmaster"
+  local successful_response_regex="\{.*\"application-status\":\"${expected_terminal_state}\".*\}"
+  local query_url="${REST_PROTOCOL}://${NODENAME}:8081/jobs/${job}/execution-result"
+  local TIMEOUT=120
+
+  echo "Waiting for job ($job) to reach terminal state $expected_terminal_state ..."
+
+  for i in $(seq 1 ${TIMEOUT}); do
+    QUERY_RESULT=$(curl "$query_url" 2> /dev/null || true)
+    # ensure the response adapts with the suceessful regex
 
 Review comment:
   Not sure how useful this comment is. Also, typo: *successful*
   
   If this *hotfix* isn't needed for FLINK-15417, I would prefer to create a separate issue and apply this to standalone mode as well. The changes definitely go beyond a hotfix commit. WDYT?
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] GJL commented on a change in pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
GJL commented on a change in pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#discussion_r365927501
 
 

 ##########
 File path: flink-end-to-end-tests/test-scripts/common_mesos_docker.sh
 ##########
 @@ -53,11 +55,22 @@ function start_flink_cluster_with_mesos() {
     set_config_key "jobmanager.rpc.address" "mesos-master"
     set_config_key "rest.address" "mesos-master"
 
-    docker exec -itd mesos-master bash -c "${FLINK_DIR}/bin/mesos-appmaster.sh -Dmesos.master=mesos-master:5050"
+    docker cp ${FLINK_DIR} mesos-master:$MESOS_FLINK_DIR
 
 Review comment:
   Why not bundle Flink and the test data inside the image?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] GJL commented on a change in pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
GJL commented on a change in pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#discussion_r374127425
 
 

 ##########
 File path: flink-end-to-end-tests/test-scripts/common_mesos_docker.sh
 ##########
 @@ -72,7 +73,25 @@ function build_image() {
 function wait_job_terminal_state_mesos {
   local job=$1
   local expected_terminal_state=$2
-  wait_job_terminal_state $1 $2 "mesos-appmaster"
+  local successful_response_regex="\{.*\"application-status\":\"${expected_terminal_state}\".*\}"
+  local query_url="${REST_PROTOCOL}://${NODENAME}:8081/jobs/${job}/execution-result"
+  local TIMEOUT=120
+
+  echo "Waiting for job ($job) to reach terminal state $expected_terminal_state ..."
+
+  for i in $(seq 1 ${TIMEOUT}); do
+    QUERY_RESULT=$(curl "$query_url" 2> /dev/null || true)
+    # ensure the response adapts with the suceessful regex
 
 Review comment:
   Not sure how useful this comment is. Also, typo: *successful*
   
   If this isn't needed for FLINK-15417, I would prefer to create a separate issue and apply this to standalone mode as well. WDYT?
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-570173870
 
 
   <!--
   Meta data
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/142858913 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   -->
   ## CI report:
   
   * 964b4874eaa647c908289dd841b660bc966fdb61 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142858913) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] KarmaGYZ edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
KarmaGYZ edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-581315998
 
 
   @GJL 
   Thanks for your comments. That's really helpful. I've updated the PR and test it on Ubuntu 18 and MacOS. It works well in both systems.
   
   One minor concern, we may not redirect the error message of `rm -rf ${TEST_DATA_DIR}`. WDYT?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-570173870
 
 
   <!--
   Meta data
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/142858913 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:FAILURE URL:https://travis-ci.com/flink-ci/flink/builds/143336203 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   Hash:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Status:PENDING URL:https://travis-ci.com/flink-ci/flink/builds/143556983 TriggerType:PUSH TriggerID:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea
   Hash:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Status:PENDING URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4191 TriggerType:PUSH TriggerID:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea
   -->
   ## CI report:
   
   * 964b4874eaa647c908289dd841b660bc966fdb61 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142858913) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045) 
   * c324abe0418f58ce00aa3bb881400bea18a34d2b Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/143336203) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145) 
   * 76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/143556983) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4191) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] wangyang0918 commented on a change in pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
wangyang0918 commented on a change in pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#discussion_r364120873
 
 

 ##########
 File path: flink-end-to-end-tests/test-scripts/common_mesos_docker.sh
 ##########
 @@ -52,11 +54,19 @@ function start_flink_cluster_with_mesos() {
     set_config_key "jobmanager.rpc.address" "mesos-master"
     set_config_key "rest.address" "mesos-master"
 
-    docker exec -itd mesos-master bash -c "${FLINK_DIR}/bin/mesos-appmaster.sh -Dmesos.master=mesos-master:5050"
+    docker cp ${FLINK_DIR} mesos-master:$MESOS_FLINK_DIR
+    docker cp ${END_TO_END_DIR}/test-scripts mesos-master:$MESOS_END_TO_END_DIR
+    docker cp ${END_TO_END_DIR}/test-scripts mesos-slave:$MESOS_END_TO_END_DIR
+
+    docker exec -itd mesos-master bash -c "${MESOS_FLINK_DIR}/bin/mesos-appmaster.sh -Dmesos.master=mesos-master:5050"
     wait_rest_endpoint_up "http://${NODENAME}:8081/taskmanagers" "Dispatcher" "\{\"taskmanagers\":\[.*\]\}"
     return 0
 }
 
+function copy_logs_from_container () {
 
 Review comment:
   If the logs could show up when test failed and be cleaned up when succeed, it will make sense to me.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] KarmaGYZ commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-574460501
 
 
   - Regarding the `wait_job_terminal_state_mesos`, I still prefer the REST way. It will free our test from the fabricated contract with LOG.
   - Regarding the `copy_logs_from_container`, yes, you're right.
   
   The `chmod -R ugo+rw` looks good to me, but can you ensure this command will not meet permission issue? Maybe we need the `sudo` privilege here. If that makes sense to you, I'll update the PR asap.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] KarmaGYZ commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-573974153
 
 
   > will we always run into permission issues
   
   Sorry, I'm not sure about this. At least so far, I could not figure out what happens in [FLINK-15377](https://issues.apache.org/jira/browse/FLINK-15377) cause the permission issue. It always runs well on my local environment(MacOS). But I think this PR could prevent that happening again.
   
   > Another approach I have found is described here
   
   It sounds good, I give a try on this approach. However, I meet `addgroup: The GID `20' is already in use` issue. It seems this approach does not run well with MacOS. I'm not sure about that.
   https://github.com/pyro-ppl/pyro/issues/700#issuecomment-359984295
   https://github.com/docker/docker-py/issues/1714
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] GJL commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
GJL commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-584030459
 
 
   Got caught up with the 1.10 release again. Will try to merge it today. Thanks for your patience.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-570173870
 
 
   <!--
   Meta data
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/142858913 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:FAILURE URL:https://travis-ci.com/flink-ci/flink/builds/143336203 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   Hash:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/143556983 TriggerType:PUSH TriggerID:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea
   Hash:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4191 TriggerType:PUSH TriggerID:76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea
   -->
   ## CI report:
   
   * 964b4874eaa647c908289dd841b660bc966fdb61 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142858913) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045) 
   * c324abe0418f58ce00aa3bb881400bea18a34d2b Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/143336203) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145) 
   * 76d658e335703f0e1a6c3b6bd4d3c1ef2d29f7ea Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143556983) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4191) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-570173870
 
 
   <!--
   Meta data
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:964b4874eaa647c908289dd841b660bc966fdb61 Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/142858913 TriggerType:PUSH TriggerID:964b4874eaa647c908289dd841b660bc966fdb61
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:PENDING URL:https://travis-ci.com/flink-ci/flink/builds/143336203 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   Hash:c324abe0418f58ce00aa3bb881400bea18a34d2b Status:PENDING URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145 TriggerType:PUSH TriggerID:c324abe0418f58ce00aa3bb881400bea18a34d2b
   -->
   ## CI report:
   
   * 964b4874eaa647c908289dd841b660bc966fdb61 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142858913) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4045) 
   * c324abe0418f58ce00aa3bb881400bea18a34d2b Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/143336203) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4145) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run travis` re-run the last Travis build
    - `@flinkbot run azure` re-run the last Azure build
   </details>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] KarmaGYZ commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-572018093
 
 
   Thanks for the review @wangyang0918 . 
   I've addressed your comments. Please take a look~

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] GJL commented on a change in pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
GJL commented on a change in pull request #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#discussion_r365927501
 
 

 ##########
 File path: flink-end-to-end-tests/test-scripts/common_mesos_docker.sh
 ##########
 @@ -53,11 +55,22 @@ function start_flink_cluster_with_mesos() {
     set_config_key "jobmanager.rpc.address" "mesos-master"
     set_config_key "rest.address" "mesos-master"
 
-    docker exec -itd mesos-master bash -c "${FLINK_DIR}/bin/mesos-appmaster.sh -Dmesos.master=mesos-master:5050"
+    docker cp ${FLINK_DIR} mesos-master:$MESOS_FLINK_DIR
 
 Review comment:
   Why not bundle Flink and the test data inside the image? My concern is that we are treating the container like a VM. However, strictly speaking a container should be immutable, and all writes should go to a volume or bind mount.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] KarmaGYZ commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…

Posted by GitBox <gi...@apache.org>.
KarmaGYZ commented on issue #10746: [FLINK-15417] Remove the docker volume or mount when starting Mesos e…
URL: https://github.com/apache/flink/pull/10746#issuecomment-573521081
 
 
   Kindly ping @GJL 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services