You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by "tejaswini-imply (via GitHub)" <gi...@apache.org> on 2023/04/05 05:40:30 UTC
[GitHub] [druid] tejaswini-imply opened a new pull request, #14031: Clean docker environment before Revised ITs test in GHA
tejaswini-imply opened a new pull request, #14031:
URL: https://github.com/apache/druid/pull/14031
Revised ITs fail to run if zookeeper, metadata docker containers exist already (Could be remnants from previous runs on the runner after unsuccessful cleanup job) with the following error.
```
Creating metadata ...
Creating zookeeper ...
Creating zookeeper ... error
ERROR: for zookeeper Cannot create container for service zookeeper: Conflict. The container name "/zookeeper" is already in use by container "9c599e6f36f5cda67d98c39f5c3397b967b4d52e432e1c7949268a961c06ddac". You have to remove (or rename) that container to be able to reuse that name.
Creating metadata ... error
ERROR: for metadata Cannot create container for service metadata: Conflict. The container name "/metadata" is already in use by container "df90f24f5c561a9c8f1e1d2b4288c18f93b269c6181ae6f439e94c947c27b4b1". You have to remove (or rename) that container to be able to reuse that name.
ERROR: for zookeeper Cannot create container for service zookeeper: Conflict. The container name "/zookeeper" is already in use by container "9c599e6f36f5cda67d98c39f5c3397b967b4d52e432e1c7949268a961c06ddac". You have to remove (or rename) that container to be able to reuse that name.
ERROR: for metadata Cannot create container for service metadata: Conflict. The container name "/metadata" is already in use by container "df90f24f5c561a9c8f1e1d2b4288c18f93b269c6181ae6f439e94c947c27b4b1". You have to remove (or rename) that container to be able to reuse that name.
Encountered errors while bringing up the project.
```
This PR cleans up any existing docker containers and prunes the volumes before the test run.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] imply-elliott commented on a diff in pull request #14031: Clean docker environment before Revised ITs test in GHA
Posted by "imply-elliott (via GitHub)" <gi...@apache.org>.
imply-elliott commented on code in PR #14031:
URL: https://github.com/apache/druid/pull/14031#discussion_r1158810493
##########
.github/workflows/reusable-revised-its.yml:
##########
@@ -117,8 +117,11 @@ jobs:
echo $DRUID_IT_IMAGE_NAME
docker save "$DRUID_IT_IMAGE_NAME" | gzip > druid-container-jdk${{ inputs.build_jdk }}.tar.gz
- - name: Load docker image
+ - name: Clean docker environment and Load docker image
run: |
+ echo "Stopping and pruning any remnant docker containers from previous run"
+ docker ps -aq | xargs -r docker rm -f
+ docker system prune -af --volumes
Review Comment:
I had a PR open for this here: https://github.com/apache/druid/pull/13940
If you read Paul's comment here https://github.com/apache/druid/pull/13940/files#r1139411707, he feels the same way. I agree, it should probably be rolled in. It's also less heavy handed than this, since this removes ALL containers on the runner, which includes some pre-cached ones that save a little time. Honestly though, the pre-cached ones are not that useful for us and the cleanup only takes a few seconds, so I didn't put much weight on being less heavy handed for the older ITs, given they're not long for the world.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] abhishekagarwal87 commented on a diff in pull request #14031: Clean docker environment before Revised ITs test in GHA
Posted by "abhishekagarwal87 (via GitHub)" <gi...@apache.org>.
abhishekagarwal87 commented on code in PR #14031:
URL: https://github.com/apache/druid/pull/14031#discussion_r1158692273
##########
.github/workflows/reusable-revised-its.yml:
##########
@@ -117,8 +117,11 @@ jobs:
echo $DRUID_IT_IMAGE_NAME
docker save "$DRUID_IT_IMAGE_NAME" | gzip > druid-container-jdk${{ inputs.build_jdk }}.tar.gz
- - name: Load docker image
+ - name: Clean docker environment and Load docker image
run: |
+ echo "Stopping and pruning any remnant docker containers from previous run"
+ docker ps -aq | xargs -r docker rm -f
+ docker system prune -af --volumes
Review Comment:
could this be done in the script itself? So that anyone running an IT test locally would not run into this problem as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] imply-elliott commented on a diff in pull request #14031: Clean docker environment before Revised ITs test in GHA
Posted by "imply-elliott (via GitHub)" <gi...@apache.org>.
imply-elliott commented on code in PR #14031:
URL: https://github.com/apache/druid/pull/14031#discussion_r1158810493
##########
.github/workflows/reusable-revised-its.yml:
##########
@@ -117,8 +117,11 @@ jobs:
echo $DRUID_IT_IMAGE_NAME
docker save "$DRUID_IT_IMAGE_NAME" | gzip > druid-container-jdk${{ inputs.build_jdk }}.tar.gz
- - name: Load docker image
+ - name: Clean docker environment and Load docker image
run: |
+ echo "Stopping and pruning any remnant docker containers from previous run"
+ docker ps -aq | xargs -r docker rm -f
+ docker system prune -af --volumes
Review Comment:
I had a PR open for this here: https://github.com/apache/druid/pull/13940
If you read Paul's comment here https://github.com/apache/druid/pull/13940/files#r1139411707, he feels the same way. I agree, it should probably be rolled in. It's also less heavy handed than this, since this removes ALL containers on the runner, which includes some pre-cached ones that save a little time. Honestly though, they're not that useful for us and the cleanup is very brief, so I didn't put much weight on being less heavy handed for the older ITs, given they're not long for the world.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] tejaswini-imply commented on a diff in pull request #14031: Clean docker environment before Revised ITs test in GHA
Posted by "tejaswini-imply (via GitHub)" <gi...@apache.org>.
tejaswini-imply commented on code in PR #14031:
URL: https://github.com/apache/druid/pull/14031#discussion_r1158814900
##########
.github/workflows/reusable-revised-its.yml:
##########
@@ -117,8 +117,11 @@ jobs:
echo $DRUID_IT_IMAGE_NAME
docker save "$DRUID_IT_IMAGE_NAME" | gzip > druid-container-jdk${{ inputs.build_jdk }}.tar.gz
- - name: Load docker image
+ - name: Clean docker environment and Load docker image
run: |
+ echo "Stopping and pruning any remnant docker containers from previous run"
+ docker ps -aq | xargs -r docker rm -f
+ docker system prune -af --volumes
Review Comment:
Oh, I'm sorry, I didn't know you had an existing PR. I'll remove this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] tejaswini-imply closed pull request #14031: Clean docker environment before Revised ITs test in GHA
Posted by "tejaswini-imply (via GitHub)" <gi...@apache.org>.
tejaswini-imply closed pull request #14031: Clean docker environment before Revised ITs test in GHA
URL: https://github.com/apache/druid/pull/14031
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] imply-elliott commented on a diff in pull request #14031: Clean docker environment before Revised ITs test in GHA
Posted by "imply-elliott (via GitHub)" <gi...@apache.org>.
imply-elliott commented on code in PR #14031:
URL: https://github.com/apache/druid/pull/14031#discussion_r1158872629
##########
.github/workflows/reusable-revised-its.yml:
##########
@@ -117,8 +117,11 @@ jobs:
echo $DRUID_IT_IMAGE_NAME
docker save "$DRUID_IT_IMAGE_NAME" | gzip > druid-container-jdk${{ inputs.build_jdk }}.tar.gz
- - name: Load docker image
+ - name: Clean docker environment and Load docker image
run: |
+ echo "Stopping and pruning any remnant docker containers from previous run"
+ docker ps -aq | xargs -r docker rm -f
+ docker system prune -af --volumes
Review Comment:
No problem, it's still an "open issue", so let's see if we can get a final call one way or the other. :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org