You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Zoltan Garaguly (Code Review)" <ge...@cloudera.org> on 2020/05/13 08:16:02 UTC
[Impala-ASF-CR] IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Zoltan Garaguly has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15898
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
......................................................................
IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Copy logs of components hdfs, yarn, kudu from folder
testdata/cluster/cdh<version-number>/node-<node-id>/var/log/
to folder logs/cluster/
Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
---
M docker/entrypoint.sh
1 file changed, 26 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/98/15898/2
--
To view, visit http://gerrit.cloudera.org:8080/15898
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Gerrit-Change-Number: 15898
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
[Impala-ASF-CR] IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Posted by "Zoltan Garaguly (Code Review)" <ge...@cloudera.org>.
Hello Laszlo Gaal, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15898
to look at the new patch set (#4).
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
......................................................................
IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Copy logs of components hdfs, yarn, kudu from folder
testdata/cluster/cdh<version-number>/node-<node-id>/var/log/
to folder logs/cluster/
Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
---
M docker/entrypoint.sh
1 file changed, 29 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/98/15898/4
--
To view, visit http://gerrit.cloudera.org:8080/15898
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Gerrit-Change-Number: 15898
Gerrit-PatchSet: 4
Gerrit-Owner: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Zoltan Garaguly <zg...@cloudera.com>
[Impala-ASF-CR] IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15898 )
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
......................................................................
Patch Set 4: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/15898
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Gerrit-Change-Number: 15898
Gerrit-PatchSet: 4
Gerrit-Owner: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Comment-Date: Wed, 20 May 2020 14:50:22 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15898 )
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/6049/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/15898
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Gerrit-Change-Number: 15898
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 May 2020 08:38:04 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15898 )
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
......................................................................
Patch Set 4:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/6112/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/15898
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Gerrit-Change-Number: 15898
Gerrit-PatchSet: 4
Gerrit-Owner: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Comment-Date: Wed, 20 May 2020 09:19:12 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/15898 )
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
......................................................................
Patch Set 6:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/15898/6/docker/entrypoint.sh
File docker/entrypoint.sh:
http://gerrit.cloudera.org:8080/#/c/15898/6/docker/entrypoint.sh@304
PS6, Line 304: copy_cluster_logs
One thing I noticed when I ran this is that this code doesn't execute if any of the commands above fail. That must be because we use "set -e" when executing these functions.
One way to fix that would be to follow the structure that test_suite() uses. In other words:
ret = 0
if ! ./buildall.sh -noclean -format -testdata -notests; then
ret = 1
echo "Build + dataload failed!"
else
echo "Build + dataload succeeded!"
fi
Then, below that point, use ret to decide what needs to do. If ret is 0, then do all the current steps. If ret is 1, skip building parquet-reader/impala-profile-tool and the hardlinking duplicate HDFS blocks, but otherwise do basically the same things. It looks like test_suite() uses a lot of "cmd || true" structure to avoid exiting, then it returns ret at the end.
--
To view, visit http://gerrit.cloudera.org:8080/15898
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Gerrit-Change-Number: 15898
Gerrit-PatchSet: 6
Gerrit-Owner: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Comment-Date: Thu, 22 Jul 2021 03:34:28 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15898 )
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
......................................................................
Patch Set 8:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7440/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/15898
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Gerrit-Change-Number: 15898
Gerrit-PatchSet: 8
Gerrit-Owner: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Comment-Date: Tue, 31 Aug 2021 00:51:49 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Posted by "Zoltan Garaguly (Code Review)" <ge...@cloudera.org>.
Zoltan Garaguly has posted comments on this change. ( http://gerrit.cloudera.org:8080/15898 )
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/15898/1/docker/entrypoint.sh
File docker/entrypoint.sh:
http://gerrit.cloudera.org:8080/#/c/15898/1/docker/entrypoint.sh@356
PS1, Line 356: CDH_VERSION=`echo $x | sed "s#testdata/cluster/\(.*\)/node-.*#\1#"`
> The readability of the sed scripts could be increased by changing the patte
Done
--
To view, visit http://gerrit.cloudera.org:8080/15898
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Gerrit-Change-Number: 15898
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 May 2020 08:16:50 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Posted by "Zoltan Garaguly (Code Review)" <ge...@cloudera.org>.
Hello Laszlo Gaal, Joe McDonnell, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15898
to look at the new patch set (#7).
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
......................................................................
IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Inside docker container copy logs of cluster components hdfs, yarn, kudu
from folder testdata/cluster/cdh<version-number>/node-<node-id>/var/log/
to folder logs/cluster/
Testing:
- running docker-based tests and checked that minicluster logs are preserved and archived
- test if minicluster logs get copied also in case when something gets wrong during build
Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
---
M docker/entrypoint.sh
1 file changed, 66 insertions(+), 21 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/98/15898/7
--
To view, visit http://gerrit.cloudera.org:8080/15898
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Gerrit-Change-Number: 15898
Gerrit-PatchSet: 7
Gerrit-Owner: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Zoltan Garaguly <zg...@cloudera.com>
[Impala-ASF-CR] IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15898 )
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
......................................................................
Patch Set 3:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5821/ DRY_RUN=true
--
To view, visit http://gerrit.cloudera.org:8080/15898
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Gerrit-Change-Number: 15898
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 May 2020 16:31:38 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15898 )
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
......................................................................
Patch Set 4:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5864/ DRY_RUN=true
--
To view, visit http://gerrit.cloudera.org:8080/15898
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Gerrit-Change-Number: 15898
Gerrit-PatchSet: 4
Gerrit-Owner: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Comment-Date: Wed, 20 May 2020 09:33:41 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Posted by "Laszlo Gaal (Code Review)" <ge...@cloudera.org>.
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/15898 )
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
......................................................................
Patch Set 3:
(2 comments)
Went through the logic; found a preexisting failure case not even described in the original ticket.
http://gerrit.cloudera.org:8080/#/c/15898/3/docker/entrypoint.sh
File docker/entrypoint.sh:
http://gerrit.cloudera.org:8080/#/c/15898/3/docker/entrypoint.sh@319
PS3, Line 319: copy_cluster_logs
I'd recommend moving this line before L317: according to its comments, that line attempts to remove the very symlinks that copy_cluster_logs" is trying to save for later (although the exact flag and predicate combination seems to be wrong there).
http://gerrit.cloudera.org:8080/#/c/15898/3/docker/entrypoint.sh@319
PS3, Line 319: copy_cluster_logs
There is also another problem here (not covered in the original ticket): For each instantiation of the container (whether for build or test purposes) the driver script test_with_docker.py mounts a different log directory at the containers "/log" mount point (see L580-581: https://github.com/apache/impala/blob/master/docker/test-with-docker.py#L580, L655, L815 https://github.com/apache/impala/blob/master/docker/test-with-docker.py#L815). This means anything created there from within the container will not be present in the log tree of a different container run, unless there is a mechanism in test-with-docker.py that replicates or recreates the artifact.
Add to this that the cluster log symlinks are created by testdata/cluster/admin in the function create_cluster(), which is run only in the build container, but not in any of the test containers.
This has the end result that none of the test containers will actually contain the symlinks from /logs/cluster to the node logs, because the test containers run only testdata/cluster/admin start_cluster(), which does not ensure that the symlinks are present.
I see two ways out of this.
a) either fix testdata/cluster/admin so that start_cluster() (re)creates these symlink if they are gone
b) or fix entrypoint.sh to the same effect.
Once this is fixed, then the call to copy_cluster_logs should also be invoked from boot_container() (or possibly from test_suite() ) to ensure that the symlinks are there to capture the log files, so that clpy_cluster_logs can later pick them up and archive them.
--
To view, visit http://gerrit.cloudera.org:8080/15898
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Gerrit-Change-Number: 15898
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Comment-Date: Mon, 18 May 2020 17:40:36 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15898 )
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
......................................................................
IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Inside docker container copy logs of cluster components hdfs, yarn, kudu
from folder testdata/cluster/cdh<version-number>/node-<node-id>/var/log/
to folder logs/cluster/
Testing:
- running docker-based tests and checked that minicluster logs are preserved and archived
- test if minicluster logs get copied also in case when something gets wrong during build
Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Reviewed-on: http://gerrit.cloudera.org:8080/15898
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M docker/entrypoint.sh
1 file changed, 66 insertions(+), 21 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/15898
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Gerrit-Change-Number: 15898
Gerrit-PatchSet: 9
Gerrit-Owner: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Zoltan Garaguly <zg...@cloudera.com>
[Impala-ASF-CR] IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15898 )
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
......................................................................
Patch Set 7:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/9385/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/15898
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Gerrit-Change-Number: 15898
Gerrit-PatchSet: 7
Gerrit-Owner: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Comment-Date: Fri, 27 Aug 2021 12:48:21 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15898 )
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
......................................................................
Patch Set 8: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/15898
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Gerrit-Change-Number: 15898
Gerrit-PatchSet: 8
Gerrit-Owner: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Comment-Date: Tue, 31 Aug 2021 00:51:48 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15898 )
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
......................................................................
Patch Set 2:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5817/ DRY_RUN=true
--
To view, visit http://gerrit.cloudera.org:8080/15898
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Gerrit-Change-Number: 15898
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 May 2020 08:18:07 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Posted by "Zoltan Garaguly (Code Review)" <ge...@cloudera.org>.
Zoltan Garaguly has posted comments on this change. ( http://gerrit.cloudera.org:8080/15898 )
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
......................................................................
Patch Set 4:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/15898/3/docker/entrypoint.sh
File docker/entrypoint.sh:
http://gerrit.cloudera.org:8080/#/c/15898/3/docker/entrypoint.sh@319
PS3, Line 319: find /logs -xtype
> There is also another problem here (not covered in the original ticket): Fo
I agree with the necessity to add copy_cluster_logs call to test_suite() function as well in order to have the cluster logs for test containers too.
On the other hand I do not understand the point of creating symlinks in test containers because the goal of copy_cluster_logs function is exactly to get rid of this symlink and copy the original content (the cluster logs) in the folder where the symlink was before. The copy happens also in case when symlink was not there.
--
To view, visit http://gerrit.cloudera.org:8080/15898
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Gerrit-Change-Number: 15898
Gerrit-PatchSet: 4
Gerrit-Owner: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Comment-Date: Wed, 20 May 2020 08:27:42 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/15898 )
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
......................................................................
Patch Set 5:
I'm running the docker based tests with this to look at the output, then I'll review.
--
To view, visit http://gerrit.cloudera.org:8080/15898
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Gerrit-Change-Number: 15898
Gerrit-PatchSet: 5
Gerrit-Owner: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Comment-Date: Tue, 09 Mar 2021 00:20:59 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15898 )
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
......................................................................
Patch Set 5:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/8144/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/15898
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Gerrit-Change-Number: 15898
Gerrit-PatchSet: 5
Gerrit-Owner: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 Feb 2021 12:52:37 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Posted by "Zoltan Garaguly (Code Review)" <ge...@cloudera.org>.
Hello Laszlo Gaal, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15898
to look at the new patch set (#5).
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
......................................................................
IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Inside docker container copy logs of cluster components hdfs, yarn, kudu
from folder testdata/cluster/cdh<version-number>/node-<node-id>/var/log/
to folder logs/cluster/
Testing:
- running docker-based tests and checked that minicluster logs are preserved and archived
Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
---
M docker/entrypoint.sh
1 file changed, 29 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/98/15898/5
--
To view, visit http://gerrit.cloudera.org:8080/15898
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Gerrit-Change-Number: 15898
Gerrit-PatchSet: 5
Gerrit-Owner: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Zoltan Garaguly <zg...@cloudera.com>
[Impala-ASF-CR] IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Posted by "Zoltan Garaguly (Code Review)" <ge...@cloudera.org>.
Zoltan Garaguly has posted comments on this change. ( http://gerrit.cloudera.org:8080/15898 )
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
......................................................................
Patch Set 6:
I rebased and re-tested the fix.
Could you please review it?
--
To view, visit http://gerrit.cloudera.org:8080/15898
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Gerrit-Change-Number: 15898
Gerrit-PatchSet: 6
Gerrit-Owner: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Comment-Date: Tue, 13 Jul 2021 09:00:25 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15898 )
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
......................................................................
Patch Set 8: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/15898
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Gerrit-Change-Number: 15898
Gerrit-PatchSet: 8
Gerrit-Owner: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Comment-Date: Tue, 31 Aug 2021 06:58:33 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15898 )
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
......................................................................
Patch Set 3: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/15898
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Gerrit-Change-Number: 15898
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 May 2020 21:24:03 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/15898 )
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
......................................................................
Patch Set 7: Code-Review+2
Thanks for working on this
--
To view, visit http://gerrit.cloudera.org:8080/15898
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Gerrit-Change-Number: 15898
Gerrit-PatchSet: 7
Gerrit-Owner: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Comment-Date: Tue, 31 Aug 2021 00:50:19 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Posted by "Zoltan Garaguly (Code Review)" <ge...@cloudera.org>.
Zoltan Garaguly has posted comments on this change. ( http://gerrit.cloudera.org:8080/15898 )
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
......................................................................
Patch Set 7:
(1 comment)
Thank you for your comments.
I fixed the script to save minicluster logs even in case something gets wrong during the build, dataload or any other process in build_impdev phase.
http://gerrit.cloudera.org:8080/#/c/15898/6/docker/entrypoint.sh
File docker/entrypoint.sh:
http://gerrit.cloudera.org:8080/#/c/15898/6/docker/entrypoint.sh@304
PS6, Line 304: # and can confuse
> One thing I noticed when I ran this is that this code doesn't execute if an
Done
--
To view, visit http://gerrit.cloudera.org:8080/15898
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Gerrit-Change-Number: 15898
Gerrit-PatchSet: 7
Gerrit-Owner: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Comment-Date: Fri, 27 Aug 2021 12:34:18 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15898 )
Change subject: IMPALA-8680: Docker-based tests fail to archive the minicluster component logs
......................................................................
Patch Set 2: Verified-1
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5817/
--
To view, visit http://gerrit.cloudera.org:8080/15898
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23e25d42992cec47c593dc388bcf0bcef828c05e
Gerrit-Change-Number: 15898
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Zoltan Garaguly <zg...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 May 2020 13:37:05 +0000
Gerrit-HasComments: No