You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Laszlo Gaal (Code Review)" <ge...@cloudera.org> on 2021/01/20 15:13:42 UTC
[Impala-ASF-CR] IMPALA-10448: Build impala-profile-tool early for Docker-based tests
Laszlo Gaal has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16965
Change subject: IMPALA-10448: Build impala-profile-tool early for Docker-based tests
......................................................................
IMPALA-10448: Build impala-profile-tool early for Docker-based tests
impala-profile-tool is a new dependency for end-to-end tests.
The tool is built together with all the other backend tests
(so the buildall.sh flag '-notests' can turn off building it), it is
actually used in the parallel phase of end-to-end tests.
This means a problem for Docker-based builds for the following reasons:
- Docker-based tests run BE, FE and various phases of the EE test in
separate Docker containers for parallel executions
- Test binaries are only built inside the container running BE tests to
cut down on the build time and the size of the Docker image that all test
containers are based on.
- This means that the EE_TEST_PARALLEL container will miss the tool
required for running test designed to test it.
The solution is to build the tool early, at the end of the build phase
running in the build container. There is already another such tool built
there (parquet-reader) for similar reason, so just add
impala-profile-tool to the same 'make' command there.
Tested by running BE_TEST and EE_TEST_PARALLEL phases in a Docker-based
build.
Change-Id: I60e78ea883f3057c59a345feca38ef08a7f6a0b8
---
M docker/entrypoint.sh
1 file changed, 1 insertion(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/65/16965/1
--
To view, visit http://gerrit.cloudera.org:8080/16965
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I60e78ea883f3057c59a345feca38ef08a7f6a0b8
Gerrit-Change-Number: 16965
Gerrit-PatchSet: 1
Gerrit-Owner: Laszlo Gaal <la...@cloudera.com>
[Impala-ASF-CR] IMPALA-10448: Build impala-profile-tool early for Docker-based tests
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16965 )
Change subject: IMPALA-10448: Build impala-profile-tool early for Docker-based tests
......................................................................
Patch Set 2:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6851/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/16965
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I60e78ea883f3057c59a345feca38ef08a7f6a0b8
Gerrit-Change-Number: 16965
Gerrit-PatchSet: 2
Gerrit-Owner: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 22 Jan 2021 14:59:57 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10448: Build impala-profile-tool early for Docker-based tests
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16965 )
Change subject: IMPALA-10448: Build impala-profile-tool early for Docker-based tests
......................................................................
Patch Set 2: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/16965
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I60e78ea883f3057c59a345feca38ef08a7f6a0b8
Gerrit-Change-Number: 16965
Gerrit-PatchSet: 2
Gerrit-Owner: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 22 Jan 2021 14:59:56 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10448: Build impala-profile-tool early for Docker-based tests
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16965 )
Change subject: IMPALA-10448: Build impala-profile-tool early for Docker-based tests
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/8029/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/16965
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I60e78ea883f3057c59a345feca38ef08a7f6a0b8
Gerrit-Change-Number: 16965
Gerrit-PatchSet: 1
Gerrit-Owner: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 20 Jan 2021 15:35:30 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10448: Build impala-profile-tool early for Docker-based tests
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16965 )
Change subject: IMPALA-10448: Build impala-profile-tool early for Docker-based tests
......................................................................
Patch Set 2: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/16965
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I60e78ea883f3057c59a345feca38ef08a7f6a0b8
Gerrit-Change-Number: 16965
Gerrit-PatchSet: 2
Gerrit-Owner: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 22 Jan 2021 20:39:57 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10448: Build impala-profile-tool early for Docker-based tests
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16965 )
Change subject: IMPALA-10448: Build impala-profile-tool early for Docker-based tests
......................................................................
IMPALA-10448: Build impala-profile-tool early for Docker-based tests
impala-profile-tool is a new dependency for end-to-end tests.
The tool is built together with all the other backend tests
(so the buildall.sh flag '-notests' can turn off building it), it is
actually used in the parallel phase of end-to-end tests.
This means a problem for Docker-based builds for the following reasons:
- Docker-based tests run BE, FE and various phases of the EE test in
separate Docker containers for parallel executions
- Test binaries are only built inside the container running BE tests to
cut down on the build time and the size of the Docker image that all test
containers are based on.
- This means that the EE_TEST_PARALLEL container will miss the tool
required for running test designed to test it.
The solution is to build the tool early, at the end of the build phase
running in the build container. There is already another such tool built
there (parquet-reader) for similar reason, so just add
impala-profile-tool to the same 'make' command there.
Tested by running BE_TEST and EE_TEST_PARALLEL phases in a Docker-based
build.
Change-Id: I60e78ea883f3057c59a345feca38ef08a7f6a0b8
Reviewed-on: http://gerrit.cloudera.org:8080/16965
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M docker/entrypoint.sh
1 file changed, 1 insertion(+), 1 deletion(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/16965
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I60e78ea883f3057c59a345feca38ef08a7f6a0b8
Gerrit-Change-Number: 16965
Gerrit-PatchSet: 3
Gerrit-Owner: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-10448: Build impala-profile-tool early for Docker-based tests
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16965 )
Change subject: IMPALA-10448: Build impala-profile-tool early for Docker-based tests
......................................................................
Patch Set 1: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/16965
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I60e78ea883f3057c59a345feca38ef08a7f6a0b8
Gerrit-Change-Number: 16965
Gerrit-PatchSet: 1
Gerrit-Owner: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 20 Jan 2021 16:53:05 +0000
Gerrit-HasComments: No