You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Philip Zeyliger (Code Review)" <ge...@cloudera.org> on 2018/04/10 16:23:43 UTC
[Impala-ASF-CR] Remove Yarn from minicluster by default. (2nd try)
Philip Zeyliger has uploaded this change for review. ( http://gerrit.cloudera.org:8080/9970
Change subject: Remove Yarn from minicluster by default. (2nd try)
......................................................................
Remove Yarn from minicluster by default. (2nd try)
Remove Yarn from minicluster by default.
Turns out that we start Yarn as part of the minicluster, but we never use it.
(HiveServer2 is configured to run MR jobs "locally" in process.) Likely, this
Yarn integration is a vestige of Yarn/Llama integration. We can save memory by
not starting it by default.
There are some less-common tooks like tests/comparison/cluster.py which use
Yarn (and Hadoop Streaming). In deference to those tools, I've left a mechanism
to start Yarn rather than excising it altogether. After running
buildall the regular way, add Yarn to the cluster by running:
testdata/cluster/admin -y start_cluster
I tested by running core tests. I did not test the kerberized minicluster.
[Due to a git mishap, a version of this was previously checked in and reverted.]
Change-Id: I97053a44bbe32048e6c35cc28680d1c7696af13f
---
M testdata/cluster/admin
1 file changed, 9 insertions(+), 4 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/70/9970/1
--
To view, visit http://gerrit.cloudera.org:8080/9970
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I97053a44bbe32048e6c35cc28680d1c7696af13f
Gerrit-Change-Number: 9970
Gerrit-PatchSet: 1
Gerrit-Owner: Philip Zeyliger <ph...@cloudera.com>
[Impala-ASF-CR] Remove Yarn from minicluster by default. (2nd try)
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/9970 )
Change subject: Remove Yarn from minicluster by default. (2nd try)
......................................................................
Patch Set 1: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/9970
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I97053a44bbe32048e6c35cc28680d1c7696af13f
Gerrit-Change-Number: 9970
Gerrit-PatchSet: 1
Gerrit-Owner: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Comment-Date: Tue, 10 Apr 2018 20:15:29 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Remove Yarn from minicluster by default. (2nd try)
Posted by "Michael Brown (Code Review)" <ge...@cloudera.org>.
Michael Brown has posted comments on this change. ( http://gerrit.cloudera.org:8080/9970 )
Change subject: Remove Yarn from minicluster by default. (2nd try)
......................................................................
Patch Set 1: Code-Review+2
Thanks!
--
To view, visit http://gerrit.cloudera.org:8080/9970
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I97053a44bbe32048e6c35cc28680d1c7696af13f
Gerrit-Change-Number: 9970
Gerrit-PatchSet: 1
Gerrit-Owner: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Comment-Date: Tue, 10 Apr 2018 16:27:04 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Remove Yarn from minicluster by default. (2nd try)
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/9970 )
Change subject: Remove Yarn from minicluster by default. (2nd try)
......................................................................
Remove Yarn from minicluster by default. (2nd try)
Remove Yarn from minicluster by default.
Turns out that we start Yarn as part of the minicluster, but we never use it.
(HiveServer2 is configured to run MR jobs "locally" in process.) Likely, this
Yarn integration is a vestige of Yarn/Llama integration. We can save memory by
not starting it by default.
There are some less-common tooks like tests/comparison/cluster.py which use
Yarn (and Hadoop Streaming). In deference to those tools, I've left a mechanism
to start Yarn rather than excising it altogether. After running
buildall the regular way, add Yarn to the cluster by running:
testdata/cluster/admin -y start_cluster
I tested by running core tests. I did not test the kerberized minicluster.
[Due to a git mishap, a version of this was previously checked in and reverted.]
Change-Id: I97053a44bbe32048e6c35cc28680d1c7696af13f
Reviewed-on: http://gerrit.cloudera.org:8080/9970
Reviewed-by: Michael Brown <mi...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M testdata/cluster/admin
1 file changed, 9 insertions(+), 4 deletions(-)
Approvals:
Michael Brown: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/9970
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I97053a44bbe32048e6c35cc28680d1c7696af13f
Gerrit-Change-Number: 9970
Gerrit-PatchSet: 2
Gerrit-Owner: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
[Impala-ASF-CR] Remove Yarn from minicluster by default. (2nd try)
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/9970 )
Change subject: Remove Yarn from minicluster by default. (2nd try)
......................................................................
Patch Set 1:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2272/
--
To view, visit http://gerrit.cloudera.org:8080/9970
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I97053a44bbe32048e6c35cc28680d1c7696af13f
Gerrit-Change-Number: 9970
Gerrit-PatchSet: 1
Gerrit-Owner: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Comment-Date: Tue, 10 Apr 2018 16:27:41 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Remove Yarn from minicluster by default. (2nd try)
Posted by "Philip Zeyliger (Code Review)" <ge...@cloudera.org>.
Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/9970 )
Change subject: Remove Yarn from minicluster by default. (2nd try)
......................................................................
Patch Set 1:
This is a continuation of https://gerrit.cloudera.org/#/c/9264/2 which I accidentally pushed before review. I need to figure out how to poison my gerrit setup from being able to do that... (I did :master instead of :refs/for/master)
--
To view, visit http://gerrit.cloudera.org:8080/9970
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I97053a44bbe32048e6c35cc28680d1c7696af13f
Gerrit-Change-Number: 9970
Gerrit-PatchSet: 1
Gerrit-Owner: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Comment-Date: Tue, 10 Apr 2018 16:25:44 +0000
Gerrit-HasComments: No