You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Tim Armstrong (Code Review)" <ge...@cloudera.org> on 2019/01/16 20:55:14 UTC
[Impala-ASF-CR] IMPALA-8066: Build coordinator and executor containers
Tim Armstrong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12228
Change subject: IMPALA-8066: Build coordinator and executor containers
......................................................................
IMPALA-8066: Build coordinator and executor containers
The containers are essentially the same except for -is_executor
and -is_coordinator flags and the open ports (executors don't
need to expose HS2 and Beeswax).
The previous impalad container was renamed to impalad_coord_exec
to be unambiguous.
Change-Id: I22f8ded167179478d7556f612b8b3e9d1b019a7a
---
M bin/start-impala-cluster.py
M docker/CMakeLists.txt
R docker/coord_exec/Dockerfile
A docker/coordinator/Dockerfile
A docker/executor/Dockerfile
5 files changed, 98 insertions(+), 12 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/28/12228/1
--
To view, visit http://gerrit.cloudera.org:8080/12228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I22f8ded167179478d7556f612b8b3e9d1b019a7a
Gerrit-Change-Number: 12228
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-8066: Build coordinator and executor containers
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12228 )
Change subject: IMPALA-8066: Build coordinator and executor containers
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/1802/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/12228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f8ded167179478d7556f612b8b3e9d1b019a7a
Gerrit-Change-Number: 12228
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 17 Jan 2019 20:55:26 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8066: Build coordinator and executor containers
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12228 )
Change subject: IMPALA-8066: Build coordinator and executor containers
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/1797/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/12228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f8ded167179478d7556f612b8b3e9d1b019a7a
Gerrit-Change-Number: 12228
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Wed, 16 Jan 2019 23:10:28 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8066: Build coordinator and executor containers
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12228 )
Change subject: IMPALA-8066: Build coordinator and executor containers
......................................................................
Patch Set 3: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/12228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f8ded167179478d7556f612b8b3e9d1b019a7a
Gerrit-Change-Number: 12228
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 18 Jan 2019 00:26:52 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8066: Build coordinator and executor containers
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12228 )
Change subject: IMPALA-8066: Build coordinator and executor containers
......................................................................
Patch Set 3:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3648/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/12228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f8ded167179478d7556f612b8b3e9d1b019a7a
Gerrit-Change-Number: 12228
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 17 Jan 2019 20:21:08 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8066: Build coordinator and executor containers
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12228 )
Change subject: IMPALA-8066: Build coordinator and executor containers
......................................................................
Patch Set 2: Code-Review+2
Actually I guess rebasing onto master just means not including the start-impala-cluster.py change, so I'll carry the +2.
--
To view, visit http://gerrit.cloudera.org:8080/12228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f8ded167179478d7556f612b8b3e9d1b019a7a
Gerrit-Change-Number: 12228
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 17 Jan 2019 20:19:07 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8066: Build coordinator and executor containers
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Hello Philip Zeyliger, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/12228
to look at the new patch set (#2).
Change subject: IMPALA-8066: Build coordinator and executor containers
......................................................................
IMPALA-8066: Build coordinator and executor containers
The containers are essentially the same except for -is_executor
and -is_coordinator flags and the open ports (executors don't
need to expose HS2 and Beeswax). Over time we may want to
specialize the configurations further.
Building separate containers on top of impala_base is lightweight enough
and this a) reduces the amount of configuration required and b) makes it
clear which ports should open. It will also nudge people in the direction
of using dedicated coordinators and executors in Kubernetes, which I
believe is the right approach.
The previous impalad container was renamed to impalad_coord_exec
to be unambiguous.
Change-Id: I22f8ded167179478d7556f612b8b3e9d1b019a7a
---
M docker/CMakeLists.txt
M docker/catalogd/Dockerfile
C docker/coord_exec/Dockerfile
R docker/coordinator/Dockerfile
C docker/executor/Dockerfile
M docker/statestored/Dockerfile
6 files changed, 47 insertions(+), 21 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/28/12228/2
--
To view, visit http://gerrit.cloudera.org:8080/12228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I22f8ded167179478d7556f612b8b3e9d1b019a7a
Gerrit-Change-Number: 12228
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
[Impala-ASF-CR] IMPALA-8066: Build coordinator and executor containers
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12228 )
Change subject: IMPALA-8066: Build coordinator and executor containers
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/1801/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/12228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f8ded167179478d7556f612b8b3e9d1b019a7a
Gerrit-Change-Number: 12228
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 17 Jan 2019 20:48:05 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8066: Build coordinator and executor containers
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12228 )
Change subject: IMPALA-8066: Build coordinator and executor containers
......................................................................
Patch Set 3: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/12228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f8ded167179478d7556f612b8b3e9d1b019a7a
Gerrit-Change-Number: 12228
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 17 Jan 2019 20:20:52 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8066: Build coordinator and executor containers
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12228 )
Change subject: IMPALA-8066: Build coordinator and executor containers
......................................................................
Patch Set 1:
Yeah, I think making it easy to use the coord/exec configuration is also useful - even if the extra configuration is easy it's another step to do and nudges people in the direction of combined coordinators and executors.
The depended on some minor changes from https://gerrit.cloudera.org/#/c/12095/ to the build scripts, so I pulled those into this patch. Would appreciate if you could take a quick look.
--
To view, visit http://gerrit.cloudera.org:8080/12228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f8ded167179478d7556f612b8b3e9d1b019a7a
Gerrit-Change-Number: 12228
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 17 Jan 2019 20:17:51 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8066: Build coordinator and executor containers
Posted by "Philip Zeyliger (Code Review)" <ge...@cloudera.org>.
Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/12228 )
Change subject: IMPALA-8066: Build coordinator and executor containers
......................................................................
Patch Set 1: Code-Review+2
You think it's useful to have these because it's easier to think of a container as always using the same ports? Perhaps include that reasoning in the commit message.
I would have naively thought that this is just configuration, as would kerberos or whatever be, but it's sensible to do it this way too.
--
To view, visit http://gerrit.cloudera.org:8080/12228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22f8ded167179478d7556f612b8b3e9d1b019a7a
Gerrit-Change-Number: 12228
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Comment-Date: Thu, 17 Jan 2019 17:12:01 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8066: Build coordinator and executor containers
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Hello Philip Zeyliger, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/12228
to look at the new patch set (#3).
Change subject: IMPALA-8066: Build coordinator and executor containers
......................................................................
IMPALA-8066: Build coordinator and executor containers
The containers are essentially the same except for -is_executor
and -is_coordinator flags and the open ports (executors don't
need to expose HS2 and Beeswax). Over time we may want to
specialize the configurations further.
Building separate containers on top of impala_base is lightweight enough
and this a) reduces the amount of configuration required and b) makes it
clear which ports should open. It will also nudge people in the direction
of using dedicated coordinators and executors in Kubernetes, which I
believe is the right approach.
The previous impalad container was renamed to impalad_coord_exec
to be unambiguous.
Change-Id: I22f8ded167179478d7556f612b8b3e9d1b019a7a
---
M docker/CMakeLists.txt
M docker/catalogd/Dockerfile
R docker/coord_exec/Dockerfile
C docker/coordinator/Dockerfile
C docker/executor/Dockerfile
M docker/statestored/Dockerfile
6 files changed, 49 insertions(+), 23 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/28/12228/3
--
To view, visit http://gerrit.cloudera.org:8080/12228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I22f8ded167179478d7556f612b8b3e9d1b019a7a
Gerrit-Change-Number: 12228
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-8066: Build coordinator and executor containers
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12228 )
Change subject: IMPALA-8066: Build coordinator and executor containers
......................................................................
IMPALA-8066: Build coordinator and executor containers
The containers are essentially the same except for -is_executor
and -is_coordinator flags and the open ports (executors don't
need to expose HS2 and Beeswax). Over time we may want to
specialize the configurations further.
Building separate containers on top of impala_base is lightweight enough
and this a) reduces the amount of configuration required and b) makes it
clear which ports should open. It will also nudge people in the direction
of using dedicated coordinators and executors in Kubernetes, which I
believe is the right approach.
The previous impalad container was renamed to impalad_coord_exec
to be unambiguous.
Change-Id: I22f8ded167179478d7556f612b8b3e9d1b019a7a
Reviewed-on: http://gerrit.cloudera.org:8080/12228
Reviewed-by: Tim Armstrong <ta...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M docker/CMakeLists.txt
M docker/catalogd/Dockerfile
R docker/coord_exec/Dockerfile
C docker/coordinator/Dockerfile
C docker/executor/Dockerfile
M docker/statestored/Dockerfile
6 files changed, 49 insertions(+), 23 deletions(-)
Approvals:
Tim Armstrong: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/12228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I22f8ded167179478d7556f612b8b3e9d1b019a7a
Gerrit-Change-Number: 12228
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>