You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Michael Ho (Code Review)" <ge...@cloudera.org> on 2019/10/08 18:51:41 UTC
[Impala-ASF-CR] IMPALA-9026: Use resolved IP address for statestore subscriber
Michael Ho has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14388
Change subject: IMPALA-9026: Use resolved IP address for statestore subscriber
......................................................................
IMPALA-9026: Use resolved IP address for statestore subscriber
This change adds a flag (FLAGS_statestore_subscriber_use_resolved_address)
which, if set to true, allows statestore subscribers to use its
resolved IP address instead of its hostname as the heartbeat
address which statestore sends heartbeats / updates to.
This flag is useful in certain deployment scenarios in which a pod
may have an IP address but its DNS entry is not available for a valid
reason (e.g. a Kubernetes pod whose readiness probe returns false).
An example is that there are more than one instances of Impala coordinator
but only one of them will be active at a time (for admission control reasons)
and the rest will serve as backup. In which case, we want the backup coordinator
to receive updates from statestore but not serve any queries until the
primary coordinator fails.
Change-Id: Ieb8302dec0e52beb9f0b88306a51c38ff42a63a2
---
M be/src/catalog/catalog-server.cc
M be/src/runtime/exec-env.cc
M be/src/statestore/statestore-subscriber.cc
M tests/custom_cluster/test_custom_statestore.py
4 files changed, 33 insertions(+), 6 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/14388/1
--
To view, visit http://gerrit.cloudera.org:8080/14388
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ieb8302dec0e52beb9f0b88306a51c38ff42a63a2
Gerrit-Change-Number: 14388
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
[Impala-ASF-CR] IMPALA-9026: Use resolved IP address for statestore subscriber
Posted by "Sahil Takiar (Code Review)" <ge...@cloudera.org>.
Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14388 )
Change subject: IMPALA-9026: Use resolved IP address for statestore subscriber
......................................................................
Patch Set 4: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/14388
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ieb8302dec0e52beb9f0b88306a51c38ff42a63a2
Gerrit-Change-Number: 14388
Gerrit-PatchSet: 4
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>
Gerrit-Comment-Date: Mon, 28 Oct 2019 18:33:13 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9026: Use resolved IP address for statestore subscriber
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14388 )
Change subject: IMPALA-9026: Use resolved IP address for statestore subscriber
......................................................................
IMPALA-9026: Use resolved IP address for statestore subscriber
This change adds a flag (--statestore_subscriber_use_resolved_address)
which, if set to true, allows statestore subscribers to use its
resolved IP address instead of its hostname as the heartbeat
address which statestore sends heartbeats / updates to.
This flag is useful in certain situation in which the subscriber's
DNS entry may not be present for a valid reason (e.g. a Kubernetes
pod whose readiness probe returns false). An example is that there
are multiple Impala coordinators but only one of them will be active
at a time (for admission control reason) and the rest will serve
as backup. In which case, we still want the backup coordinators to
receive updates from statestore but not serve any queries.
Change-Id: Ieb8302dec0e52beb9f0b88306a51c38ff42a63a2
Reviewed-on: http://gerrit.cloudera.org:8080/14388
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/catalog/catalog-server.cc
M be/src/runtime/exec-env.cc
M be/src/statestore/statestore-subscriber.cc
M be/src/statestore/statestore-subscriber.h
M tests/custom_cluster/test_custom_statestore.py
5 files changed, 35 insertions(+), 6 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/14388
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ieb8302dec0e52beb9f0b88306a51c38ff42a63a2
Gerrit-Change-Number: 14388
Gerrit-PatchSet: 6
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>
[Impala-ASF-CR] IMPALA-9026: Use resolved IP address for statestore subscriber
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14388 )
Change subject: IMPALA-9026: Use resolved IP address for statestore subscriber
......................................................................
Patch Set 5:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5147/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/14388
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ieb8302dec0e52beb9f0b88306a51c38ff42a63a2
Gerrit-Change-Number: 14388
Gerrit-PatchSet: 5
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>
Gerrit-Comment-Date: Tue, 29 Oct 2019 18:50:31 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9026: Use resolved IP address for statestore subscriber
Posted by "Sahil Takiar (Code Review)" <ge...@cloudera.org>.
Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/14388 )
Change subject: IMPALA-9026: Use resolved IP address for statestore subscriber
......................................................................
Patch Set 2:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/14388/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/14388/2//COMMIT_MSG@19
PS2, Line 19: coordinator
nit: line too long
http://gerrit.cloudera.org:8080/#/c/14388/2/be/src/runtime/exec-env.cc
File be/src/runtime/exec-env.cc:
http://gerrit.cloudera.org:8080/#/c/14388/2/be/src/runtime/exec-env.cc@222
PS2, Line 222: // Resolve hostname to IP address.
: ABORT_IF_ERROR(HostnameToIpAddr(FLAGS_hostname, &ip_address_));
:
: // KRPC relies on resolved IP address.
: krpc_address_.__set_hostname(ip_address_);
whats the reason for moving these from Init() to the constructor?
http://gerrit.cloudera.org:8080/#/c/14388/2/be/src/statestore/statestore-subscriber.cc
File be/src/statestore/statestore-subscriber.cc:
http://gerrit.cloudera.org:8080/#/c/14388/2/be/src/statestore/statestore-subscriber.cc@262
PS2, Line 262: hostname
document that the hostname can now either be the actual hostname, or the ip_address of the host?
another approach might be to add an optional ip_address field to the Thrift struct; users of TNetworkAddress first check if the ip_address is present, if not, it uses the hostname. Might not be worth the effort, but thought I would mention it.
--
To view, visit http://gerrit.cloudera.org:8080/14388
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ieb8302dec0e52beb9f0b88306a51c38ff42a63a2
Gerrit-Change-Number: 14388
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>
Gerrit-Comment-Date: Wed, 09 Oct 2019 17:19:30 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9026: Use resolved IP address for statestore subscriber
Posted by "Michael Ho (Code Review)" <ge...@cloudera.org>.
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/14388 )
Change subject: IMPALA-9026: Use resolved IP address for statestore subscriber
......................................................................
Patch Set 1:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/14388/1/tests/custom_cluster/test_custom_statestore.py
File tests/custom_cluster/test_custom_statestore.py:
http://gerrit.cloudera.org:8080/#/c/14388/1/tests/custom_cluster/test_custom_statestore.py@100
PS1, Line 100: ;
> flake8: E703 statement ends with a semicolon
Done
http://gerrit.cloudera.org:8080/#/c/14388/1/tests/custom_cluster/test_custom_statestore.py@102
PS1, Line 102: ;
> flake8: E703 statement ends with a semicolon
Done
--
To view, visit http://gerrit.cloudera.org:8080/14388
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ieb8302dec0e52beb9f0b88306a51c38ff42a63a2
Gerrit-Change-Number: 14388
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Comment-Date: Tue, 08 Oct 2019 19:16:31 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9026: Use resolved IP address for statestore subscriber
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14388 )
Change subject: IMPALA-9026: Use resolved IP address for statestore subscriber
......................................................................
Patch Set 5: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/14388
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ieb8302dec0e52beb9f0b88306a51c38ff42a63a2
Gerrit-Change-Number: 14388
Gerrit-PatchSet: 5
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>
Gerrit-Comment-Date: Tue, 29 Oct 2019 23:11:46 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9026: Use resolved IP address for statestore subscriber
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14388 )
Change subject: IMPALA-9026: Use resolved IP address for statestore subscriber
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/4857/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/14388
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ieb8302dec0e52beb9f0b88306a51c38ff42a63a2
Gerrit-Change-Number: 14388
Gerrit-PatchSet: 3
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>
Gerrit-Comment-Date: Wed, 23 Oct 2019 03:01:43 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9026: Use resolved IP address for statestore subscriber
Posted by "Michael Ho (Code Review)" <ge...@cloudera.org>.
Hello Sahil Takiar, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/14388
to look at the new patch set (#3).
Change subject: IMPALA-9026: Use resolved IP address for statestore subscriber
......................................................................
IMPALA-9026: Use resolved IP address for statestore subscriber
This change adds a flag (--statestore_subscriber_use_resolved_address)
which, if set to true, allows statestore subscribers to use its
resolved IP address instead of its hostname as the heartbeat
address which statestore sends heartbeats / updates to.
This flag is useful in certain situation in which the subscriber
DNS entry may not be present for a valid reason (e.g. a Kubernetes
pod whose readiness probe returns false). An example is that there
are muplitle Impala coordinator but only one of them will be active
at a time (for admission control reasons) and the rest will serve
as backup. In which case, we still want the backup coordinators to
receive updates from statestore but not serve any queries.
Change-Id: Ieb8302dec0e52beb9f0b88306a51c38ff42a63a2
---
M be/src/catalog/catalog-server.cc
M be/src/runtime/exec-env.cc
M be/src/statestore/statestore-subscriber.cc
M be/src/statestore/statestore-subscriber.h
M tests/custom_cluster/test_custom_statestore.py
5 files changed, 35 insertions(+), 6 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/14388/3
--
To view, visit http://gerrit.cloudera.org:8080/14388
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ieb8302dec0e52beb9f0b88306a51c38ff42a63a2
Gerrit-Change-Number: 14388
Gerrit-PatchSet: 3
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>
[Impala-ASF-CR] IMPALA-9026: Use resolved IP address for statestore subscriber
Posted by "Michael Ho (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/14388
to look at the new patch set (#2).
Change subject: IMPALA-9026: Use resolved IP address for statestore subscriber
......................................................................
IMPALA-9026: Use resolved IP address for statestore subscriber
This change adds a flag (FLAGS_statestore_subscriber_use_resolved_address)
which, if set to true, allows statestore subscribers to use its
resolved IP address instead of its hostname as the heartbeat
address which statestore sends heartbeats / updates to.
This flag is useful in certain deployment scenarios in which a pod
may have an IP address but its DNS entry is not available for a valid
reason (e.g. a Kubernetes pod whose readiness probe returns false).
An example is that there are more than one instances of Impala coordinator
but only one of them will be active at a time (for admission control reasons)
and the rest will serve as backup. In which case, we want the backup coordinator
to receive updates from statestore but not serve any queries until the
primary coordinator fails.
Change-Id: Ieb8302dec0e52beb9f0b88306a51c38ff42a63a2
---
M be/src/catalog/catalog-server.cc
M be/src/runtime/exec-env.cc
M be/src/statestore/statestore-subscriber.cc
M tests/custom_cluster/test_custom_statestore.py
4 files changed, 33 insertions(+), 6 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/14388/2
--
To view, visit http://gerrit.cloudera.org:8080/14388
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ieb8302dec0e52beb9f0b88306a51c38ff42a63a2
Gerrit-Change-Number: 14388
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-9026: Use resolved IP address for statestore subscriber
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14388 )
Change subject: IMPALA-9026: Use resolved IP address for statestore subscriber
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/4751/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/14388
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ieb8302dec0e52beb9f0b88306a51c38ff42a63a2
Gerrit-Change-Number: 14388
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>
Gerrit-Comment-Date: Tue, 08 Oct 2019 19:55:20 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9026: Use resolved IP address for statestore subscriber
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14388 )
Change subject: IMPALA-9026: Use resolved IP address for statestore subscriber
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/4750/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/14388
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ieb8302dec0e52beb9f0b88306a51c38ff42a63a2
Gerrit-Change-Number: 14388
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>
Gerrit-Comment-Date: Tue, 08 Oct 2019 19:33:35 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9026: Use resolved IP address for statestore subscriber
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14388 )
Change subject: IMPALA-9026: Use resolved IP address for statestore subscriber
......................................................................
Patch Set 5: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/14388
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ieb8302dec0e52beb9f0b88306a51c38ff42a63a2
Gerrit-Change-Number: 14388
Gerrit-PatchSet: 5
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>
Gerrit-Comment-Date: Tue, 29 Oct 2019 18:50:30 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9026: Use resolved IP address for statestore subscriber
Posted by "Michael Ho (Code Review)" <ge...@cloudera.org>.
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/14388 )
Change subject: IMPALA-9026: Use resolved IP address for statestore subscriber
......................................................................
Patch Set 3:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/14388/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/14388/3//COMMIT_MSG@17
PS3, Line 17: muplitle
> typo
Done
--
To view, visit http://gerrit.cloudera.org:8080/14388
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ieb8302dec0e52beb9f0b88306a51c38ff42a63a2
Gerrit-Change-Number: 14388
Gerrit-PatchSet: 3
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>
Gerrit-Comment-Date: Wed, 23 Oct 2019 05:33:47 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9026: Use resolved IP address for statestore subscriber
Posted by "Michael Ho (Code Review)" <ge...@cloudera.org>.
Hello Sahil Takiar, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/14388
to look at the new patch set (#4).
Change subject: IMPALA-9026: Use resolved IP address for statestore subscriber
......................................................................
IMPALA-9026: Use resolved IP address for statestore subscriber
This change adds a flag (--statestore_subscriber_use_resolved_address)
which, if set to true, allows statestore subscribers to use its
resolved IP address instead of its hostname as the heartbeat
address which statestore sends heartbeats / updates to.
This flag is useful in certain situation in which the subscriber's
DNS entry may not be present for a valid reason (e.g. a Kubernetes
pod whose readiness probe returns false). An example is that there
are multiple Impala coordinators but only one of them will be active
at a time (for admission control reason) and the rest will serve
as backup. In which case, we still want the backup coordinators to
receive updates from statestore but not serve any queries.
Change-Id: Ieb8302dec0e52beb9f0b88306a51c38ff42a63a2
---
M be/src/catalog/catalog-server.cc
M be/src/runtime/exec-env.cc
M be/src/statestore/statestore-subscriber.cc
M be/src/statestore/statestore-subscriber.h
M tests/custom_cluster/test_custom_statestore.py
5 files changed, 35 insertions(+), 6 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/14388/4
--
To view, visit http://gerrit.cloudera.org:8080/14388
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ieb8302dec0e52beb9f0b88306a51c38ff42a63a2
Gerrit-Change-Number: 14388
Gerrit-PatchSet: 4
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>
[Impala-ASF-CR] IMPALA-9026: Use resolved IP address for statestore subscriber
Posted by "Michael Ho (Code Review)" <ge...@cloudera.org>.
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/14388 )
Change subject: IMPALA-9026: Use resolved IP address for statestore subscriber
......................................................................
Patch Set 3:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/14388/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/14388/2//COMMIT_MSG@19
PS2, Line 19:
> nit: line too long
Done
http://gerrit.cloudera.org:8080/#/c/14388/2/be/src/runtime/exec-env.cc
File be/src/runtime/exec-env.cc:
http://gerrit.cloudera.org:8080/#/c/14388/2/be/src/runtime/exec-env.cc@222
PS2, Line 222: tmp_file_mgr_(new TmpFileMgr),
: frontend_(new Frontend()),
: async_rpc_pool_(new CallableThreadPool("rpc-pool", "async-rpc-sender", 8, 10000)),
: query_exec_mgr_(new QueryExecMgr()),
: rpc_metrics_(metrics_->GetOrCreateChildG
> whats the reason for moving these from Init() to the constructor?
Failing in Init() below or here are both fatal (i.e. Impala will not start) so I may as well remove the complication of splitting the initialization of krpc_address_ into two functions.
In a previous version of the patch I needed the IP address in this function to initialize the statestore subscriber but has since modified the code but I feel the simplification above is still warranted.
http://gerrit.cloudera.org:8080/#/c/14388/2/be/src/statestore/statestore-subscriber.cc
File be/src/statestore/statestore-subscriber.cc:
http://gerrit.cloudera.org:8080/#/c/14388/2/be/src/statestore/statestore-subscriber.cc@262
PS2, Line 262: hostname
> document that the hostname can now either be the actual hostname, or the ip
Comments updated in the header filer. Seems simpler to just resolve the hostname here instead of having the server choose between multiple addresses.
--
To view, visit http://gerrit.cloudera.org:8080/14388
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ieb8302dec0e52beb9f0b88306a51c38ff42a63a2
Gerrit-Change-Number: 14388
Gerrit-PatchSet: 3
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>
Gerrit-Comment-Date: Wed, 23 Oct 2019 02:16:40 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9026: Use resolved IP address for statestore subscriber
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14388 )
Change subject: IMPALA-9026: Use resolved IP address for statestore subscriber
......................................................................
Patch Set 1:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/14388/1/tests/custom_cluster/test_custom_statestore.py
File tests/custom_cluster/test_custom_statestore.py:
http://gerrit.cloudera.org:8080/#/c/14388/1/tests/custom_cluster/test_custom_statestore.py@100
PS1, Line 100: ;
flake8: E703 statement ends with a semicolon
http://gerrit.cloudera.org:8080/#/c/14388/1/tests/custom_cluster/test_custom_statestore.py@102
PS1, Line 102: ;
flake8: E703 statement ends with a semicolon
--
To view, visit http://gerrit.cloudera.org:8080/14388
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ieb8302dec0e52beb9f0b88306a51c38ff42a63a2
Gerrit-Change-Number: 14388
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 08 Oct 2019 18:53:31 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9026: Use resolved IP address for statestore subscriber
Posted by "Michael Ho (Code Review)" <ge...@cloudera.org>.
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/14388 )
Change subject: IMPALA-9026: Use resolved IP address for statestore subscriber
......................................................................
Patch Set 3:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/14388/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/14388/3//COMMIT_MSG@17
PS3, Line 17: muplitle
typo
--
To view, visit http://gerrit.cloudera.org:8080/14388
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ieb8302dec0e52beb9f0b88306a51c38ff42a63a2
Gerrit-Change-Number: 14388
Gerrit-PatchSet: 3
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>
Gerrit-Comment-Date: Wed, 23 Oct 2019 02:17:35 +0000
Gerrit-HasComments: Yes