You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Abhishek Rawat (Code Review)" <ge...@cloudera.org> on 2023/01/05 23:49:10 UTC

[Impala-ASF-CR] IMPALA-11827: do not cache admission control service's IP address in impalad

Abhishek Rawat has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19403


Change subject: IMPALA-11827: do not cache admission control service's IP address in impalad
......................................................................

IMPALA-11827: do not cache admission control service's IP address in impalad

The Impalad's ExecEnv caches resolved IP address for admission control
service. The Impala server runs a heart beat thread for admission
control service and it relies on the cached resolved address in ExecEnv.
This breaks when IP address for admission control services changes.

This patch removes the 'admission_service_address_' member and its
getter in the ExecEnv class and adds a GetAdmissionServiceAddress()
interface to dynamically get the resolved address, if global admission
control service is enabled ('admission_service_host' flag is set).

Testing: Manually tested by restarting global admission controller
service in a Kubernetes setup where it gets new IP address on startup.
Coordinator's admission heart beat thread was able to connect to the
admission control service using the new IP address. Also made sure that
the coordinator and admissiond communication was working as expected
upon admissiond restarts.

Change-Id: I09b4c52644f9e1b3c1f0e68c9900464d722517af
---
M be/src/runtime/exec-env.cc
M be/src/runtime/exec-env.h
M be/src/scheduling/admission-control-service.cc
M be/src/service/impala-server.cc
4 files changed, 20 insertions(+), 18 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/03/19403/1
-- 
To view, visit http://gerrit.cloudera.org:8080/19403
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I09b4c52644f9e1b3c1f0e68c9900464d722517af
Gerrit-Change-Number: 19403
Gerrit-PatchSet: 1
Gerrit-Owner: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[Impala-ASF-CR] IMPALA-11827: do not cache admission control service's IP address in impalad

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19403 )

Change subject: IMPALA-11827: do not cache admission control service's IP address in impalad
......................................................................


Patch Set 3:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8952/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/19403
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I09b4c52644f9e1b3c1f0e68c9900464d722517af
Gerrit-Change-Number: 19403
Gerrit-PatchSet: 3
Gerrit-Owner: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 06 Jan 2023 01:41:53 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11827: do not cache admission control service's IP address in impalad

Posted by "Abhishek Rawat (Code Review)" <ge...@cloudera.org>.
Abhishek Rawat has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/19403 )

Change subject: IMPALA-11827: do not cache admission control service's IP address in impalad
......................................................................

IMPALA-11827: do not cache admission control service's IP address in impalad

The Impalad's ExecEnv caches resolved IP address for admission control
service. The Impala server runs a heart beat thread for admission
control service and it relies on the cached resolved address in ExecEnv.
This breaks when IP address for admission control services changes.

This patch removes the 'admission_service_address_' member and its
getter in the ExecEnv class and adds a GetAdmissionServiceAddress()
interface to dynamically get the resolved address, if global admission
control service is enabled ('admission_service_host' flag is set).

Testing: Manually tested by restarting global admission controller
service in a Kubernetes setup where it gets new IP address on startup.
Coordinator's admission heart beat thread was able to connect to the
admission control service using the new IP address. Also made sure that
the coordinator and admissiond communication was working as expected
upon admissiond restarts.

Change-Id: I09b4c52644f9e1b3c1f0e68c9900464d722517af
---
M be/src/runtime/exec-env.cc
M be/src/runtime/exec-env.h
M be/src/scheduling/admission-control-service.cc
M be/src/service/impala-server.cc
4 files changed, 21 insertions(+), 18 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/03/19403/2
-- 
To view, visit http://gerrit.cloudera.org:8080/19403
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I09b4c52644f9e1b3c1f0e68c9900464d722517af
Gerrit-Change-Number: 19403
Gerrit-PatchSet: 2
Gerrit-Owner: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[Impala-ASF-CR] IMPALA-11827: do not cache admission control service's IP address in impalad

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19403 )

Change subject: IMPALA-11827: do not cache admission control service's IP address in impalad
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/12115/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19403
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I09b4c52644f9e1b3c1f0e68c9900464d722517af
Gerrit-Change-Number: 19403
Gerrit-PatchSet: 2
Gerrit-Owner: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 06 Jan 2023 00:13:02 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11827: do not cache admission control service's IP address in impalad

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19403 )

Change subject: IMPALA-11827: do not cache admission control service's IP address in impalad
......................................................................


Patch Set 3: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/19403
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I09b4c52644f9e1b3c1f0e68c9900464d722517af
Gerrit-Change-Number: 19403
Gerrit-PatchSet: 3
Gerrit-Owner: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 06 Jan 2023 01:41:52 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11827: do not cache admission control service's IP address in impalad

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19403 )

Change subject: IMPALA-11827: do not cache admission control service's IP address in impalad
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19403/1/be/src/runtime/exec-env.cc
File be/src/runtime/exec-env.cc:

http://gerrit.cloudera.org:8080/#/c/19403/1/be/src/runtime/exec-env.cc@621
PS1, Line 621: Status ExecEnv::GetAdmissionServiceAddress(NetworkAddressPB& admission_service_address) const {
line too long (95 > 90)



-- 
To view, visit http://gerrit.cloudera.org:8080/19403
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I09b4c52644f9e1b3c1f0e68c9900464d722517af
Gerrit-Change-Number: 19403
Gerrit-PatchSet: 1
Gerrit-Owner: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Thu, 05 Jan 2023 23:50:05 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11827: do not cache admission control service's IP address in impalad

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19403 )

Change subject: IMPALA-11827: do not cache admission control service's IP address in impalad
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/12114/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19403
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I09b4c52644f9e1b3c1f0e68c9900464d722517af
Gerrit-Change-Number: 19403
Gerrit-PatchSet: 1
Gerrit-Owner: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 06 Jan 2023 00:09:46 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11827: do not cache admission control service's IP address in impalad

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/19403 )

Change subject: IMPALA-11827: do not cache admission control service's IP address in impalad
......................................................................


Patch Set 2: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/19403
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I09b4c52644f9e1b3c1f0e68c9900464d722517af
Gerrit-Change-Number: 19403
Gerrit-PatchSet: 2
Gerrit-Owner: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 06 Jan 2023 01:40:47 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11827: do not cache admission control service's IP address in impalad

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19403 )

Change subject: IMPALA-11827: do not cache admission control service's IP address in impalad
......................................................................


Patch Set 3: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/19403
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I09b4c52644f9e1b3c1f0e68c9900464d722517af
Gerrit-Change-Number: 19403
Gerrit-PatchSet: 3
Gerrit-Owner: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 06 Jan 2023 06:53:52 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11827: do not cache admission control service's IP address in impalad

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19403 )

Change subject: IMPALA-11827: do not cache admission control service's IP address in impalad
......................................................................

IMPALA-11827: do not cache admission control service's IP address in impalad

The Impalad's ExecEnv caches resolved IP address for admission control
service. The Impala server runs a heart beat thread for admission
control service and it relies on the cached resolved address in ExecEnv.
This breaks when IP address for admission control services changes.

This patch removes the 'admission_service_address_' member and its
getter in the ExecEnv class and adds a GetAdmissionServiceAddress()
interface to dynamically get the resolved address, if global admission
control service is enabled ('admission_service_host' flag is set).

Testing: Manually tested by restarting global admission controller
service in a Kubernetes setup where it gets new IP address on startup.
Coordinator's admission heart beat thread was able to connect to the
admission control service using the new IP address. Also made sure that
the coordinator and admissiond communication was working as expected
upon admissiond restarts.

Change-Id: I09b4c52644f9e1b3c1f0e68c9900464d722517af
Reviewed-on: http://gerrit.cloudera.org:8080/19403
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/runtime/exec-env.cc
M be/src/runtime/exec-env.h
M be/src/scheduling/admission-control-service.cc
M be/src/service/impala-server.cc
4 files changed, 21 insertions(+), 18 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/19403
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I09b4c52644f9e1b3c1f0e68c9900464d722517af
Gerrit-Change-Number: 19403
Gerrit-PatchSet: 4
Gerrit-Owner: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[Impala-ASF-CR] IMPALA-11827: do not cache admission control service's IP address in impalad

Posted by "Andrew Sherman (Code Review)" <ge...@cloudera.org>.
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/19403 )

Change subject: IMPALA-11827: do not cache admission control service's IP address in impalad
......................................................................


Patch Set 2: Code-Review+2

LGTM


-- 
To view, visit http://gerrit.cloudera.org:8080/19403
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I09b4c52644f9e1b3c1f0e68c9900464d722517af
Gerrit-Change-Number: 19403
Gerrit-PatchSet: 2
Gerrit-Owner: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 06 Jan 2023 00:37:46 +0000
Gerrit-HasComments: No