You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Michael Ho (Code Review)" <ge...@cloudera.org> on 2018/07/18 18:22:07 UTC
[Impala-ASF-CR] IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
Michael Ho has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10980
Change subject: IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
......................................................................
IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
Previously, we pass the resolved IP address of a KRPC destination
host as the hostname when creating a proxy for making KRPC calls.
This may lead to connection negotiation failure in KRPC when Kerberos
is enabled. In particular, if reversed DNS isn't enabled in Kerberos,
KDC may fail to look up the principal of the destination host if the
principal includes the hostname instead of resolved IP address.
This change fixes the problem above by passing the actual hostname
of the destination host when calling RpcMgr::GetProxy().
rpc-mgr-kerberized-test.cc is also updated to use hostname
instead of the resolved IP address as Kerberos principal.
Change-Id: I3e3e978746cf03766eee151835aad5877d9ed63e
---
M be/src/rpc/rpc-mgr-kerberized-test.cc
M be/src/rpc/rpc-mgr-test-base.h
M be/src/rpc/rpc-mgr-test.cc
M be/src/rpc/rpc-mgr.h
M be/src/rpc/rpc-mgr.inline.h
M be/src/runtime/data-stream-sender.cc
M be/src/runtime/data-stream-test.cc
M be/src/runtime/krpc-data-stream-sender.cc
M be/src/scheduling/scheduler.cc
M common/thrift/ImpalaInternalService.thrift
10 files changed, 33 insertions(+), 27 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/80/10980/1
--
To view, visit http://gerrit.cloudera.org:8080/10980
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I3e3e978746cf03766eee151835aad5877d9ed63e
Gerrit-Change-Number: 10980
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
[Impala-ASF-CR] IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
Posted by "Michael Ho (Code Review)" <ge...@cloudera.org>.
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/10980 )
Change subject: IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/10980/1/be/src/runtime/krpc-data-stream-sender.cc
File be/src/runtime/krpc-data-stream-sender.cc:
http://gerrit.cloudera.org:8080/#/c/10980/1/be/src/runtime/krpc-data-stream-sender.cc@598
PS1, Line 598: destinations[i].thrift_backend.hostname,
> Instead of adding this new parameter, would it be possible to ensure that "
destinations[i].krpc_backend.hostname is actually set to the resolved IP address of hostname so we have to pull it from thrift_backend.hosname. As we remove Thrift from communications between Impala backends in the future, we need to move thrift_backend.hostname into a separate field in TPlanFragmentDestination.
--
To view, visit http://gerrit.cloudera.org:8080/10980
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3e3e978746cf03766eee151835aad5877d9ed63e
Gerrit-Change-Number: 10980
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sa...@cloudera.com>
Gerrit-Comment-Date: Thu, 19 Jul 2018 00:47:50 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10980 )
Change subject: IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
......................................................................
Patch Set 2: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/10980
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3e3e978746cf03766eee151835aad5877d9ed63e
Gerrit-Change-Number: 10980
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sa...@cloudera.com>
Gerrit-Comment-Date: Thu, 19 Jul 2018 05:00:09 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10980 )
Change subject: IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
......................................................................
Patch Set 2:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2839/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/10980
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3e3e978746cf03766eee151835aad5877d9ed63e
Gerrit-Change-Number: 10980
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sa...@cloudera.com>
Gerrit-Comment-Date: Thu, 19 Jul 2018 01:57:27 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
Posted by "Sailesh Mukil (Code Review)" <ge...@cloudera.org>.
Sailesh Mukil has posted comments on this change. ( http://gerrit.cloudera.org:8080/10980 )
Change subject: IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
......................................................................
Patch Set 1: Code-Review+2
Forgot to add +2
--
To view, visit http://gerrit.cloudera.org:8080/10980
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3e3e978746cf03766eee151835aad5877d9ed63e
Gerrit-Change-Number: 10980
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sa...@cloudera.com>
Gerrit-Comment-Date: Thu, 19 Jul 2018 00:54:23 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10980 )
Change subject: IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
......................................................................
IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
Previously, we pass the resolved IP address of a KRPC destination
host as the hostname when creating a proxy for making KRPC calls.
This may lead to connection negotiation failure in KRPC when Kerberos
is enabled. In particular, if reversed DNS isn't enabled in Kerberos,
KDC may fail to look up the principal of the destination host if the
principal includes the hostname instead of resolved IP address.
This change fixes the problem above by passing the actual hostname
of the destination host when calling RpcMgr::GetProxy().
rpc-mgr-kerberized-test.cc is also updated to use hostname
instead of the resolved IP address as Kerberos principal.
Change-Id: I3e3e978746cf03766eee151835aad5877d9ed63e
Reviewed-on: http://gerrit.cloudera.org:8080/10980
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/rpc/rpc-mgr-kerberized-test.cc
M be/src/rpc/rpc-mgr-test-base.h
M be/src/rpc/rpc-mgr-test.cc
M be/src/rpc/rpc-mgr.h
M be/src/rpc/rpc-mgr.inline.h
M be/src/runtime/data-stream-sender.cc
M be/src/runtime/data-stream-test.cc
M be/src/runtime/krpc-data-stream-sender.cc
M be/src/scheduling/scheduler.cc
M common/thrift/ImpalaInternalService.thrift
10 files changed, 33 insertions(+), 27 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/10980
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I3e3e978746cf03766eee151835aad5877d9ed63e
Gerrit-Change-Number: 10980
Gerrit-PatchSet: 3
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sa...@cloudera.com>
[Impala-ASF-CR] IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
Posted by "Sailesh Mukil (Code Review)" <ge...@cloudera.org>.
Sailesh Mukil has posted comments on this change. ( http://gerrit.cloudera.org:8080/10980 )
Change subject: IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
......................................................................
Patch Set 1:
(1 comment)
Patch LGTM. Thanks for doing this.
http://gerrit.cloudera.org:8080/#/c/10980/1/be/src/runtime/krpc-data-stream-sender.cc
File be/src/runtime/krpc-data-stream-sender.cc:
http://gerrit.cloudera.org:8080/#/c/10980/1/be/src/runtime/krpc-data-stream-sender.cc@598
PS1, Line 598: destinations[i].thrift_backend.hostname,
> destinations[i].krpc_backend.hostname is actually set to the resolved IP ad
Ok thanks for the explanation.
--
To view, visit http://gerrit.cloudera.org:8080/10980
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3e3e978746cf03766eee151835aad5877d9ed63e
Gerrit-Change-Number: 10980
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sa...@cloudera.com>
Gerrit-Comment-Date: Thu, 19 Jul 2018 00:53:59 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
Posted by "Sailesh Mukil (Code Review)" <ge...@cloudera.org>.
Sailesh Mukil has posted comments on this change. ( http://gerrit.cloudera.org:8080/10980 )
Change subject: IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/10980/1/be/src/runtime/krpc-data-stream-sender.cc
File be/src/runtime/krpc-data-stream-sender.cc:
http://gerrit.cloudera.org:8080/#/c/10980/1/be/src/runtime/krpc-data-stream-sender.cc@598
PS1, Line 598: destinations[i].thrift_backend.hostname,
Instead of adding this new parameter, would it be possible to ensure that "destinations[i].krpc_backend.hostname" is set? That way, we can avoid a lot of code changes.
--
To view, visit http://gerrit.cloudera.org:8080/10980
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3e3e978746cf03766eee151835aad5877d9ed63e
Gerrit-Change-Number: 10980
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sa...@cloudera.com>
Gerrit-Comment-Date: Thu, 19 Jul 2018 00:36:31 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10980 )
Change subject: IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
......................................................................
Patch Set 2: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/10980
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3e3e978746cf03766eee151835aad5877d9ed63e
Gerrit-Change-Number: 10980
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Ho <kw...@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sa...@cloudera.com>
Gerrit-Comment-Date: Thu, 19 Jul 2018 01:57:26 +0000
Gerrit-HasComments: No