You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Vincent Tran (Code Review)" <ge...@cloudera.org> on 2017/06/21 02:19:00 UTC

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos. 

Vincent Tran has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/7241

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................

IMPALA-2782: Allow impala-shell to connect directly to impalad when
configured with load balancer and kerberos.

This change adds an impala-shell option -b <host:port>. This allows
user to optionally specify the load-balancer's host:port so that
impala-shell will accept a direct connection to impala daemons in
a kerberized cluster.

Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
---
M shell/impala_client.py
M shell/impala_shell.py
M shell/option_parser.py
3 files changed, 11 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/7241/1
-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/7241 )

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................


Patch Set 5:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2144/


-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-Change-Number: 7241
Gerrit-PatchSet: 5
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: andy@phdata.io
Gerrit-Comment-Date: Wed, 21 Mar 2018 16:57:18 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.

Posted by "Dan Hecht (Code Review)" <ge...@cloudera.org>.
Dan Hecht has posted comments on this change. ( http://gerrit.cloudera.org:8080/7241 )

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................


Patch Set 2:

Ping, what's the next step here. This CR is quite old...


-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-Change-Number: 7241
Gerrit-PatchSet: 2
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: andy@phdata.io
Gerrit-Comment-Date: Thu, 16 Nov 2017 01:20:31 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.

Posted by "Philip Zeyliger (Code Review)" <ge...@cloudera.org>.
Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/7241 )

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................


Patch Set 4:

astadler: So you're saying that it's the client that is checking the host section of the principal it's connecting to, and that's what we're overriding. If so, for consistency with JDBC, I think it's a fine thing to do this.

> The kerberos_host_fqdn option exposes the SASL client's hostname attribute to
> If set, it will be the sasl transport client's hostname used" " to authenticate via kerberos")

Both of these strings confused me. It's not meaningfully the "client's hostname" in my reading of it.

Perhaps:
"If set, overrides the expected hostname of the Impalad's kerberos service principal. impala-shell will check that the server's principal matches this hostname. This may be used when impalad is configured to be accessed via a load-balancer, but it is desired for impala-shell to talk to a specific impalad directly."

Is that accurate? Do you think it's clearer?

Does impyla need a similar option?


-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-Change-Number: 7241
Gerrit-PatchSet: 4
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: andy@phdata.io
Gerrit-Comment-Date: Sat, 17 Mar 2018 16:50:48 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.

Posted by "Vincent Tran (Code Review)" <ge...@cloudera.org>.
Hello andy@phdata.io, Lars Volker, Philip Zeyliger, Tim Armstrong, Alex Behm, Dan Hecht, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/7241

to look at the new patch set (#5).

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................

IMPALA-2782: Allow impala-shell to connect directly to impalad when
configured with load balancer and kerberos.

This change adds an impala-shell option -b / --kerberos_host_fqdn.
This allows user to optionally specify the load-balancer's host so
that impala-shell will accept a direct connection to impala daemons
in a kerberized cluster.

Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
---
M shell/impala_client.py
M shell/impala_shell.py
M shell/impala_shell_config_defaults.py
M shell/option_parser.py
4 files changed, 24 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/7241/5
-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-Change-Number: 7241
Gerrit-PatchSet: 5
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: andy@phdata.io

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.

Posted by "Vincent Tran (Code Review)" <ge...@cloudera.org>.
Vincent Tran has uploaded a new patch set (#2).

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................

IMPALA-2782: Allow impala-shell to connect directly to impalad when
configured with load balancer and kerberos.

This change adds an impala-shell option -b <host:port>. This allows
user to optionally specify the load-balancer's host:port so that
impala-shell will accept a direct connection to impala daemons in
a kerberized cluster.

Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
---
M shell/impala_client.py
M shell/impala_shell.py
M shell/impala_shell_config_defaults.py
M shell/option_parser.py
4 files changed, 12 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/7241/2
-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/7241 )

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................


Patch Set 5: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-Change-Number: 7241
Gerrit-PatchSet: 5
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: andy@phdata.io
Gerrit-Comment-Date: Wed, 21 Mar 2018 20:45:47 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/7241 )

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................


Patch Set 4:

It looks like this got stuck. It sounds like there's some uncertainty about whether this solution is worth doing or whether we should be implementing a more comprehensive solution that requires less user configuration. Should we have that discussion on the JIRA? Or do people want to move forward with this approach?


-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-Change-Number: 7241
Gerrit-PatchSet: 4
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: andy@phdata.io
Gerrit-Comment-Date: Thu, 15 Mar 2018 00:42:29 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.

Posted by "Vincent Tran (Code Review)" <ge...@cloudera.org>.
Vincent Tran has posted comments on this change.

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................


Patch Set 2:

Added default option as None. Still missing a test though, I will make sure to work on the test after I get feedback.

-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.

Posted by "Philip Zeyliger (Code Review)" <ge...@cloudera.org>.
Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/7241 )

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................


Patch Set 5: Code-Review+2

Thanks for the contribution!


-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-Change-Number: 7241
Gerrit-PatchSet: 5
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: andy@phdata.io
Gerrit-Comment-Date: Wed, 21 Mar 2018 16:57:10 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.

Posted by "Vincent Tran (Code Review)" <ge...@cloudera.org>.
Vincent Tran has posted comments on this change.

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................


Patch Set 1:

I'm not sure how to write a test for this one as the mini cluster lacks kerberos and lb. Looking for suggestions.

-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/7241 )

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................

IMPALA-2782: Allow impala-shell to connect directly to impalad when
configured with load balancer and kerberos.

This change adds an impala-shell option -b / --kerberos_host_fqdn.
This allows user to optionally specify the load-balancer's host so
that impala-shell will accept a direct connection to impala daemons
in a kerberized cluster.

Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Reviewed-on: http://gerrit.cloudera.org:8080/7241
Reviewed-by: <an...@phdata.io>
Reviewed-by: Philip Zeyliger <ph...@cloudera.com>
Tested-by: Impala Public Jenkins
---
M shell/impala_client.py
M shell/impala_shell.py
M shell/impala_shell_config_defaults.py
M shell/option_parser.py
4 files changed, 24 insertions(+), 5 deletions(-)

Approvals:
  andy@phdata.io: Looks good to me, but someone else must approve
  Philip Zeyliger: Looks good to me, approved
  Impala Public Jenkins: Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-Change-Number: 7241
Gerrit-PatchSet: 6
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: andy@phdata.io

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
andy@phdata.io has posted comments on this change. ( http://gerrit.cloudera.org:8080/7241 )

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................


Patch Set 4:

Tim I for one am good with the latest patch changes. As far as Philips concern about always assuming the hostname is in the service principal we could start forcing people to specify and accept _HOST as a wildcard to use the hostname supplied from the -i input but I think that would be a change that affected all end-users and I think this is really only going to be used for by administrators and engineers for debugging so it's probably preferred to do it this way since most users won't care about this feature.


-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-Change-Number: 7241
Gerrit-PatchSet: 4
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: andy@phdata.io
Gerrit-Comment-Date: Fri, 16 Mar 2018 13:37:13 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.

Posted by "Vincent Tran (Code Review)" <ge...@cloudera.org>.
Hello andy@phdata.io, Lars Volker, Philip Zeyliger, Alex Behm, Dan Hecht, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/7241

to look at the new patch set (#4).

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................

IMPALA-2782: Allow impala-shell to connect directly to impalad when
configured with load balancer and kerberos.

This change adds an impala-shell option -b / --kerberos_host_fqdn.
This allows user to optionally specify the load-balancer's host so
that impala-shell will accept a direct connection to impala daemons
in a kerberized cluster.

Testing:
Deployed the change to a kerberized and load-balanced nightly cluster
and verified manually that the option allows succesful direct connection
to the impala daemons.

Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
---
M shell/impala_client.py
M shell/impala_shell.py
M shell/impala_shell_config_defaults.py
M shell/option_parser.py
4 files changed, 17 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/7241/4
-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-Change-Number: 7241
Gerrit-PatchSet: 4
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: andy@phdata.io

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
andy@phdata.io has posted comments on this change.

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................


Patch Set 2:

I think we should use impalad[1] for the port still since then you'll still get the default 21000 if you specify nothing and its less you would have to enter for the -b flag. I also question whether this should be called something like kerberos_host_name so its more inline with kerberos_service_name change. 

host, port = self.lb.encode('ascii', 'ignore'), int(self.impalad[1])

-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: andy@phdata.io
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
andy@phdata.io has posted comments on this change. ( http://gerrit.cloudera.org:8080/7241 )

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................


Patch Set 2:

Dan I'm willing to take this we've been looking for this for a while at customers.


-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-Change-Number: 7241
Gerrit-PatchSet: 2
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: andy@phdata.io
Gerrit-Comment-Date: Wed, 29 Nov 2017 19:09:59 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
andy@phdata.io has posted comments on this change. ( http://gerrit.cloudera.org:8080/7241 )

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................


Patch Set 4:

Philip,

I agree that makes more sense since the "client" Vincent is referring to is sasl.Client() which is not exposed to the user and would probably just confuse users.

I don't think it would hurt to add it to impyla, it might be useful to be able to go directly to a daemon when the load-balancer is configured.


-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-Change-Number: 7241
Gerrit-PatchSet: 4
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: andy@phdata.io
Gerrit-Comment-Date: Sat, 17 Mar 2018 17:59:20 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.

Posted by "Dan Hecht (Code Review)" <ge...@cloudera.org>.
Dan Hecht has posted comments on this change. ( http://gerrit.cloudera.org:8080/7241 )

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................


Patch Set 2:

> Dan I'm willing to take this we've been looking for this for a
 > while at customers.

Works for me. Vincent can chime in if he prefers to move this forward himself.

Also looks like Vincent was looking for some feedback on the approach. Do you, or anyone else, have any?


-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-Change-Number: 7241
Gerrit-PatchSet: 2
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: andy@phdata.io
Gerrit-Comment-Date: Wed, 29 Nov 2017 21:45:13 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.

Posted by "Vincent Tran (Code Review)" <ge...@cloudera.org>.
Hello andy@phdata.io, Philip Zeyliger, Dan Hecht, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/7241

to look at the new patch set (#3).

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................

IMPALA-2782: Allow impala-shell to connect directly to impalad when
configured with load balancer and kerberos.

This change adds an impala-shell option -b / --kerberos_host_fqdn.
This allows user to optionally specify the load-balancer's host so
that impala-shell will accept a direct connection to impala daemons
in a kerberized cluster.

Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
---
M shell/impala_client.py
M shell/impala_shell.py
M shell/impala_shell_config_defaults.py
M shell/option_parser.py
4 files changed, 17 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/7241/3
-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-Change-Number: 7241
Gerrit-PatchSet: 3
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: andy@phdata.io

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.

Posted by "Vincent Tran (Code Review)" <ge...@cloudera.org>.
Vincent Tran has posted comments on this change. ( http://gerrit.cloudera.org:8080/7241 )

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................


Patch Set 2:

> Patch Set 2:
> 
> I think we should use impalad[1] for the port still since then you'll still get the default 21000 if you specify nothing and its less you would have to enter for the -b flag. I also question whether this should be called something like kerberos_host_name so its more inline with kerberos_service_name change. 
> 
> host, port = self.lb.encode('ascii', 'ignore'), int(self.impalad[1])

Good point.


-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-Change-Number: 7241
Gerrit-PatchSet: 2
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: andy@phdata.io
Gerrit-Comment-Date: Sun, 11 Feb 2018 03:04:30 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
andy@phdata.io has posted comments on this change. ( http://gerrit.cloudera.org:8080/7241 )

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................


Patch Set 5: Code-Review+1

LGTM


-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-Change-Number: 7241
Gerrit-PatchSet: 5
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: andy@phdata.io
Gerrit-Comment-Date: Wed, 21 Mar 2018 14:54:59 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.

Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
andy@phdata.io has posted comments on this change. ( http://gerrit.cloudera.org:8080/7241 )

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................


Patch Set 4:

There is no security bug. The issue is that when you add a load-balancer in front of impala, or any service for that matter you have to change the hostname in the service principal to be that of the load balancer url e.g. service/hostname to impala/loadbalancer.apache.org, since you as the client don't know which host you are going to get when going through a load-balancer.

The problem we are trying to fix is that impala-shell client currently doesn't let you configure the hostname of the daemon you want to connect to and the host section of the service principal you are expecting from the daemon independently. Currently it always assumes they are the same https://github.com/apache/impala/blob/63f17e9ceaed92a28ea12567a36b746e54fffdb3/shell/impala_client.py#L278. So when you want to go around the load-balancer and target a daemon directly to troubleshoot a load-balancer issue for example you can not do so using impala-shell. 

This functionality is already in the JDBC driver by configuring KrbHostFQDN=node1.example.com;KrbServiceName=impala . Obviously there is situations where you would like to be able to test api v1 and api v2 which is why we are trying to implement this feature that already exists in the JDBC driver in impala-shell as well.

Hope that clarifies things.


-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-Change-Number: 7241
Gerrit-PatchSet: 4
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: andy@phdata.io
Gerrit-Comment-Date: Sat, 17 Mar 2018 16:24:51 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.

Posted by "Vincent Tran (Code Review)" <ge...@cloudera.org>.
Vincent Tran has posted comments on this change. ( http://gerrit.cloudera.org:8080/7241 )

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................


Patch Set 2:

> Patch Set 2:
> 
> I had to look at the JIRA to understand what this is trying to accomplish. That suggests that the important change in impala_client.py (where you set up the SASL) needs explanation. You've changed what Thrift/SASL/Kerberos is expecting, and it's worth having a comment that explains the scenario.
> 
> That said, I think Hrishikesh Gadre's comment on the JIRA is also worth addressing. It feels weird that when load-balancers are configured, impalad always has this issue (that it impersonates the load balancer?). Is that condition relaxable? The change you have works here, but it would need to be repeated for JDBC drivers and for impyla and anything else.
> 
> As a nit, I think "self.lb" is a poor variable name; self.load_balancer is better. Even there, I'd worry that the client is going to get confused: the point of the load balancer is that it acts like an impalad, so I worry people are going to specify both their impalad (?) and their load-balancer, even in scenarios when they don't need to. These are quite user-visible flags.

Good point on the lack of explanation.

In the Simba JDBC driver, the equivalent variable is already exposed as KrbHostFQDN. In the ODBC driver, the field is exposed as "Host FQDN". Both of which are still somewhat confusing to a neophyte like myself.

As for the variable name, perhaps we can use a similar name (i.e. self.KrbHostFQDN) for the sake of consistency.
I can also lean the other way (i.e. self.load_balancer) since it is more appropriate for its purpose in my opinion.

What are your thoughts?


-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-Change-Number: 7241
Gerrit-PatchSet: 2
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: andy@phdata.io
Gerrit-Comment-Date: Sun, 11 Feb 2018 02:54:00 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.

Posted by "Philip Zeyliger (Code Review)" <ge...@cloudera.org>.
Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/7241 )

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................


Patch Set 4:

I'm having a little trouble understanding how this all works. Where does the server check the client's hostname? And is it a bug that it can be impersonated so easily?


-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-Change-Number: 7241
Gerrit-PatchSet: 4
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: andy@phdata.io
Gerrit-Comment-Date: Sat, 17 Mar 2018 03:33:50 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.

Posted by "Vincent Tran (Code Review)" <ge...@cloudera.org>.
Vincent Tran has posted comments on this change. ( http://gerrit.cloudera.org:8080/7241 )

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................


Patch Set 2:

Andy,

I'm happy for you to take over this change.
Completely agrees on the nits.


-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-Change-Number: 7241
Gerrit-PatchSet: 2
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: andy@phdata.io
Gerrit-Comment-Date: Wed, 29 Nov 2017 23:50:37 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.

Posted by "Vincent Tran (Code Review)" <ge...@cloudera.org>.
Vincent Tran has posted comments on this change. ( http://gerrit.cloudera.org:8080/7241 )

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................


Patch Set 5:

Thank you for the feedback, all.
Addressed the verbiage concerns.


-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-Change-Number: 7241
Gerrit-PatchSet: 5
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: andy@phdata.io
Gerrit-Comment-Date: Wed, 21 Mar 2018 11:06:52 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.

Posted by "Philip Zeyliger (Code Review)" <ge...@cloudera.org>.
Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/7241 )

Change subject: IMPALA-2782: Allow impala-shell to connect directly to impalad when configured with load balancer and kerberos.
......................................................................


Patch Set 2:

I had to look at the JIRA to understand what this is trying to accomplish. That suggests that the important change in impala_client.py (where you set up the SASL) needs explanation. You've changed what Thrift/SASL/Kerberos is expecting, and it's worth having a comment that explains the scenario.

That said, I think Hrishikesh Gadre's comment on the JIRA is also worth addressing. It feels weird that when load-balancers are configured, impalad always has this issue (that it impersonates the load balancer?). Is that condition relaxable? The change you have works here, but it would need to be repeated for JDBC drivers and for impyla and anything else.

As a nit, I think "self.lb" is a poor variable name; self.load_balancer is better. Even there, I'd worry that the client is going to get confused: the point of the load balancer is that it acts like an impalad, so I worry people are going to specify both their impalad (?) and their load-balancer, even in scenarios when they don't need to. These are quite user-visible flags.


-- 
To view, visit http://gerrit.cloudera.org:8080/7241
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4726226a7a3817421b133f74dd4f4cf8c52135f9
Gerrit-Change-Number: 7241
Gerrit-PatchSet: 2
Gerrit-Owner: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Reviewer: andy@phdata.io
Gerrit-Comment-Date: Wed, 29 Nov 2017 22:00:50 +0000
Gerrit-HasComments: No