You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org> on 2021/01/13 21:41:54 UTC
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Csaba Ringhofer has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16833
Change subject: [WiP] SAML implementation in Impala
......................................................................
[WiP] SAML implementation in Impala
The bulk of the SAML2 related code is done on Java side because:
- There is already a POC in Hive that could be reused.
- The only SAML lib for c++ seems to be OpenSaml, which is seemed
quite hard to use and a heavy dependency.
Doing authentication in Java needed some plumbing, as the hs2-http
port is listened to in c++ and http related processing happens in
THttpServer/THttpTransport, which is not a "real" web server, just
a simple http implementation that processes the headers and passes
content to the thrift service.
- Http headers (and in one case body) are inspected and if it is
SAML related, the http request is wrapped in TWrappedHttpRequest
and sent to the Frontend. The Frontend processes it and returns
a TWrappedHttpResponse with the info to return to the client.
- After the last SAML message (with the bearer token) we generate
an auth cookie in c++ (which can be validated in c++), so later
requests in the session don't need to call to Java.
State of implementation:
- The java side is more or less ok, will be updated when the Hive
implementation changes. I would do a proper cleanup / documentation
once the Hive code is more final.
- The c++ side probsbluneeds a lot of cleanup.
- Compatibility with other auth mechanisms should be decided:
- Whether other clients should be able to auth with ldap/kerberos
is not clear yet.
- Only the first step of the auth workflow have automatic tests.
Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
---
M be/src/common/global-flags.cc
M be/src/rpc/auth-provider.h
M be/src/rpc/authentication-test.cc
M be/src/rpc/authentication.cc
M be/src/rpc/authentication.h
M be/src/rpc/hs2-http-test.cc
M be/src/rpc/thrift-server.h
M be/src/service/frontend.cc
M be/src/service/frontend.h
M be/src/service/impala-server.cc
M be/src/transport/THttpServer.cpp
M be/src/transport/THttpServer.h
M be/src/transport/THttpTransport.cpp
M be/src/transport/THttpTransport.h
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
M common/thrift/Frontend.thrift
M common/thrift/metrics.json
M fe/pom.xml
A fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlHttpServlet.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlUtils.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlAuthenticationException.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlNoGroupsMatchedException.java
A fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
A fe/src/main/java/org/apache/impala/authentication/saml/NullSessionStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/WrappedWebContext.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
M java/pom.xml
A test_saml.py
A testdata/authentication/saml2_sso.jks
A testdata/authentication/saml2_sso_metadata.xml
A tests/custom_cluster/test_saml2_sso.py
39 files changed, 1,976 insertions(+), 51 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/16833/9
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 9
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
Patch Set 24:
(13 comments)
http://gerrit.cloudera.org:8080/#/c/16833/23//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/16833/23//COMMIT_MSG@10
PS23, Line 10: implementa
> I think mentioning the upstream HIVE jira (HIVE-24543) here would be a usef
Done
http://gerrit.cloudera.org:8080/#/c/16833/23/be/src/common/global-flags.cc
File be/src/common/global-flags.cc:
http://gerrit.cloudera.org:8080/#/c/16833/23/be/src/common/global-flags.cc@352
PS23, Line 352: // ++========================++
> nit, blank new line.
Done
http://gerrit.cloudera.org:8080/#/c/16833/23/be/src/rpc/authentication.cc
File be/src/rpc/authentication.cc:
http://gerrit.cloudera.org:8080/#/c/16833/23/be/src/rpc/authentication.cc@1338
PS23, Line 1338: LOG(INFO) << "External communication can be also authenticated with SAML2 SSO";
> do we need to add a similar msg as above for SAML?
Done
http://gerrit.cloudera.org:8080/#/c/16833/23/be/src/service/frontend.h
File be/src/service/frontend.h:
http://gerrit.cloudera.org:8080/#/c/16833/23/be/src/service/frontend.h@201
PS23, Line 201: // Call FE to create a http response that redirects to the SSO service.
: Status GetSaml2Redirect(const TWrappedHttpRequest& request,
: TWrappedHttpResponse* response);
:
: // Call FE to validate the SAML2 AuthNResponse.
> Can we add some comments here?
Done
http://gerrit.cloudera.org:8080/#/c/16833/23/common/thrift/metrics.json
File common/thrift/metrics.json:
http://gerrit.cloudera.org:8080/#/c/16833/23/common/thrift/metrics.json@1318
PS23, Line 1318: ],
: "label": "Hive
> Do we need this for catalogd and statestored?
Thanks for spotting - actually the description and the label were also wrong.
http://gerrit.cloudera.org:8080/#/c/16833/23/common/thrift/metrics.json@1330
PS23, Line 1330: "units": "NONE",
: "kind": "COUNT
> Do we need this for catalogd and statestored?
Done
http://gerrit.cloudera.org:8080/#/c/16833/23/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/23/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java@20
PS23, Line 20: https://github.com/vihangk1/hive/blob/master_saml
> This points to a private branch. Unfortunately, we have not merged this yet
Yes, I plan to update these once the Hive patch is merged, but this may happen in a follow up patch if this change is merged before Hive.
http://gerrit.cloudera.org:8080/#/c/16833/23/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlHttpServlet.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlHttpServlet.java:
http://gerrit.cloudera.org:8080/#/c/16833/23/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlHttpServlet.java@58
PS23, Line 58: generateFormData(webContext, "http://127.0.0.1:" +
> This can be removed. Also, the latest hive patch uses 127.0.0.1 instead of
Done
http://gerrit.cloudera.org:8080/#/c/16833/23/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlHttpServlet.java@66
PS23, Line 66: tokenGenerator.get(nameId, relayState), true,
> this can be removed. Also, please replace localhost with 127.0.0.1
Done
http://gerrit.cloudera.org:8080/#/c/16833/23/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java:
http://gerrit.cloudera.org:8080/#/c/16833/23/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java@23
PS23, Line 23: g clientIden
> the latest hive patch renames this field to clientIdentifier which is more
Done
http://gerrit.cloudera.org:8080/#/c/16833/23/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
File fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java:
http://gerrit.cloudera.org:8080/#/c/16833/23/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@76
PS23, Line 76:
> Do we need to validate this call back URL? e.g. the port number is same as
This path is already parsed in the backend, so I think that it is easier to check this there. I will add validation in a later patch.
http://gerrit.cloudera.org:8080/#/c/16833/23/tests/custom_cluster/test_saml2_sso.py
File tests/custom_cluster/test_saml2_sso.py:
http://gerrit.cloudera.org:8080/#/c/16833/23/tests/custom_cluster/test_saml2_sso.py@37
PS23, Line 37: return response
> I think it would be generally more readable if we have a class level commen
Done
http://gerrit.cloudera.org:8080/#/c/16833/23/tests/custom_cluster/test_saml2_sso.py@71
PS23, Line 71: ge
> nit, WITH
Done
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 24
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 Feb 2021 00:26:18 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Hello Thomas Tauber-Marshall, Vihang Karajgaonkar, Joe McDonnell, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16833
to look at the new patch set (#24).
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
IMPALA-10496: SAML implementation in Impala
The bulk of the SAML2 related code is done on Java side because:
- There is already an implementation for Hive on review (HIVE-24543).
- The only SAML lib for c++ seems to be OpenSaml, which is seemed
quite hard to use and a heavy dependency.
Doing authentication in Java needed some plumbing, as the hs2-http
port is listened to in c++ and http related processing happens in
THttpServer/THttpTransport, which is not a "real" web server, just
a simple http implementation that processes the headers and passes
content to the thrift service.
- Http headers (and in one case body) are inspected and if it is
SAML related, the http request is wrapped in TWrappedHttpRequest
and sent to the Frontend. The Frontend processes it and returns
a TWrappedHttpResponse with the info to return to the client.
- After the last SAML message (with the bearer token) we generate
an auth cookie in c++ (which can be validated in c++), so later
requests in the session don't need to call to Java.
SAML auth can work alongside LDAP and Kerberos - for each hs2-http
request the path and the http headers are inspected to decide
whether it is SAML related, and if not, then we fallback to other
auth mechanisms. This "mixed mode" has no tests yet, so I consider it
experimental.
Planned followup work:
- It would be great to import the logic implemented in Hive instead
of copy-pasting most of it. I plan to do this in a followup commit,
as this needs changes on the Hive side too.
- Adding more tests will be much easier once we will have a hs2-http
client that supports SAML. See IMPALA-10496 for Impyla support.
Testing:
- Added EE tests that use Python's urllib2 to sent SAML
requests to Impala. Impala works slightly differently
during tests (saml2_ee_test_mode=true).
Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
---
M be/src/rpc/auth-provider.h
M be/src/rpc/authentication-test.cc
M be/src/rpc/authentication.cc
M be/src/rpc/authentication.h
M be/src/rpc/hs2-http-test.cc
M be/src/rpc/thrift-server.h
M be/src/service/frontend.cc
M be/src/service/frontend.h
M be/src/service/impala-server.cc
M be/src/transport/THttpServer.cpp
M be/src/transport/THttpServer.h
M be/src/transport/THttpTransport.cpp
M be/src/transport/THttpTransport.h
M be/src/util/backend-gflag-util.cc
M bin/rat_exclude_files.txt
M common/thrift/BackendGflags.thrift
M common/thrift/Frontend.thrift
M common/thrift/metrics.json
M fe/pom.xml
A fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlHttpServlet.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlUtils.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlAuthenticationException.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlNoGroupsMatchedException.java
A fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
A fe/src/main/java/org/apache/impala/authentication/saml/NullSessionStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/WrappedWebContext.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
M java/pom.xml
A testdata/authentication/saml2_sso.jks
A testdata/authentication/saml2_sso_metadata.xml
A tests/custom_cluster/test_saml2_sso.py
38 files changed, 2,166 insertions(+), 49 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/16833/24
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 24
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Vihang Karajgaonkar (Code Review)" <ge...@cloudera.org>.
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: [WiP] SAML implementation in Impala
......................................................................
Patch Set 15:
(10 comments)
Thanks for taking this up! I took a first pass and left some questions/suggestions. I will take another pass at it once you respond/update the patch.
http://gerrit.cloudera.org:8080/#/c/16833/15/be/src/common/global-flags.cc
File be/src/common/global-flags.cc:
http://gerrit.cloudera.org:8080/#/c/16833/15/be/src/common/global-flags.cc@347
PS15, Line 347: DEFINE_string(saml2_keystore_path, "", "Keystore path to the saml2 client.");
I guess some of these configurations have place holder descriptions and you plan to add more details in subsequent patchset?
http://gerrit.cloudera.org:8080/#/c/16833/15/be/src/rpc/authentication.cc
File be/src/rpc/authentication.cc:
http://gerrit.cloudera.org:8080/#/c/16833/15/be/src/rpc/authentication.cc@599
PS15, Line 599: porpuses
nit, spelling
http://gerrit.cloudera.org:8080/#/c/16833/15/be/src/transport/THttpServer.h
File be/src/transport/THttpServer.h:
http://gerrit.cloudera.org:8080/#/c/16833/15/be/src/transport/THttpServer.h@165
PS15, Line 165: // SAML can be used alongside LDAP or Kerberos - if the SAML related path of headers
: // are not detected, Impala fall back to other authentications.
: bool has_saml_ = false;
Does this mean that Impala can support both SAML and Ldap concurrently? Has it always been the case or something which was added in this patch?
http://gerrit.cloudera.org:8080/#/c/16833/15/be/src/transport/THttpServer.cpp
File be/src/transport/THttpServer.cpp:
http://gerrit.cloudera.org:8080/#/c/16833/15/be/src/transport/THttpServer.cpp@137
PS15, Line 137: Token
Are you planning to add engine specific identifiers in these header names? eg. X-Impala-Token-Response-Port instead of X-Token-Response-Port. In case of Hive, we use X-Hive-Token-Response-Port
http://gerrit.cloudera.org:8080/#/c/16833/15/be/src/transport/THttpServer.cpp@261
PS15, Line 261: authorized
if the bearer token is present but not valid, do we reach here?
http://gerrit.cloudera.org:8080/#/c/16833/15/common/thrift/BackendGflags.thrift
File common/thrift/BackendGflags.thrift:
http://gerrit.cloudera.org:8080/#/c/16833/15/common/thrift/BackendGflags.thrift@185
PS15, Line 185: roken
s/roken/token/
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/pom.xml
File fe/pom.xml:
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/pom.xml@491
PS15, Line 491: <dependency>
pac4j brings in tons of dependencies. We should exclude as much as possible. Also, I found that it includes some dependencies which have known security vulnerabilities. I have excluded those and separated added the updated versions here: https://github.com/apache/hive/pull/1791/commits/e92e911dd33a19f938abb44aa8d7364d9fcc0129 You may want to do the same. the other way (may be a followup later) could be to add the Hive classes with the hive-exec reduced jar and try to reuse them.
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@39
PS15, Line 39: HiveSamlAuthTokenGenerator
Ideally, it would great to use these classes by importing them via hive-exec reduced jar that we already have in the classpath. That way updates to this class would be available for free to us. On the flip side, hive side should make sure changes to these files don't break Impala. I can annotate the classes as LimitedPrivate("Apache Impala") if that helps.
I understand the need to copy was due to the fact that we are having the configs from BackendConfig. May be I can introduce a build class so that we can pass the values appropriately. Thoughts?
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/src/main/java/org/apache/impala/service/JniFrontend.java
File fe/src/main/java/org/apache/impala/service/JniFrontend.java:
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/src/main/java/org/apache/impala/service/JniFrontend.java@716
PS15, Line 716: frontend_.getSaml2Client().setRedirect(webContext);
I think it would help to improve the readability if we state that the this method adds the http status code and the redirect location in the response object.
http://gerrit.cloudera.org:8080/#/c/16833/15/tests/custom_cluster/test_saml2_sso.py
File tests/custom_cluster/test_saml2_sso.py:
http://gerrit.cloudera.org:8080/#/c/16833/15/tests/custom_cluster/test_saml2_sso.py@70
PS15, Line 70: # SAML2 should not affect non http protocols.
: args = ["--protocol=%s" % protocol, "-q", "select 1 + 2"]
: run_impala_shell_cmd(vector, args, expect_success=True)
: continue
Does this mean that when SAML is configured, users can continue to use binary protocol over LDAP/Kerberos?
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 15
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Fri, 29 Jan 2021 01:32:13 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
Patch Set 21: Verified-1
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/6878/
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 21
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 10 Feb 2021 03:53:16 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
Patch Set 21:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6878/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 21
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Tue, 09 Feb 2021 22:16:12 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Hello Thomas Tauber-Marshall, Vihang Karajgaonkar, Joe McDonnell, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16833
to look at the new patch set (#25).
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
IMPALA-10496: SAML implementation in Impala
The bulk of the SAML2 related code is done on Java side because:
- There is already an implementation for Hive on review (HIVE-24543).
- The only SAML lib for c++ seems to be OpenSaml, which is seemed
quite hard to use and a heavy dependency.
Doing authentication in Java needed some plumbing, as the hs2-http
port is listened to in c++ and http related processing happens in
THttpServer/THttpTransport, which is not a "real" web server, just
a simple http implementation that processes the headers and passes
content to the thrift service.
- Http headers (and in one case body) are inspected and if it is
SAML related, the http request is wrapped in TWrappedHttpRequest
and sent to the Frontend. The Frontend processes it and returns
a TWrappedHttpResponse with the info to return to the client.
- After the last SAML message (with the bearer token) we generate
an auth cookie in c++ (which can be validated in c++), so later
requests in the session don't need to call to Java.
SAML auth can work alongside LDAP and Kerberos - for each hs2-http
request the path and the http headers are inspected to decide
whether it is SAML related, and if not, then we fallback to other
auth mechanisms. This "mixed mode" has no tests yet, so I consider it
experimental.
Planned followup work:
- It would be great to import the logic implemented in Hive instead
of copy-pasting most of it. I plan to do this in a followup commit,
as this needs changes on the Hive side too.
- Adding more tests will be much easier once we will have a hs2-http
client that supports SAML. See IMPALA-10496 for Impyla support.
- Currently the debug webserver does not support SAML auth.
Implementing SAML for the webserver is problematic on the statestore
which doesn't have a Frontend.
Testing:
- Added EE tests that use Python's urllib2 to sent SAML
requests to Impala. Impala works slightly differently
during tests (saml2_ee_test_mode=true).
Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
---
M be/src/rpc/auth-provider.h
M be/src/rpc/authentication-test.cc
M be/src/rpc/authentication.cc
M be/src/rpc/authentication.h
M be/src/rpc/hs2-http-test.cc
M be/src/rpc/thrift-server.h
M be/src/service/frontend.cc
M be/src/service/frontend.h
M be/src/service/impala-server.cc
M be/src/transport/THttpServer.cpp
M be/src/transport/THttpServer.h
M be/src/transport/THttpTransport.cpp
M be/src/transport/THttpTransport.h
M be/src/util/backend-gflag-util.cc
M bin/rat_exclude_files.txt
M common/thrift/BackendGflags.thrift
M common/thrift/Frontend.thrift
M common/thrift/metrics.json
M fe/pom.xml
A fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlHttpServlet.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlUtils.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlAuthenticationException.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlNoGroupsMatchedException.java
A fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
A fe/src/main/java/org/apache/impala/authentication/saml/NullSessionStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/WrappedWebContext.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
M java/pom.xml
A testdata/authentication/saml2_sso.jks
A testdata/authentication/saml2_sso_metadata.xml
A tests/custom_cluster/test_saml2_sso.py
38 files changed, 2,202 insertions(+), 53 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/16833/25
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 25
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
Patch Set 23:
PS 22 is a one line fix for the ldap test failures introduced + unintended removal of the rebase that was done by jenkins.
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 23
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 10 Feb 2021 15:23:50 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: [WiP] SAML implementation in Impala
......................................................................
Patch Set 15:
(11 comments)
Sorry for the many small patchsets, I missed some comments on the first run.
Note that the Hive implementation is still on review (https://github.com/apache/hive/pull/1791 ), so some parts can change, e.g. header/cookie names.
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/common/global-flags.cc
File be/src/common/global-flags.cc:
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/common/global-flags.cc@349
PS11, Line 349: saml2_keystore_password
> Its probably better to make this 'saml2_keystore_password_cmd', similar to
Thanks for spotting this - I postponed changing it for now, as the plan is to make this as similar with Hive as possible, so we should find a similar solution there too.
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/common/global-flags.cc@349
PS11, Line 349: keystore
> typo
Done
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/common/global-flags.cc@361
PS11, Line 361: --hs2_http_por
> Might be worth specifically saying this is the --hs2_http_port flag.
Done
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/common/global-flags.cc@363
PS11, Line 363: DEFINE_bool(saml2_want_assertations_signed, true,
> Is there any situation where it would be reasonable to set this to 'false'
Yes, it should be false only during testing, as the signature proves that the user actually signed in at identity provider.
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/common/global-flags.cc@372
PS11, Line 372: saml2_callback_roken_ttl
> What's the units on this, seconds?
Added seconds to the description.
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/transport/THttpServer.h
File be/src/transport/THttpServer.h:
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/transport/THttpServer.h@151
PS11, Line 151: of
> typo
Done
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/transport/THttpServer.cpp
File be/src/transport/THttpServer.cpp:
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/transport/THttpServer.cpp@132
PS11, Line 132: se if (THRIFT_strncasecmp(header, "Expect", sz) == 0) {
> Any reason to have two copies of this string, rather than just having 'head
I moved setting this to the SAML part of headersDone(). auth_value_ is used by Kerberos/LDAP too, which do not use wrapped_request_, so I think it is simpler to use auth_value_ until the point where we actually need the wrapped_request_.
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/transport/THttpServer.cpp@219
PS11, Line 219: cookies, as t
> As noted elsewhere, I find the name 'readWholeBody_' to be confusing. Maybe
I choose readWholeBodyForAuth_ to make it more informative, but I can switch to other names.
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/transport/THttpServer.cpp@232
PS11, Line 232: (!cookie_v
> This will always be 'false' here
Done
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/transport/THttpTransport.cpp
File be/src/transport/THttpTransport.cpp:
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/transport/THttpTransport.cpp@104
PS11, Line 104: if (readWholeBodyForAuth_) {
> Are we guaranteed that the saml response won't be chunked? I guess presumab
I think that it shouldn't be chunked - the content is a small compressed xml, max a few KB I guess. Unlike other http requests, this comes from the SP provider, which will not send further requests.
Added a check to bodyDone() to throw an exception if chunked_ is true - I think that this is less error prone than adding logic to actually support.
http://gerrit.cloudera.org:8080/#/c/16833/11/common/thrift/Frontend.thrift
File common/thrift/Frontend.thrift:
http://gerrit.cloudera.org:8080/#/c/16833/11/common/thrift/Frontend.thrift@988
PS11, Line 988: // Contains all information from a HTTP request.
> Needs some comments, at least for the overall struct, not necessarily every
TSamlAuthnResponse was actually no longer used. Added comments for TWrappedHttpRequest and TWrappedHttpResponse.
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 15
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Tue, 19 Jan 2021 21:26:25 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: [WiP] SAML implementation in Impala
......................................................................
Patch Set 12:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/8022/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 12
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Tue, 19 Jan 2021 21:14:09 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: [WiP] SAML implementation in Impala
......................................................................
Patch Set 11:
(27 comments)
http://gerrit.cloudera.org:8080/#/c/16833/11/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/11/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java@20
PS11, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/AuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/11/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/11/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@32
PS11, Line 32: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlAuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/11/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@52
PS11, Line 52: private static final Logger LOG = LoggerFactory.getLogger(HiveSamlAuthTokenGenerator.class);
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/11/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java:
http://gerrit.cloudera.org:8080/#/c/16833/11/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java@31
PS11, Line 31: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlGroupNameFilter.java
line too long (132 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/11/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java:
http://gerrit.cloudera.org:8080/#/c/16833/11/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java@20
PS11, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (139 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/11/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java:
http://gerrit.cloudera.org:8080/#/c/16833/11/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java@32
PS11, Line 32: // slightly modified copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/11/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java@32
PS11, Line 32: // slightly modified copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (158 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/11/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
File fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java:
http://gerrit.cloudera.org:8080/#/c/16833/11/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@42
PS11, Line 42: // modified version of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSaml2Client.java
line too long (144 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/11/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@65
PS11, Line 65: //TODO handle the replayCache as described in http://www.pac4j.org/docs/clients/saml.html
line too long (93 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/11/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@146
PS11, Line 146: // This is done to keep original structure by Vihang + keep ImpalaSamlClient as the only
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/11/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@191
PS11, Line 191: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpServlet.java
line too long (129 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/11/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@192
PS11, Line 192: private String doSamlAuth(WrappedWebContext webContext) throws HttpSamlAuthenticationException {
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/11/fe/src/main/java/org/apache/impala/service/BackendConfig.java
File fe/src/main/java/org/apache/impala/service/BackendConfig.java:
http://gerrit.cloudera.org:8080/#/c/16833/11/fe/src/main/java/org/apache/impala/service/BackendConfig.java@237
PS11, Line 237: public String getSaml2GroupAttibuteName() { return backendCfg_.saml2_group_attribute_name; }
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/11/fe/src/main/java/org/apache/impala/service/JniFrontend.java
File fe/src/main/java/org/apache/impala/service/JniFrontend.java:
http://gerrit.cloudera.org:8080/#/c/16833/11/fe/src/main/java/org/apache/impala/service/JniFrontend.java@709
PS11, Line 709:
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/11/tests/custom_cluster/test_saml2_sso.py
File tests/custom_cluster/test_saml2_sso.py:
http://gerrit.cloudera.org:8080/#/c/16833/11/tests/custom_cluster/test_saml2_sso.py@20
PS11, Line 20: import logging
flake8: F401 'logging' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/11/tests/custom_cluster/test_saml2_sso.py@28
PS11, Line 28: from tests.common.impala_service import ImpaladService
flake8: F401 'tests.common.impala_service.ImpaladService' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/11/tests/custom_cluster/test_saml2_sso.py@31
PS11, Line 31: from tests.shell.util import run_impala_shell_cmd, run_impala_shell_cmd_no_expect, \
flake8: F401 'tests.shell.util.run_impala_shell_cmd_no_expect' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/11/tests/custom_cluster/test_saml2_sso.py@31
PS11, Line 31: from tests.shell.util import run_impala_shell_cmd, run_impala_shell_cmd_no_expect, \
flake8: F401 'tests.shell.util.ImpalaShell' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/11/tests/custom_cluster/test_saml2_sso.py@34
PS11, Line 34: class NoRedirection(urllib2.HTTPErrorProcessor):
flake8: E302 expected 2 blank lines, found 1
http://gerrit.cloudera.org:8080/#/c/16833/11/tests/custom_cluster/test_saml2_sso.py@41
PS11, Line 41: class TestClientSsl(CustomClusterTestSuite):
flake8: E303 too many blank lines (3)
http://gerrit.cloudera.org:8080/#/c/16833/11/tests/custom_cluster/test_saml2_sso.py@46
PS11, Line 46:
flake8: W291 trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/11/tests/custom_cluster/test_saml2_sso.py@46
PS11, Line 46: SP_URL = "https://localhost:8443/simplesaml/saml2/idp/SSOService.php"
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/11/tests/custom_cluster/test_saml2_sso.py@82
PS11, Line 82: r
flake8: F841 local variable 'relay_state' is assigned to but never used
http://gerrit.cloudera.org:8080/#/c/16833/11/tests/custom_cluster/test_saml2_sso.py@97
PS11, Line 97: d
flake8: F841 local variable 'decoded_saml_req' is assigned to but never used
http://gerrit.cloudera.org:8080/#/c/16833/11/tests/custom_cluster/test_saml2_sso.py@103
PS11, Line 103:
flake8: W293 blank line contains whitespace
http://gerrit.cloudera.org:8080/#/c/16833/11/tests/custom_cluster/test_saml2_sso.py@103
PS11, Line 103:
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/11/tests/custom_cluster/test_saml2_sso.py@104
PS11, Line 104:
flake8: W391 blank line at end of file
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 11
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Jan 2021 23:31:23 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: [WiP] SAML implementation in Impala
......................................................................
Patch Set 13:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/8023/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 13
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Tue, 19 Jan 2021 21:15:01 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Vihang Karajgaonkar (Code Review)" <ge...@cloudera.org>.
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
Patch Set 24: Code-Review+1
(1 comment)
Patch looks good to me. Thanks for making the suggested changes.
http://gerrit.cloudera.org:8080/#/c/16833/24/tests/custom_cluster/test_saml2_sso.py
File tests/custom_cluster/test_saml2_sso.py:
http://gerrit.cloudera.org:8080/#/c/16833/24/tests/custom_cluster/test_saml2_sso.py@48
PS24, Line 48: Most
Thanks for adding a detailed doc. Looks great!
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 24
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 Feb 2021 01:15:44 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
Patch Set 20:
(37 comments)
http://gerrit.cloudera.org:8080/#/c/16833/20/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/20/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java@20
PS20, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/AuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/20/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/20/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@32
PS20, Line 32: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlAuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/20/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@52
PS20, Line 52: private static final Logger LOG = LoggerFactory.getLogger(HiveSamlAuthTokenGenerator.class);
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/20/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java:
http://gerrit.cloudera.org:8080/#/c/16833/20/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java@31
PS20, Line 31: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlGroupNameFilter.java
line too long (132 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/20/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java:
http://gerrit.cloudera.org:8080/#/c/16833/20/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java@20
PS20, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (139 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/20/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java:
http://gerrit.cloudera.org:8080/#/c/16833/20/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java@32
PS20, Line 32: // slightly modified copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (157 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/20/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
File fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java:
http://gerrit.cloudera.org:8080/#/c/16833/20/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@43
PS20, Line 43: // modified version of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSaml2Client.java
line too long (144 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/20/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@69
PS20, Line 69: //TODO handle the replayCache as described in http://www.pac4j.org/docs/clients/saml.html
line too long (93 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/20/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@148
PS20, Line 148: // This is done to keep original structure by Vihang + keep ImpalaSamlClient as the only
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/20/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@188
PS20, Line 188: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpServlet.java
line too long (129 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/20/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@189
PS20, Line 189: private String doSamlAuth(WrappedWebContext webContext) throws HttpSamlAuthenticationException {
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/20/fe/src/main/java/org/apache/impala/service/BackendConfig.java
File fe/src/main/java/org/apache/impala/service/BackendConfig.java:
http://gerrit.cloudera.org:8080/#/c/16833/20/fe/src/main/java/org/apache/impala/service/BackendConfig.java@237
PS20, Line 237: public String getSaml2GroupAttibuteName() { return backendCfg_.saml2_group_attribute_name; }
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/20/tests/custom_cluster/test_saml2_sso.py
File tests/custom_cluster/test_saml2_sso.py:
http://gerrit.cloudera.org:8080/#/c/16833/20/tests/custom_cluster/test_saml2_sso.py@21
PS20, Line 21: import logging
flake8: F401 'logging' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/20/tests/custom_cluster/test_saml2_sso.py@31
PS20, Line 31: from tests.common.impala_service import ImpaladService
flake8: F401 'tests.common.impala_service.ImpaladService' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/20/tests/custom_cluster/test_saml2_sso.py@34
PS20, Line 34: from tests.shell.util import run_impala_shell_cmd, run_impala_shell_cmd_no_expect, \
flake8: F401 'tests.shell.util.run_impala_shell_cmd_no_expect' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/20/tests/custom_cluster/test_saml2_sso.py@34
PS20, Line 34: from tests.shell.util import run_impala_shell_cmd, run_impala_shell_cmd_no_expect, \
flake8: F401 'tests.shell.util.ImpalaShell' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/20/tests/custom_cluster/test_saml2_sso.py@37
PS20, Line 37: class NoRedirection(urllib2.HTTPErrorProcessor):
flake8: E302 expected 2 blank lines, found 1
http://gerrit.cloudera.org:8080/#/c/16833/20/tests/custom_cluster/test_saml2_sso.py@47
PS20, Line 47: class TestClientSaml(CustomClusterTestSuite):
flake8: E302 expected 2 blank lines, found 1
http://gerrit.cloudera.org:8080/#/c/16833/20/tests/custom_cluster/test_saml2_sso.py@71
PS20, Line 71: \
flake8: E502 the backslash is redundant between brackets
http://gerrit.cloudera.org:8080/#/c/16833/20/tests/custom_cluster/test_saml2_sso.py@188
PS20, Line 188:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/20/tests/custom_cluster/test_saml2_sso.py@188
PS20, Line 188:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/20/tests/custom_cluster/test_saml2_sso.py@189
PS20, Line 189:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/20/tests/custom_cluster/test_saml2_sso.py@189
PS20, Line 189:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/20/tests/custom_cluster/test_saml2_sso.py@190
PS20, Line 190:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/20/tests/custom_cluster/test_saml2_sso.py@190
PS20, Line 190:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/20/tests/custom_cluster/test_saml2_sso.py@191
PS20, Line 191:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/20/tests/custom_cluster/test_saml2_sso.py@191
PS20, Line 191:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/20/tests/custom_cluster/test_saml2_sso.py@192
PS20, Line 192:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/20/tests/custom_cluster/test_saml2_sso.py@192
PS20, Line 192:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/20/tests/custom_cluster/test_saml2_sso.py@193
PS20, Line 193:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/20/tests/custom_cluster/test_saml2_sso.py@193
PS20, Line 193:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/20/tests/custom_cluster/test_saml2_sso.py@194
PS20, Line 194:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/20/tests/custom_cluster/test_saml2_sso.py@194
PS20, Line 194:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/20/tests/custom_cluster/test_saml2_sso.py@274
PS20, Line 274: #
flake8: E116 unexpected indentation (comment)
http://gerrit.cloudera.org:8080/#/c/16833/20/tests/custom_cluster/test_saml2_sso.py@277
PS20, Line 277: a
flake8: E501 line too long (96 > 90 characters)
http://gerrit.cloudera.org:8080/#/c/16833/20/tests/custom_cluster/test_saml2_sso.py@280
PS20, Line 280: a
flake8: E501 line too long (113 > 90 characters)
http://gerrit.cloudera.org:8080/#/c/16833/20/tests/custom_cluster/test_saml2_sso.py@283
PS20, Line 283: :
flake8: E501 line too long (98 > 90 characters)
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 20
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Tue, 09 Feb 2021 22:11:28 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
Patch Set 24:
(16 comments)
http://gerrit.cloudera.org:8080/#/c/16833/24/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/24/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java@20
PS24, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/AuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/24/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/24/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@32
PS24, Line 32: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlAuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/24/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@52
PS24, Line 52: private static final Logger LOG = LoggerFactory.getLogger(HiveSamlAuthTokenGenerator.class);
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/24/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java:
http://gerrit.cloudera.org:8080/#/c/16833/24/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java@31
PS24, Line 31: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlGroupNameFilter.java
line too long (132 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/24/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java:
http://gerrit.cloudera.org:8080/#/c/16833/24/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java@20
PS24, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (139 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/24/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java:
http://gerrit.cloudera.org:8080/#/c/16833/24/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java@32
PS24, Line 32: // slightly modified copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (157 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/24/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
File fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java:
http://gerrit.cloudera.org:8080/#/c/16833/24/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@43
PS24, Line 43: // modified version of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSaml2Client.java
line too long (144 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/24/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@69
PS24, Line 69: //TODO handle the replayCache as described in http://www.pac4j.org/docs/clients/saml.html
line too long (93 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/24/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@147
PS24, Line 147: // This is done to keep original structure by Vihang + keep ImpalaSamlClient as the only
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/24/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@187
PS24, Line 187: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpServlet.java
line too long (129 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/24/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@188
PS24, Line 188: private String doSamlAuth(WrappedWebContext webContext) throws HttpSamlAuthenticationException {
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/24/fe/src/main/java/org/apache/impala/service/BackendConfig.java
File fe/src/main/java/org/apache/impala/service/BackendConfig.java:
http://gerrit.cloudera.org:8080/#/c/16833/24/fe/src/main/java/org/apache/impala/service/BackendConfig.java@237
PS24, Line 237: public String getSaml2GroupAttibuteName() { return backendCfg_.saml2_group_attribute_name; }
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/24/tests/custom_cluster/test_saml2_sso.py
File tests/custom_cluster/test_saml2_sso.py:
http://gerrit.cloudera.org:8080/#/c/16833/24/tests/custom_cluster/test_saml2_sso.py@34
PS24, Line 34: class NoRedirection(urllib2.HTTPErrorProcessor):
flake8: E302 expected 2 blank lines, found 1
http://gerrit.cloudera.org:8080/#/c/16833/24/tests/custom_cluster/test_saml2_sso.py@303
PS24, Line 303: a
flake8: E501 line too long (96 > 90 characters)
http://gerrit.cloudera.org:8080/#/c/16833/24/tests/custom_cluster/test_saml2_sso.py@306
PS24, Line 306: a
flake8: E501 line too long (113 > 90 characters)
http://gerrit.cloudera.org:8080/#/c/16833/24/tests/custom_cluster/test_saml2_sso.py@309
PS24, Line 309: :
flake8: E501 line too long (98 > 90 characters)
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 24
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 Feb 2021 00:19:24 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: [WiP] SAML implementation in Impala
......................................................................
Patch Set 12:
(27 comments)
http://gerrit.cloudera.org:8080/#/c/16833/12/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/12/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java@20
PS12, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/AuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/12/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/12/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@32
PS12, Line 32: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlAuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/12/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@52
PS12, Line 52: private static final Logger LOG = LoggerFactory.getLogger(HiveSamlAuthTokenGenerator.class);
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/12/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java:
http://gerrit.cloudera.org:8080/#/c/16833/12/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java@31
PS12, Line 31: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlGroupNameFilter.java
line too long (132 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/12/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java:
http://gerrit.cloudera.org:8080/#/c/16833/12/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java@20
PS12, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (139 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/12/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java:
http://gerrit.cloudera.org:8080/#/c/16833/12/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java@32
PS12, Line 32: // slightly modified copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/12/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java@32
PS12, Line 32: // slightly modified copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (158 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/12/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
File fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java:
http://gerrit.cloudera.org:8080/#/c/16833/12/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@42
PS12, Line 42: // modified version of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSaml2Client.java
line too long (144 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/12/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@65
PS12, Line 65: //TODO handle the replayCache as described in http://www.pac4j.org/docs/clients/saml.html
line too long (93 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/12/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@146
PS12, Line 146: // This is done to keep original structure by Vihang + keep ImpalaSamlClient as the only
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/12/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@191
PS12, Line 191: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpServlet.java
line too long (129 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/12/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@192
PS12, Line 192: private String doSamlAuth(WrappedWebContext webContext) throws HttpSamlAuthenticationException {
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/12/fe/src/main/java/org/apache/impala/service/BackendConfig.java
File fe/src/main/java/org/apache/impala/service/BackendConfig.java:
http://gerrit.cloudera.org:8080/#/c/16833/12/fe/src/main/java/org/apache/impala/service/BackendConfig.java@237
PS12, Line 237: public String getSaml2GroupAttibuteName() { return backendCfg_.saml2_group_attribute_name; }
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/12/fe/src/main/java/org/apache/impala/service/JniFrontend.java
File fe/src/main/java/org/apache/impala/service/JniFrontend.java:
http://gerrit.cloudera.org:8080/#/c/16833/12/fe/src/main/java/org/apache/impala/service/JniFrontend.java@709
PS12, Line 709:
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/12/tests/custom_cluster/test_saml2_sso.py
File tests/custom_cluster/test_saml2_sso.py:
http://gerrit.cloudera.org:8080/#/c/16833/12/tests/custom_cluster/test_saml2_sso.py@20
PS12, Line 20: import logging
flake8: F401 'logging' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/12/tests/custom_cluster/test_saml2_sso.py@28
PS12, Line 28: from tests.common.impala_service import ImpaladService
flake8: F401 'tests.common.impala_service.ImpaladService' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/12/tests/custom_cluster/test_saml2_sso.py@31
PS12, Line 31: from tests.shell.util import run_impala_shell_cmd, run_impala_shell_cmd_no_expect, \
flake8: F401 'tests.shell.util.run_impala_shell_cmd_no_expect' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/12/tests/custom_cluster/test_saml2_sso.py@31
PS12, Line 31: from tests.shell.util import run_impala_shell_cmd, run_impala_shell_cmd_no_expect, \
flake8: F401 'tests.shell.util.ImpalaShell' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/12/tests/custom_cluster/test_saml2_sso.py@34
PS12, Line 34: class NoRedirection(urllib2.HTTPErrorProcessor):
flake8: E302 expected 2 blank lines, found 1
http://gerrit.cloudera.org:8080/#/c/16833/12/tests/custom_cluster/test_saml2_sso.py@41
PS12, Line 41: class TestClientSsl(CustomClusterTestSuite):
flake8: E303 too many blank lines (3)
http://gerrit.cloudera.org:8080/#/c/16833/12/tests/custom_cluster/test_saml2_sso.py@46
PS12, Line 46:
flake8: W291 trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/12/tests/custom_cluster/test_saml2_sso.py@46
PS12, Line 46: SP_URL = "https://localhost:8443/simplesaml/saml2/idp/SSOService.php"
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/12/tests/custom_cluster/test_saml2_sso.py@82
PS12, Line 82: r
flake8: F841 local variable 'relay_state' is assigned to but never used
http://gerrit.cloudera.org:8080/#/c/16833/12/tests/custom_cluster/test_saml2_sso.py@97
PS12, Line 97: d
flake8: F841 local variable 'decoded_saml_req' is assigned to but never used
http://gerrit.cloudera.org:8080/#/c/16833/12/tests/custom_cluster/test_saml2_sso.py@103
PS12, Line 103:
flake8: W293 blank line contains whitespace
http://gerrit.cloudera.org:8080/#/c/16833/12/tests/custom_cluster/test_saml2_sso.py@103
PS12, Line 103:
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/12/tests/custom_cluster/test_saml2_sso.py@104
PS12, Line 104:
flake8: W391 blank line at end of file
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 12
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Tue, 19 Jan 2021 20:52:45 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
Patch Set 22:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/8117/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 22
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 10 Feb 2021 15:36:04 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
Patch Set 26:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6894/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 26
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 Feb 2021 17:03:55 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
Patch Set 23: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 23
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 10 Feb 2021 21:06:35 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
Patch Set 24:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/8143/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 24
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 Feb 2021 00:40:21 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has uploaded a new patch set (#20). ( http://gerrit.cloudera.org:8080/16833 )
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
IMPALA-10496: SAML implementation in Impala
The bulk of the SAML2 related code is done on Java side because:
- There is already a POC in Hive that could be reused.
- The only SAML lib for c++ seems to be OpenSaml, which is seemed
quite hard to use and a heavy dependency.
Doing authentication in Java needed some plumbing, as the hs2-http
port is listened to in c++ and http related processing happens in
THttpServer/THttpTransport, which is not a "real" web server, just
a simple http implementation that processes the headers and passes
content to the thrift service.
- Http headers (and in one case body) are inspected and if it is
SAML related, the http request is wrapped in TWrappedHttpRequest
and sent to the Frontend. The Frontend processes it and returns
a TWrappedHttpResponse with the info to return to the client.
- After the last SAML message (with the bearer token) we generate
an auth cookie in c++ (which can be validated in c++), so later
requests in the session don't need to call to Java.
State of implementation:
- The java side is more or less ok, will be updated when the Hive
implementation changes. I would do a proper cleanup / documentation
once the Hive code is more final.
- Compatibility with other auth mechanisms should be decided:
- Whether other clients should be able to auth with ldap/kerberos
is not clear yet.
Testing:
- Added EE tests that use Python's urllib2 to sent SAML
requests to Impala. Impala works slightly differently
during tests (saml2_ee_test_mode=true).
Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
---
M be/src/common/global-flags.cc
M be/src/rpc/auth-provider.h
M be/src/rpc/authentication-test.cc
M be/src/rpc/authentication.cc
M be/src/rpc/authentication.h
M be/src/rpc/hs2-http-test.cc
M be/src/rpc/thrift-server.h
M be/src/service/frontend.cc
M be/src/service/frontend.h
M be/src/service/impala-server.cc
M be/src/transport/THttpServer.cpp
M be/src/transport/THttpServer.h
M be/src/transport/THttpTransport.cpp
M be/src/transport/THttpTransport.h
M be/src/util/backend-gflag-util.cc
M bin/rat_exclude_files.txt
M common/thrift/BackendGflags.thrift
M common/thrift/Frontend.thrift
M common/thrift/metrics.json
M fe/pom.xml
A fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlHttpServlet.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlUtils.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlAuthenticationException.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlNoGroupsMatchedException.java
A fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
A fe/src/main/java/org/apache/impala/authentication/saml/NullSessionStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/WrappedWebContext.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
M java/pom.xml
A testdata/authentication/saml2_sso.jks
A testdata/authentication/saml2_sso_metadata.xml
A tests/custom_cluster/test_saml2_sso.py
39 files changed, 2,142 insertions(+), 50 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/16833/20
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 20
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: [WiP] SAML implementation in Impala
......................................................................
Patch Set 11:
Patch 11 is mainly a rebase + a few style and rat fix mixed in by mistake.
I do not plan to fix the Java style issues while the Hive solution is still on review (https://github.com/apache/hive/pull/1791)
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 11
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Jan 2021 23:35:49 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Hello Thomas Tauber-Marshall, Vihang Karajgaonkar, Joe McDonnell, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16833
to look at the new patch set (#10).
Change subject: [WiP] SAML implementation in Impala
......................................................................
[WiP] SAML implementation in Impala
The bulk of the SAML2 related code is done on Java side because:
- There is already a POC in Hive that could be reused.
- The only SAML lib for c++ seems to be OpenSaml, which is seemed
quite hard to use and a heavy dependency.
Doing authentication in Java needed some plumbing, as the hs2-http
port is listened to in c++ and http related processing happens in
THttpServer/THttpTransport, which is not a "real" web server, just
a simple http implementation that processes the headers and passes
content to the thrift service.
- Http headers (and in one case body) are inspected and if it is
SAML related, the http request is wrapped in TWrappedHttpRequest
and sent to the Frontend. The Frontend processes it and returns
a TWrappedHttpResponse with the info to return to the client.
- After the last SAML message (with the bearer token) we generate
an auth cookie in c++ (which can be validated in c++), so later
requests in the session don't need to call to Java.
State of implementation:
- The java side is more or less ok, will be updated when the Hive
implementation changes. I would do a proper cleanup / documentation
once the Hive code is more final.
- The c++ side probsbluneeds a lot of cleanup.
- Compatibility with other auth mechanisms should be decided:
- Whether other clients should be able to auth with ldap/kerberos
is not clear yet.
- Only the first step of the auth workflow have automatic tests.
Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
---
M be/src/common/global-flags.cc
M be/src/rpc/auth-provider.h
M be/src/rpc/authentication-test.cc
M be/src/rpc/authentication.cc
M be/src/rpc/authentication.h
M be/src/rpc/hs2-http-test.cc
M be/src/rpc/thrift-server.h
M be/src/service/frontend.cc
M be/src/service/frontend.h
M be/src/service/impala-server.cc
M be/src/transport/THttpServer.cpp
M be/src/transport/THttpServer.h
M be/src/transport/THttpTransport.cpp
M be/src/transport/THttpTransport.h
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
M common/thrift/Frontend.thrift
M common/thrift/metrics.json
M fe/pom.xml
A fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlHttpServlet.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlUtils.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlAuthenticationException.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlNoGroupsMatchedException.java
A fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
A fe/src/main/java/org/apache/impala/authentication/saml/NullSessionStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/WrappedWebContext.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
M java/pom.xml
A testdata/authentication/saml2_sso.jks
A testdata/authentication/saml2_sso_metadata.xml
A tests/custom_cluster/test_saml2_sso.py
38 files changed, 1,881 insertions(+), 51 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/16833/10
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 10
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
Patch Set 25:
(11 comments)
http://gerrit.cloudera.org:8080/#/c/16833/24/be/src/rpc/authentication.h
File be/src/rpc/authentication.h:
http://gerrit.cloudera.org:8080/#/c/16833/24/be/src/rpc/authentication.h@58
PS24, Line 58: /// Returns the authentication provider to use for "external" communication with
> brief comment
Done
http://gerrit.cloudera.org:8080/#/c/16833/24/be/src/rpc/authentication.cc
File be/src/rpc/authentication.cc:
http://gerrit.cloudera.org:8080/#/c/16833/24/be/src/rpc/authentication.cc@125
PS24, Line 125:
> nit: allows
Done
http://gerrit.cloudera.org:8080/#/c/16833/24/be/src/rpc/authentication.cc@573
PS24, Line 573: }
> Brief comment about the parameters here.
Done
http://gerrit.cloudera.org:8080/#/c/16833/24/be/src/rpc/authentication.cc@629
PS24, Line 629: est
> nit: we prefer nullptr, here and elsewhere
Done
http://gerrit.cloudera.org:8080/#/c/16833/24/be/src/rpc/authentication.cc@683
PS24, Line 683:
> probably worth explicitly saying what still needs to be done here
I added some comments that explain why did I do this.
http://gerrit.cloudera.org:8080/#/c/16833/24/be/src/transport/THttpServer.h
File be/src/transport/THttpServer.h:
http://gerrit.cloudera.org:8080/#/c/16833/24/be/src/transport/THttpServer.h@85
PS24, Line 85: purposes
> typo: purposes
Done
http://gerrit.cloudera.org:8080/#/c/16833/24/be/src/transport/THttpServer.cpp
File be/src/transport/THttpServer.cpp:
http://gerrit.cloudera.org:8080/#/c/16833/24/be/src/transport/THttpServer.cpp@228
PS24, Line 228: // Try authenticating with cookies first.
> nit: unnecessary whitespace
Done
http://gerrit.cloudera.org:8080/#/c/16833/24/be/src/transport/THttpServer.cpp@252
PS24, Line 252: throw TTransportException("HTTP auth - SAML redirection.");
: }
: } else if (!auth_value_
> might be nice to put this logic in some sort of resetConnection() helper fu
Done
http://gerrit.cloudera.org:8080/#/c/16833/24/common/thrift/metrics.json
File common/thrift/metrics.json:
http://gerrit.cloudera.org:8080/#/c/16833/24/common/thrift/metrics.json@1315
PS24, Line 1315: emon that
> This maybe gets a little confusing, since usually when we talk about the "w
Done
http://gerrit.cloudera.org:8080/#/c/16833/24/common/thrift/metrics.json@1329
PS24, Line 1329: I
> typo: space
Done
http://gerrit.cloudera.org:8080/#/c/16833/23/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
File fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java:
http://gerrit.cloudera.org:8080/#/c/16833/23/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@76
PS23, Line 76:
> This path is already parsed in the backend, so I think that it is easier to
Added validation to authentication.cc . This also revealed a test issue that the port is only correct on the first Impalad. For now the fix was to change the tests to use a 1 sized cluster.
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 25
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 Feb 2021 17:03:12 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: [WiP] SAML implementation in Impala
......................................................................
Patch Set 9:
(55 comments)
http://gerrit.cloudera.org:8080/#/c/16833/9/be/src/rpc/authentication.cc
File be/src/rpc/authentication.cc:
http://gerrit.cloudera.org:8080/#/c/16833/9/be/src/rpc/authentication.cc@1160
PS9, Line 1160: callbacks.get_saml_redirect_fn = std::bind(GetSaml2Redirect, connection_ptr.get());
line too long (91 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/9/be/src/transport/THttpServer.h
File be/src/transport/THttpServer.h:
http://gerrit.cloudera.org:8080/#/c/16833/9/be/src/transport/THttpServer.h@160
PS9, Line 160: // 2. validating an authNRespone from the SSO provider
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/9/be/src/transport/THttpServer.h@192
PS9, Line 192: // Used to collect all information about the http request. Can be passed to the Frontend.
line too long (91 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/9/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/9/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java@20
PS9, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/AuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/9/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/9/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@32
PS9, Line 32: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlAuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/9/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@52
PS9, Line 52: private static final Logger LOG = LoggerFactory.getLogger(HiveSamlAuthTokenGenerator.class);
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/9/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java:
http://gerrit.cloudera.org:8080/#/c/16833/9/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java@31
PS9, Line 31: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlGroupNameFilter.java
line too long (132 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/9/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java:
http://gerrit.cloudera.org:8080/#/c/16833/9/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java@20
PS9, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (139 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/9/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java:
http://gerrit.cloudera.org:8080/#/c/16833/9/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java@32
PS9, Line 32: // slightly modified copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/9/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java@32
PS9, Line 32: // slightly modified copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (158 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/9/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
File fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java:
http://gerrit.cloudera.org:8080/#/c/16833/9/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@42
PS9, Line 42: // modified version of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSaml2Client.java
line too long (144 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/9/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@65
PS9, Line 65: //TODO handle the replayCache as described in http://www.pac4j.org/docs/clients/saml.html
line too long (93 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/9/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@146
PS9, Line 146: // This is done to keep original structure by Vihang + keep ImpalaSamlClient as the only
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/9/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@191
PS9, Line 191: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpServlet.java
line too long (129 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/9/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@192
PS9, Line 192: private String doSamlAuth(WrappedWebContext webContext) throws HttpSamlAuthenticationException {
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/9/fe/src/main/java/org/apache/impala/service/BackendConfig.java
File fe/src/main/java/org/apache/impala/service/BackendConfig.java:
http://gerrit.cloudera.org:8080/#/c/16833/9/fe/src/main/java/org/apache/impala/service/BackendConfig.java@245
PS9, Line 245: public String getSaml2GroupAttibuteName() { return backendCfg_.saml2_group_attribute_name; }
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/9/fe/src/main/java/org/apache/impala/service/JniFrontend.java
File fe/src/main/java/org/apache/impala/service/JniFrontend.java:
http://gerrit.cloudera.org:8080/#/c/16833/9/fe/src/main/java/org/apache/impala/service/JniFrontend.java@688
PS9, Line 688:
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/9/test_saml.py
File test_saml.py:
http://gerrit.cloudera.org:8080/#/c/16833/9/test_saml.py@6
PS9, Line 6: class NoRedirection(urllib2.HTTPErrorProcessor):
flake8: E302 expected 2 blank lines, found 1
http://gerrit.cloudera.org:8080/#/c/16833/9/test_saml.py@11
PS9, Line 11: opener = urllib2.build_opener(NoRedirection)
flake8: E305 expected 2 blank lines after class or function definition, found 1
http://gerrit.cloudera.org:8080/#/c/16833/9/test_saml.py@15
PS9, Line 15: #print result.geturl()
flake8: E265 block comment should start with '# '
http://gerrit.cloudera.org:8080/#/c/16833/9/test_saml.py@16
PS9, Line 16: #print result.info()
flake8: E265 block comment should start with '# '
http://gerrit.cloudera.org:8080/#/c/16833/9/test_saml.py@21
PS9, Line 21: #print query
flake8: E265 block comment should start with '# '
http://gerrit.cloudera.org:8080/#/c/16833/9/test_saml.py@22
PS9, Line 22: =
flake8: E225 missing whitespace around operator
http://gerrit.cloudera.org:8080/#/c/16833/9/test_saml.py@23
PS9, Line 23: #print saml_req
flake8: E265 block comment should start with '# '
http://gerrit.cloudera.org:8080/#/c/16833/9/test_saml.py@25
PS9, Line 25: #saml_req = urlparse.unquote(saml_req)
flake8: E265 block comment should start with '# '
http://gerrit.cloudera.org:8080/#/c/16833/9/test_saml.py@26
PS9, Line 26: #print "unquoted " + saml_req
flake8: E265 block comment should start with '# '
http://gerrit.cloudera.org:8080/#/c/16833/9/test_saml.py@80
PS9, Line 80: =
flake8: E225 missing whitespace around operator
http://gerrit.cloudera.org:8080/#/c/16833/9/test_saml.py@82
PS9, Line 82: #req = urllib2.Request("http://localhost:28000/SAML2/SSO/POST", "SAMLResponse=" + samlResponse + "&RelayState=1234")
flake8: E265 block comment should start with '# '
http://gerrit.cloudera.org:8080/#/c/16833/9/test_saml.py@82
PS9, Line 82: o
flake8: E501 line too long (116 > 90 characters)
http://gerrit.cloudera.org:8080/#/c/16833/9/test_saml.py@83
PS9, Line 83: #result = opener.open(req)
flake8: E265 block comment should start with '# '
http://gerrit.cloudera.org:8080/#/c/16833/9/test_saml.py@84
PS9, Line 84: #print result.headers
flake8: E265 block comment should start with '# '
http://gerrit.cloudera.org:8080/#/c/16833/9/test_saml.py@85
PS9, Line 85: #new_url = result.info().getheader("location")
flake8: E265 block comment should start with '# '
http://gerrit.cloudera.org:8080/#/c/16833/9/test_saml.py@86
PS9, Line 86: #print new_url
flake8: E265 block comment should start with '# '
http://gerrit.cloudera.org:8080/#/c/16833/9/test_saml.py@87
PS9, Line 87: #bearer_token = result.info().getheader("WWW-Authenticate").split(" ")[1];
flake8: E265 block comment should start with '# '
http://gerrit.cloudera.org:8080/#/c/16833/9/test_saml.py@88
PS9, Line 88: #print bearer_token
flake8: E265 block comment should start with '# '
http://gerrit.cloudera.org:8080/#/c/16833/9/test_saml.py@90
PS9, Line 90: #bearer_token="u=_02c81a85938a2639ed9b7fb9bdae7b697e76b80bdc;id=7016226164652493581;time=1608051695185;rs=e9d08eac-458b-44e3-afbd-a3c3870d7c20;sg=eFOkCkMXTRoj7QrmaLFeZP7pGtxmcj2+5izt7qWwnz8="
flake8: E265 block comment should start with '# '
http://gerrit.cloudera.org:8080/#/c/16833/9/test_saml.py@90
PS9, Line 90: 6
flake8: E501 line too long (191 > 90 characters)
http://gerrit.cloudera.org:8080/#/c/16833/9/test_saml.py@91
PS9, Line 91: #req = urllib2.Request("http://localhost:28000", " ")
flake8: E265 block comment should start with '# '
http://gerrit.cloudera.org:8080/#/c/16833/9/test_saml.py@92
PS9, Line 92: #req.add_header('Authorization', 'Bearer ' + bearer_token)
flake8: E265 block comment should start with '# '
http://gerrit.cloudera.org:8080/#/c/16833/9/test_saml.py@93
PS9, Line 93: #result = opener.open(req)
flake8: E265 block comment should start with '# '
http://gerrit.cloudera.org:8080/#/c/16833/9/test_saml.py@94
PS9, Line 94: #print result.info()
flake8: E265 block comment should start with '# '
http://gerrit.cloudera.org:8080/#/c/16833/9/test_saml.py@95
PS9, Line 95: #print result.getcode()
flake8: E265 block comment should start with '# '
http://gerrit.cloudera.org:8080/#/c/16833/9/tests/custom_cluster/test_saml2_sso.py
File tests/custom_cluster/test_saml2_sso.py:
http://gerrit.cloudera.org:8080/#/c/16833/9/tests/custom_cluster/test_saml2_sso.py@20
PS9, Line 20: import logging
flake8: F401 'logging' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/9/tests/custom_cluster/test_saml2_sso.py@28
PS9, Line 28: from tests.common.impala_service import ImpaladService
flake8: F401 'tests.common.impala_service.ImpaladService' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/9/tests/custom_cluster/test_saml2_sso.py@31
PS9, Line 31: from tests.shell.util import run_impala_shell_cmd, run_impala_shell_cmd_no_expect, \
flake8: F401 'tests.shell.util.run_impala_shell_cmd_no_expect' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/9/tests/custom_cluster/test_saml2_sso.py@31
PS9, Line 31: from tests.shell.util import run_impala_shell_cmd, run_impala_shell_cmd_no_expect, \
flake8: F401 'tests.shell.util.ImpalaShell' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/9/tests/custom_cluster/test_saml2_sso.py@34
PS9, Line 34: class NoRedirection(urllib2.HTTPErrorProcessor):
flake8: E302 expected 2 blank lines, found 1
http://gerrit.cloudera.org:8080/#/c/16833/9/tests/custom_cluster/test_saml2_sso.py@41
PS9, Line 41: class TestClientSsl(CustomClusterTestSuite):
flake8: E303 too many blank lines (3)
http://gerrit.cloudera.org:8080/#/c/16833/9/tests/custom_cluster/test_saml2_sso.py@46
PS9, Line 46:
flake8: W291 trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/9/tests/custom_cluster/test_saml2_sso.py@46
PS9, Line 46: SP_URL = "https://localhost:8443/simplesaml/saml2/idp/SSOService.php"
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/9/tests/custom_cluster/test_saml2_sso.py@82
PS9, Line 82: r
flake8: F841 local variable 'relay_state' is assigned to but never used
http://gerrit.cloudera.org:8080/#/c/16833/9/tests/custom_cluster/test_saml2_sso.py@97
PS9, Line 97: d
flake8: F841 local variable 'decoded_saml_req' is assigned to but never used
http://gerrit.cloudera.org:8080/#/c/16833/9/tests/custom_cluster/test_saml2_sso.py@103
PS9, Line 103:
flake8: W293 blank line contains whitespace
http://gerrit.cloudera.org:8080/#/c/16833/9/tests/custom_cluster/test_saml2_sso.py@103
PS9, Line 103:
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/9/tests/custom_cluster/test_saml2_sso.py@104
PS9, Line 104:
flake8: W391 blank line at end of file
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 9
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Jan 2021 21:42:42 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Hello Thomas Tauber-Marshall, Vihang Karajgaonkar, Joe McDonnell, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16833
to look at the new patch set (#16).
Change subject: [WiP] SAML implementation in Impala
......................................................................
[WiP] SAML implementation in Impala
The bulk of the SAML2 related code is done on Java side because:
- There is already a POC in Hive that could be reused.
- The only SAML lib for c++ seems to be OpenSaml, which is seemed
quite hard to use and a heavy dependency.
Doing authentication in Java needed some plumbing, as the hs2-http
port is listened to in c++ and http related processing happens in
THttpServer/THttpTransport, which is not a "real" web server, just
a simple http implementation that processes the headers and passes
content to the thrift service.
- Http headers (and in one case body) are inspected and if it is
SAML related, the http request is wrapped in TWrappedHttpRequest
and sent to the Frontend. The Frontend processes it and returns
a TWrappedHttpResponse with the info to return to the client.
- After the last SAML message (with the bearer token) we generate
an auth cookie in c++ (which can be validated in c++), so later
requests in the session don't need to call to Java.
State of implementation:
- The java side is more or less ok, will be updated when the Hive
implementation changes. I would do a proper cleanup / documentation
once the Hive code is more final.
- Compatibility with other auth mechanisms should be decided:
- Whether other clients should be able to auth with ldap/kerberos
is not clear yet.
- Only the first step of the auth workflow have automatic tests.
Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
---
M be/src/common/global-flags.cc
M be/src/kudu/rpc/CMakeLists.txt
M be/src/rpc/auth-provider.h
M be/src/rpc/authentication-test.cc
M be/src/rpc/authentication.cc
M be/src/rpc/authentication.h
M be/src/rpc/hs2-http-test.cc
M be/src/rpc/thrift-server.h
M be/src/service/frontend.cc
M be/src/service/frontend.h
M be/src/service/impala-server.cc
M be/src/transport/THttpServer.cpp
M be/src/transport/THttpServer.h
M be/src/transport/THttpTransport.cpp
M be/src/transport/THttpTransport.h
M be/src/util/backend-gflag-util.cc
M bin/rat_exclude_files.txt
M common/thrift/BackendGflags.thrift
M common/thrift/Frontend.thrift
M common/thrift/metrics.json
M fe/pom.xml
A fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlHttpServlet.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlUtils.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlAuthenticationException.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlNoGroupsMatchedException.java
A fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
A fe/src/main/java/org/apache/impala/authentication/saml/NullSessionStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/WrappedWebContext.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
M java/pom.xml
A testdata/authentication/saml2_sso.jks
A testdata/authentication/saml2_sso_metadata.xml
A tests/custom_cluster/test_saml2_sso.py
40 files changed, 1,980 insertions(+), 51 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/16833/16
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 16
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
Patch Set 22:
(37 comments)
http://gerrit.cloudera.org:8080/#/c/16833/22/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/22/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java@20
PS22, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/AuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/22/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/22/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@32
PS22, Line 32: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlAuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/22/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@52
PS22, Line 52: private static final Logger LOG = LoggerFactory.getLogger(HiveSamlAuthTokenGenerator.class);
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/22/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java:
http://gerrit.cloudera.org:8080/#/c/16833/22/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java@31
PS22, Line 31: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlGroupNameFilter.java
line too long (132 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/22/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java:
http://gerrit.cloudera.org:8080/#/c/16833/22/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java@20
PS22, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (139 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/22/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java:
http://gerrit.cloudera.org:8080/#/c/16833/22/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java@32
PS22, Line 32: // slightly modified copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (157 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/22/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
File fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java:
http://gerrit.cloudera.org:8080/#/c/16833/22/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@43
PS22, Line 43: // modified version of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSaml2Client.java
line too long (144 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/22/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@69
PS22, Line 69: //TODO handle the replayCache as described in http://www.pac4j.org/docs/clients/saml.html
line too long (93 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/22/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@148
PS22, Line 148: // This is done to keep original structure by Vihang + keep ImpalaSamlClient as the only
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/22/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@188
PS22, Line 188: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpServlet.java
line too long (129 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/22/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@189
PS22, Line 189: private String doSamlAuth(WrappedWebContext webContext) throws HttpSamlAuthenticationException {
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/22/fe/src/main/java/org/apache/impala/service/BackendConfig.java
File fe/src/main/java/org/apache/impala/service/BackendConfig.java:
http://gerrit.cloudera.org:8080/#/c/16833/22/fe/src/main/java/org/apache/impala/service/BackendConfig.java@237
PS22, Line 237: public String getSaml2GroupAttibuteName() { return backendCfg_.saml2_group_attribute_name; }
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/22/tests/custom_cluster/test_saml2_sso.py
File tests/custom_cluster/test_saml2_sso.py:
http://gerrit.cloudera.org:8080/#/c/16833/22/tests/custom_cluster/test_saml2_sso.py@21
PS22, Line 21: import logging
flake8: F401 'logging' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/22/tests/custom_cluster/test_saml2_sso.py@31
PS22, Line 31: from tests.common.impala_service import ImpaladService
flake8: F401 'tests.common.impala_service.ImpaladService' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/22/tests/custom_cluster/test_saml2_sso.py@34
PS22, Line 34: from tests.shell.util import run_impala_shell_cmd, run_impala_shell_cmd_no_expect, \
flake8: F401 'tests.shell.util.run_impala_shell_cmd_no_expect' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/22/tests/custom_cluster/test_saml2_sso.py@34
PS22, Line 34: from tests.shell.util import run_impala_shell_cmd, run_impala_shell_cmd_no_expect, \
flake8: F401 'tests.shell.util.ImpalaShell' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/22/tests/custom_cluster/test_saml2_sso.py@37
PS22, Line 37: class NoRedirection(urllib2.HTTPErrorProcessor):
flake8: E302 expected 2 blank lines, found 1
http://gerrit.cloudera.org:8080/#/c/16833/22/tests/custom_cluster/test_saml2_sso.py@47
PS22, Line 47: class TestClientSaml(CustomClusterTestSuite):
flake8: E302 expected 2 blank lines, found 1
http://gerrit.cloudera.org:8080/#/c/16833/22/tests/custom_cluster/test_saml2_sso.py@71
PS22, Line 71: \
flake8: E502 the backslash is redundant between brackets
http://gerrit.cloudera.org:8080/#/c/16833/22/tests/custom_cluster/test_saml2_sso.py@188
PS22, Line 188:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/22/tests/custom_cluster/test_saml2_sso.py@188
PS22, Line 188:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/22/tests/custom_cluster/test_saml2_sso.py@189
PS22, Line 189:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/22/tests/custom_cluster/test_saml2_sso.py@189
PS22, Line 189:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/22/tests/custom_cluster/test_saml2_sso.py@190
PS22, Line 190:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/22/tests/custom_cluster/test_saml2_sso.py@190
PS22, Line 190:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/22/tests/custom_cluster/test_saml2_sso.py@191
PS22, Line 191:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/22/tests/custom_cluster/test_saml2_sso.py@191
PS22, Line 191:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/22/tests/custom_cluster/test_saml2_sso.py@192
PS22, Line 192:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/22/tests/custom_cluster/test_saml2_sso.py@192
PS22, Line 192:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/22/tests/custom_cluster/test_saml2_sso.py@193
PS22, Line 193:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/22/tests/custom_cluster/test_saml2_sso.py@193
PS22, Line 193:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/22/tests/custom_cluster/test_saml2_sso.py@194
PS22, Line 194:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/22/tests/custom_cluster/test_saml2_sso.py@194
PS22, Line 194:
flake8: E251 unexpected spaces around keyword / parameter equals
http://gerrit.cloudera.org:8080/#/c/16833/22/tests/custom_cluster/test_saml2_sso.py@274
PS22, Line 274: #
flake8: E116 unexpected indentation (comment)
http://gerrit.cloudera.org:8080/#/c/16833/22/tests/custom_cluster/test_saml2_sso.py@277
PS22, Line 277: a
flake8: E501 line too long (96 > 90 characters)
http://gerrit.cloudera.org:8080/#/c/16833/22/tests/custom_cluster/test_saml2_sso.py@280
PS22, Line 280: a
flake8: E501 line too long (113 > 90 characters)
http://gerrit.cloudera.org:8080/#/c/16833/22/tests/custom_cluster/test_saml2_sso.py@283
PS22, Line 283: :
flake8: E501 line too long (98 > 90 characters)
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 22
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 10 Feb 2021 15:15:01 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Hello Thomas Tauber-Marshall, Vihang Karajgaonkar, Joe McDonnell, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16833
to look at the new patch set (#18).
Change subject: [WiP] SAML implementation in Impala
......................................................................
[WiP] SAML implementation in Impala
The bulk of the SAML2 related code is done on Java side because:
- There is already a POC in Hive that could be reused.
- The only SAML lib for c++ seems to be OpenSaml, which is seemed
quite hard to use and a heavy dependency.
Doing authentication in Java needed some plumbing, as the hs2-http
port is listened to in c++ and http related processing happens in
THttpServer/THttpTransport, which is not a "real" web server, just
a simple http implementation that processes the headers and passes
content to the thrift service.
- Http headers (and in one case body) are inspected and if it is
SAML related, the http request is wrapped in TWrappedHttpRequest
and sent to the Frontend. The Frontend processes it and returns
a TWrappedHttpResponse with the info to return to the client.
- After the last SAML message (with the bearer token) we generate
an auth cookie in c++ (which can be validated in c++), so later
requests in the session don't need to call to Java.
State of implementation:
- The java side is more or less ok, will be updated when the Hive
implementation changes. I would do a proper cleanup / documentation
once the Hive code is more final.
- Compatibility with other auth mechanisms should be decided:
- Whether other clients should be able to auth with ldap/kerberos
is not clear yet.
- Only the first step of the auth workflow have automatic tests.
Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
---
M be/src/common/global-flags.cc
M be/src/rpc/auth-provider.h
M be/src/rpc/authentication-test.cc
M be/src/rpc/authentication.cc
M be/src/rpc/authentication.h
M be/src/rpc/hs2-http-test.cc
M be/src/rpc/thrift-server.h
M be/src/service/frontend.cc
M be/src/service/frontend.h
M be/src/service/impala-server.cc
M be/src/transport/THttpServer.cpp
M be/src/transport/THttpServer.h
M be/src/transport/THttpTransport.cpp
M be/src/transport/THttpTransport.h
M be/src/util/backend-gflag-util.cc
M bin/rat_exclude_files.txt
M common/thrift/BackendGflags.thrift
M common/thrift/Frontend.thrift
M common/thrift/metrics.json
M fe/pom.xml
A fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlHttpServlet.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlUtils.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlAuthenticationException.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlNoGroupsMatchedException.java
A fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
A fe/src/main/java/org/apache/impala/authentication/saml/NullSessionStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/WrappedWebContext.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
M java/pom.xml
A testdata/authentication/saml2_sso.jks
A testdata/authentication/saml2_sso_metadata.xml
A tests/custom_cluster/test_saml2_sso.py
39 files changed, 1,968 insertions(+), 50 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/16833/18
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 18
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
Patch Set 26: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 26
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 Feb 2021 22:52:04 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Hello Thomas Tauber-Marshall, Vihang Karajgaonkar, Joe McDonnell, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16833
to look at the new patch set (#17).
Change subject: [WiP] SAML implementation in Impala
......................................................................
[WiP] SAML implementation in Impala
The bulk of the SAML2 related code is done on Java side because:
- There is already a POC in Hive that could be reused.
- The only SAML lib for c++ seems to be OpenSaml, which is seemed
quite hard to use and a heavy dependency.
Doing authentication in Java needed some plumbing, as the hs2-http
port is listened to in c++ and http related processing happens in
THttpServer/THttpTransport, which is not a "real" web server, just
a simple http implementation that processes the headers and passes
content to the thrift service.
- Http headers (and in one case body) are inspected and if it is
SAML related, the http request is wrapped in TWrappedHttpRequest
and sent to the Frontend. The Frontend processes it and returns
a TWrappedHttpResponse with the info to return to the client.
- After the last SAML message (with the bearer token) we generate
an auth cookie in c++ (which can be validated in c++), so later
requests in the session don't need to call to Java.
State of implementation:
- The java side is more or less ok, will be updated when the Hive
implementation changes. I would do a proper cleanup / documentation
once the Hive code is more final.
- Compatibility with other auth mechanisms should be decided:
- Whether other clients should be able to auth with ldap/kerberos
is not clear yet.
- Only the first step of the auth workflow have automatic tests.
Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
---
M be/src/common/global-flags.cc
M be/src/kudu/rpc/CMakeLists.txt
M be/src/rpc/auth-provider.h
M be/src/rpc/authentication-test.cc
M be/src/rpc/authentication.cc
M be/src/rpc/authentication.h
M be/src/rpc/hs2-http-test.cc
M be/src/rpc/thrift-server.h
M be/src/service/frontend.cc
M be/src/service/frontend.h
M be/src/service/impala-server.cc
M be/src/transport/THttpServer.cpp
M be/src/transport/THttpServer.h
M be/src/transport/THttpTransport.cpp
M be/src/transport/THttpTransport.h
M be/src/util/backend-gflag-util.cc
M bin/rat_exclude_files.txt
M common/thrift/BackendGflags.thrift
M common/thrift/Frontend.thrift
M common/thrift/metrics.json
M fe/pom.xml
A fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlHttpServlet.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlUtils.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlAuthenticationException.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlNoGroupsMatchedException.java
A fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
A fe/src/main/java/org/apache/impala/authentication/saml/NullSessionStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/WrappedWebContext.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
M java/pom.xml
A testdata/authentication/saml2_sso.jks
A testdata/authentication/saml2_sso_metadata.xml
A tests/custom_cluster/test_saml2_sso.py
40 files changed, 1,980 insertions(+), 51 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/16833/17
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 17
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Thomas Tauber-Marshall (Code Review)" <ge...@cloudera.org>.
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
Patch Set 24:
(10 comments)
http://gerrit.cloudera.org:8080/#/c/16833/24/be/src/rpc/authentication.h
File be/src/rpc/authentication.h:
http://gerrit.cloudera.org:8080/#/c/16833/24/be/src/rpc/authentication.h@58
PS24, Line 58: AuthProvider* GetExternalHttpAuthProvider();
brief comment
http://gerrit.cloudera.org:8080/#/c/16833/24/be/src/rpc/authentication.cc
File be/src/rpc/authentication.cc:
http://gerrit.cloudera.org:8080/#/c/16833/24/be/src/rpc/authentication.cc@125
PS24, Line 125: allow
nit: allows
http://gerrit.cloudera.org:8080/#/c/16833/24/be/src/rpc/authentication.cc@573
PS24, Line 573: bool ParseParams(std::map<string, string*>& params_to_check,
Brief comment about the parameters here.
http://gerrit.cloudera.org:8080/#/c/16833/24/be/src/rpc/authentication.cc@629
PS24, Line 629: NULL
nit: we prefer nullptr, here and elsewhere
http://gerrit.cloudera.org:8080/#/c/16833/24/be/src/rpc/authentication.cc@683
PS24, Line 683: TODO
probably worth explicitly saying what still needs to be done here
http://gerrit.cloudera.org:8080/#/c/16833/24/be/src/transport/THttpServer.h
File be/src/transport/THttpServer.h:
http://gerrit.cloudera.org:8080/#/c/16833/24/be/src/transport/THttpServer.h@85
PS24, Line 85: porpuses
typo: purposes
http://gerrit.cloudera.org:8080/#/c/16833/24/be/src/transport/THttpServer.cpp
File be/src/transport/THttpServer.cpp:
http://gerrit.cloudera.org:8080/#/c/16833/24/be/src/transport/THttpServer.cpp@228
PS24, Line 228:
nit: unnecessary whitespace
http://gerrit.cloudera.org:8080/#/c/16833/24/be/src/transport/THttpServer.cpp@252
PS24, Line 252: saml_port_ = -1;
: auth_value_ = "";
: cookie_value_ = "";
might be nice to put this logic in some sort of resetConnection() helper function that could be called here, below in the 'if (!authorized && !fallback_to_other_auths)' block and at the very end of headersDone() when we've succeeded.
Could help avoid bugs, eg. if I follow correctly we're currently not resetting saml_port to -1 when SAML succeeds. Possibly that doesn't matter, but better to be safe.
http://gerrit.cloudera.org:8080/#/c/16833/24/common/thrift/metrics.json
File common/thrift/metrics.json:
http://gerrit.cloudera.org:8080/#/c/16833/24/common/thrift/metrics.json@1315
PS24, Line 1315: webserver
This maybe gets a little confusing, since usually when we talk about the "webserver" we're referring to the webui. I think you can just leave this word off and it'll be clear since it says the "HiveServer2 HTTP API", here and below
http://gerrit.cloudera.org:8080/#/c/16833/24/common/thrift/metrics.json@1329
PS24, Line 1329: IC
typo: space
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 24
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 Feb 2021 01:07:36 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: [WiP] SAML implementation in Impala
......................................................................
Patch Set 15:
(26 comments)
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java@20
PS15, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/AuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@32
PS15, Line 32: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlAuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@52
PS15, Line 52: private static final Logger LOG = LoggerFactory.getLogger(HiveSamlAuthTokenGenerator.class);
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java:
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java@31
PS15, Line 31: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlGroupNameFilter.java
line too long (132 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java:
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java@20
PS15, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (139 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java:
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java@32
PS15, Line 32: // slightly modified copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java@32
PS15, Line 32: // slightly modified copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (158 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
File fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java:
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@42
PS15, Line 42: // modified version of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSaml2Client.java
line too long (144 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@65
PS15, Line 65: //TODO handle the replayCache as described in http://www.pac4j.org/docs/clients/saml.html
line too long (93 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@146
PS15, Line 146: // This is done to keep original structure by Vihang + keep ImpalaSamlClient as the only
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@191
PS15, Line 191: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpServlet.java
line too long (129 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@192
PS15, Line 192: private String doSamlAuth(WrappedWebContext webContext) throws HttpSamlAuthenticationException {
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/src/main/java/org/apache/impala/service/BackendConfig.java
File fe/src/main/java/org/apache/impala/service/BackendConfig.java:
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/src/main/java/org/apache/impala/service/BackendConfig.java@237
PS15, Line 237: public String getSaml2GroupAttibuteName() { return backendCfg_.saml2_group_attribute_name; }
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/15/tests/custom_cluster/test_saml2_sso.py
File tests/custom_cluster/test_saml2_sso.py:
http://gerrit.cloudera.org:8080/#/c/16833/15/tests/custom_cluster/test_saml2_sso.py@20
PS15, Line 20: import logging
flake8: F401 'logging' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/15/tests/custom_cluster/test_saml2_sso.py@28
PS15, Line 28: from tests.common.impala_service import ImpaladService
flake8: F401 'tests.common.impala_service.ImpaladService' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/15/tests/custom_cluster/test_saml2_sso.py@31
PS15, Line 31: from tests.shell.util import run_impala_shell_cmd, run_impala_shell_cmd_no_expect, \
flake8: F401 'tests.shell.util.run_impala_shell_cmd_no_expect' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/15/tests/custom_cluster/test_saml2_sso.py@31
PS15, Line 31: from tests.shell.util import run_impala_shell_cmd, run_impala_shell_cmd_no_expect, \
flake8: F401 'tests.shell.util.ImpalaShell' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/15/tests/custom_cluster/test_saml2_sso.py@34
PS15, Line 34: class NoRedirection(urllib2.HTTPErrorProcessor):
flake8: E302 expected 2 blank lines, found 1
http://gerrit.cloudera.org:8080/#/c/16833/15/tests/custom_cluster/test_saml2_sso.py@41
PS15, Line 41: class TestClientSsl(CustomClusterTestSuite):
flake8: E303 too many blank lines (3)
http://gerrit.cloudera.org:8080/#/c/16833/15/tests/custom_cluster/test_saml2_sso.py@46
PS15, Line 46:
flake8: W291 trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/15/tests/custom_cluster/test_saml2_sso.py@46
PS15, Line 46: SP_URL = "https://localhost:8443/simplesaml/saml2/idp/SSOService.php"
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/15/tests/custom_cluster/test_saml2_sso.py@82
PS15, Line 82: r
flake8: F841 local variable 'relay_state' is assigned to but never used
http://gerrit.cloudera.org:8080/#/c/16833/15/tests/custom_cluster/test_saml2_sso.py@97
PS15, Line 97: d
flake8: F841 local variable 'decoded_saml_req' is assigned to but never used
http://gerrit.cloudera.org:8080/#/c/16833/15/tests/custom_cluster/test_saml2_sso.py@103
PS15, Line 103:
flake8: W293 blank line contains whitespace
http://gerrit.cloudera.org:8080/#/c/16833/15/tests/custom_cluster/test_saml2_sso.py@103
PS15, Line 103:
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/15/tests/custom_cluster/test_saml2_sso.py@104
PS15, Line 104:
flake8: W391 blank line at end of file
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 15
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Tue, 19 Jan 2021 21:24:30 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: [WiP] SAML implementation in Impala
......................................................................
Patch Set 11:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/7992/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 11
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Jan 2021 23:52:56 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Vihang Karajgaonkar (Code Review)" <ge...@cloudera.org>.
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
Patch Set 23:
(14 comments)
I mostly have some minor suggestions, but overall the patch looks good to me and I can +1 once the comments are resolved. Will wait for Thomas to look at this patch as well.
http://gerrit.cloudera.org:8080/#/c/16833/23//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/16833/23//COMMIT_MSG@10
PS23, Line 10: POC in Hive
I think mentioning the upstream HIVE jira (HIVE-24543) here would be a useful reference since it links to the design doc.
http://gerrit.cloudera.org:8080/#/c/16833/23/be/src/common/global-flags.cc
File be/src/common/global-flags.cc:
http://gerrit.cloudera.org:8080/#/c/16833/23/be/src/common/global-flags.cc@352
PS23, Line 352:
nit, blank new line.
http://gerrit.cloudera.org:8080/#/c/16833/23/be/src/rpc/authentication.cc
File be/src/rpc/authentication.cc:
http://gerrit.cloudera.org:8080/#/c/16833/23/be/src/rpc/authentication.cc@1338
PS23, Line 1338: sap->InitSaml();
do we need to add a similar msg as above for SAML?
http://gerrit.cloudera.org:8080/#/c/16833/23/be/src/service/frontend.h
File be/src/service/frontend.h:
http://gerrit.cloudera.org:8080/#/c/16833/23/be/src/service/frontend.h@201
PS23, Line 201: Status GetSaml2Redirect(const TWrappedHttpRequest& request,
: TWrappedHttpResponse* response);
:
: Status ValidateSaml2Response(
: const TWrappedHttpRequest& request, TWrappedHttpResponse* response);
Can we add some comments here?
http://gerrit.cloudera.org:8080/#/c/16833/23/common/thrift/metrics.json
File common/thrift/metrics.json:
http://gerrit.cloudera.org:8080/#/c/16833/23/common/thrift/metrics.json@1318
PS23, Line 1318: "CATALOGSERVER",
: "STATESTORE"
Do we need this for catalogd and statestored?
http://gerrit.cloudera.org:8080/#/c/16833/23/common/thrift/metrics.json@1330
PS23, Line 1330: "CATALOGSERVER",
: "STATESTORE"
Do we need this for catalogd and statestored?
http://gerrit.cloudera.org:8080/#/c/16833/23/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/23/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java@20
PS23, Line 20: https://github.com/vihangk1/hive/blob/master_saml
This points to a private branch. Unfortunately, we have not merged this yet in the master branch. This comment might need an update once the Hive patch is merged.
http://gerrit.cloudera.org:8080/#/c/16833/23/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlHttpServlet.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlHttpServlet.java:
http://gerrit.cloudera.org:8080/#/c/16833/23/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlHttpServlet.java@58
PS23, Line 58: //TODO(Vihang) do we need a https://localhost here?
This can be removed. Also, the latest hive patch uses 127.0.0.1 instead of localhost as per the specification RFC 8252 section 7.3 (https://tools.ietf.org/html/rfc8252#section-7.3).
http://gerrit.cloudera.org:8080/#/c/16833/23/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlHttpServlet.java@66
PS23, Line 66: //TODO(Vihang) do we need a https://localhost her
this can be removed. Also, please replace localhost with 127.0.0.1
http://gerrit.cloudera.org:8080/#/c/16833/23/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java:
http://gerrit.cloudera.org:8080/#/c/16833/23/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java@23
PS23, Line 23: codeVerifier
the latest hive patch renames this field to clientIdentifier which is more readable in my opinion.
http://gerrit.cloudera.org:8080/#/c/16833/23/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
File fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java:
http://gerrit.cloudera.org:8080/#/c/16833/23/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@76
PS23, Line 76: conf.getSaml2SpCallbackUrl()
Do we need to validate this call back URL? e.g. the port number is same as http port? and it is using https?
http://gerrit.cloudera.org:8080/#/c/16833/23/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@196
PS23, Line 196: codeVerifier
may be rename this to clientIdentifier.
http://gerrit.cloudera.org:8080/#/c/16833/23/tests/custom_cluster/test_saml2_sso.py
File tests/custom_cluster/test_saml2_sso.py:
http://gerrit.cloudera.org:8080/#/c/16833/23/tests/custom_cluster/test_saml2_sso.py@37
PS23, Line 37: class NoRedirection(urllib2.HTTPErrorProcessor):
I think it would be generally more readable if we have a class level comment giving a high level overview of what the test is doing.
http://gerrit.cloudera.org:8080/#/c/16833/23/tests/custom_cluster/test_saml2_sso.py@71
PS23, Line 71: wIT
nit, WITH
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 23
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Tue, 16 Feb 2021 18:33:36 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Thomas Tauber-Marshall (Code Review)" <ge...@cloudera.org>.
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
Patch Set 26: Code-Review+2
(1 comment)
Looks good, just one final thing I noticed
http://gerrit.cloudera.org:8080/#/c/16833/26/be/src/rpc/authentication.cc
File be/src/rpc/authentication.cc:
http://gerrit.cloudera.org:8080/#/c/16833/26/be/src/rpc/authentication.cc@1368
PS26, Line 1368: if (use_saml) {
This case makes me kind of nervous - easy to imagine a user setting up SAML and not realizing that they're leaving other endpoints entirely unsecured. Maybe we should log an error in that case, eg. if either FLAGS_beeswax_port or FLAGS_hs2_port are non-zero so we're actually using external_auth_provider_, or even disallow it by returning an error.
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 26
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 Feb 2021 22:45:58 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Hello Thomas Tauber-Marshall, Vihang Karajgaonkar, Joe McDonnell, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16833
to look at the new patch set (#14).
Change subject: [WiP] SAML implementation in Impala
......................................................................
[WiP] SAML implementation in Impala
The bulk of the SAML2 related code is done on Java side because:
- There is already a POC in Hive that could be reused.
- The only SAML lib for c++ seems to be OpenSaml, which is seemed
quite hard to use and a heavy dependency.
Doing authentication in Java needed some plumbing, as the hs2-http
port is listened to in c++ and http related processing happens in
THttpServer/THttpTransport, which is not a "real" web server, just
a simple http implementation that processes the headers and passes
content to the thrift service.
- Http headers (and in one case body) are inspected and if it is
SAML related, the http request is wrapped in TWrappedHttpRequest
and sent to the Frontend. The Frontend processes it and returns
a TWrappedHttpResponse with the info to return to the client.
- After the last SAML message (with the bearer token) we generate
an auth cookie in c++ (which can be validated in c++), so later
requests in the session don't need to call to Java.
State of implementation:
- The java side is more or less ok, will be updated when the Hive
implementation changes. I would do a proper cleanup / documentation
once the Hive code is more final.
- The c++ side probsbluneeds a lot of cleanup.
- Compatibility with other auth mechanisms should be decided:
- Whether other clients should be able to auth with ldap/kerberos
is not clear yet.
- Only the first step of the auth workflow have automatic tests.
Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
---
M be/src/common/global-flags.cc
M be/src/rpc/auth-provider.h
M be/src/rpc/authentication-test.cc
M be/src/rpc/authentication.cc
M be/src/rpc/authentication.h
M be/src/rpc/hs2-http-test.cc
M be/src/rpc/thrift-server.h
M be/src/service/frontend.cc
M be/src/service/frontend.h
M be/src/service/impala-server.cc
M be/src/transport/THttpServer.cpp
M be/src/transport/THttpServer.h
M be/src/transport/THttpTransport.cpp
M be/src/transport/THttpTransport.h
M be/src/util/backend-gflag-util.cc
M bin/rat_exclude_files.txt
M common/thrift/BackendGflags.thrift
M common/thrift/Frontend.thrift
M common/thrift/metrics.json
M fe/pom.xml
A fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlHttpServlet.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlUtils.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlAuthenticationException.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlNoGroupsMatchedException.java
A fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
A fe/src/main/java/org/apache/impala/authentication/saml/NullSessionStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/WrappedWebContext.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
M java/pom.xml
A testdata/authentication/saml2_sso.jks
A testdata/authentication/saml2_sso_metadata.xml
A tests/custom_cluster/test_saml2_sso.py
39 files changed, 1,886 insertions(+), 50 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/16833/14
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 14
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: [WiP] SAML implementation in Impala
......................................................................
Patch Set 16:
(20 comments)
http://gerrit.cloudera.org:8080/#/c/16833/16/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/16/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java@20
PS16, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/AuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/16/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/16/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@32
PS16, Line 32: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlAuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/16/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@52
PS16, Line 52: private static final Logger LOG = LoggerFactory.getLogger(HiveSamlAuthTokenGenerator.class);
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/16/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java:
http://gerrit.cloudera.org:8080/#/c/16833/16/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java@31
PS16, Line 31: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlGroupNameFilter.java
line too long (132 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/16/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java:
http://gerrit.cloudera.org:8080/#/c/16833/16/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java@20
PS16, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (139 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/16/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java:
http://gerrit.cloudera.org:8080/#/c/16833/16/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java@32
PS16, Line 32: // slightly modified copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (157 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/16/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
File fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java:
http://gerrit.cloudera.org:8080/#/c/16833/16/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@43
PS16, Line 43: // modified version of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSaml2Client.java
line too long (144 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/16/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@69
PS16, Line 69: //TODO handle the replayCache as described in http://www.pac4j.org/docs/clients/saml.html
line too long (93 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/16/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@150
PS16, Line 150: // This is done to keep original structure by Vihang + keep ImpalaSamlClient as the only
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/16/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@195
PS16, Line 195: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpServlet.java
line too long (129 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/16/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@196
PS16, Line 196: private String doSamlAuth(WrappedWebContext webContext) throws HttpSamlAuthenticationException {
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/16/fe/src/main/java/org/apache/impala/service/BackendConfig.java
File fe/src/main/java/org/apache/impala/service/BackendConfig.java:
http://gerrit.cloudera.org:8080/#/c/16833/16/fe/src/main/java/org/apache/impala/service/BackendConfig.java@237
PS16, Line 237: public String getSaml2GroupAttibuteName() { return backendCfg_.saml2_group_attribute_name; }
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/16/tests/custom_cluster/test_saml2_sso.py
File tests/custom_cluster/test_saml2_sso.py:
http://gerrit.cloudera.org:8080/#/c/16833/16/tests/custom_cluster/test_saml2_sso.py@20
PS16, Line 20: import logging
flake8: F401 'logging' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/16/tests/custom_cluster/test_saml2_sso.py@28
PS16, Line 28: from tests.common.impala_service import ImpaladService
flake8: F401 'tests.common.impala_service.ImpaladService' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/16/tests/custom_cluster/test_saml2_sso.py@31
PS16, Line 31: from tests.shell.util import run_impala_shell_cmd, run_impala_shell_cmd_no_expect, \
flake8: F401 'tests.shell.util.run_impala_shell_cmd_no_expect' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/16/tests/custom_cluster/test_saml2_sso.py@31
PS16, Line 31: from tests.shell.util import run_impala_shell_cmd, run_impala_shell_cmd_no_expect, \
flake8: F401 'tests.shell.util.ImpalaShell' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/16/tests/custom_cluster/test_saml2_sso.py@34
PS16, Line 34: class NoRedirection(urllib2.HTTPErrorProcessor):
flake8: E302 expected 2 blank lines, found 1
http://gerrit.cloudera.org:8080/#/c/16833/16/tests/custom_cluster/test_saml2_sso.py@41
PS16, Line 41: class TestClientSsl(CustomClusterTestSuite):
flake8: E303 too many blank lines (3)
http://gerrit.cloudera.org:8080/#/c/16833/16/tests/custom_cluster/test_saml2_sso.py@82
PS16, Line 82: r
flake8: F841 local variable 'relay_state' is assigned to but never used
http://gerrit.cloudera.org:8080/#/c/16833/16/tests/custom_cluster/test_saml2_sso.py@97
PS16, Line 97: d
flake8: F841 local variable 'decoded_saml_req' is assigned to but never used
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 16
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Fri, 05 Feb 2021 00:51:42 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: [WiP] SAML implementation in Impala
......................................................................
Patch Set 9:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/7990/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 9
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Jan 2021 22:03:42 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: [WiP] SAML implementation in Impala
......................................................................
Patch Set 17:
(7 comments)
http://gerrit.cloudera.org:8080/#/c/16833/15/be/src/common/global-flags.cc
File be/src/common/global-flags.cc:
http://gerrit.cloudera.org:8080/#/c/16833/15/be/src/common/global-flags.cc@347
PS15, Line 347: // HS2 SAML2.0 configuration
> I guess some of these configurations have place holder descriptions and you
Yes - have extended the descriptions, mainly based on the Hive solution.
http://gerrit.cloudera.org:8080/#/c/16833/15/be/src/rpc/authentication.cc
File be/src/rpc/authentication.cc:
http://gerrit.cloudera.org:8080/#/c/16833/15/be/src/rpc/authentication.cc@599
PS15, Line 599: purposes
> nit, spelling
Done
http://gerrit.cloudera.org:8080/#/c/16833/15/be/src/transport/THttpServer.h
File be/src/transport/THttpServer.h:
http://gerrit.cloudera.org:8080/#/c/16833/15/be/src/transport/THttpServer.h@165
PS15, Line 165: // SAML can be used alongside LDAP or Kerberos - if the SAML related path of headers
: // are not detected, Impala fall back to other authentications.
: bool has_saml_ = false;
> Does this mean that Impala can support both SAML and Ldap concurrently? Has
I always planned to do it like this - my assumption is that while SAML browser profile is very useful when a user can do manual interactions like logging in in the browser, it is not suitable for programmatic access, e.g. when a tool like Hue authenticates the users and connects to Impala.
For this reason I kept trusted domain + Ldap + Kerberos auth as fallbacks if they are configured.
http://gerrit.cloudera.org:8080/#/c/16833/15/be/src/transport/THttpServer.cpp
File be/src/transport/THttpServer.cpp:
http://gerrit.cloudera.org:8080/#/c/16833/15/be/src/transport/THttpServer.cpp@137
PS15, Line 137:
> Are you planning to add engine specific identifiers in these header names?
No, I plan to use the exact same names as Hive to make client development easier.
http://gerrit.cloudera.org:8080/#/c/16833/15/be/src/transport/THttpServer.cpp@261
PS15, Line 261: TryStripP
> if the bearer token is present but not valid, do we reach here?
I have excluded this case in patch 16 + 17:
1. if X-Token-Response-Port is present, we always redirect
2. if not then we check for bearer token
3. if there is no X-Token-Response-Port nor bearer token we fallback to other mechanisms
http://gerrit.cloudera.org:8080/#/c/16833/15/common/thrift/BackendGflags.thrift
File common/thrift/BackendGflags.thrift:
http://gerrit.cloudera.org:8080/#/c/16833/15/common/thrift/BackendGflags.thrift@185
PS15, Line 185: token
> s/roken/token/
Done
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/pom.xml
File fe/pom.xml:
http://gerrit.cloudera.org:8080/#/c/16833/15/fe/pom.xml@491
PS15, Line 491: <dependency>
> pac4j brings in tons of dependencies. We should exclude as much as possible
I excluded some dependencies similarly to Hive, but I couldn't exclude some others, as Impala does not depend on them yet, so excluding would mean that pac4j would miss a dependency. I will do another pass to bring the dependencies closer to Hive's.
About reusing Hive classes: that would be great but would need the new classes to be present in a CDP version we use for dependencies. We can discuss whether this is possible in the near future.
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 17
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Fri, 05 Feb 2021 01:13:29 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: [WiP] SAML implementation in Impala
......................................................................
Patch Set 10:
(30 comments)
http://gerrit.cloudera.org:8080/#/c/16833/10/be/src/rpc/authentication.cc
File be/src/rpc/authentication.cc:
http://gerrit.cloudera.org:8080/#/c/16833/10/be/src/rpc/authentication.cc@1160
PS10, Line 1160: callbacks.get_saml_redirect_fn = std::bind(GetSaml2Redirect, connection_ptr.get());
line too long (91 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/10/be/src/transport/THttpServer.h
File be/src/transport/THttpServer.h:
http://gerrit.cloudera.org:8080/#/c/16833/10/be/src/transport/THttpServer.h@160
PS10, Line 160: // 2. validating an authNRespone from the SSO provider
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/10/be/src/transport/THttpServer.h@192
PS10, Line 192: // Used to collect all information about the http request. Can be passed to the Frontend.
line too long (91 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/10/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/10/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java@20
PS10, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/AuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/10/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/10/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@32
PS10, Line 32: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlAuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/10/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@52
PS10, Line 52: private static final Logger LOG = LoggerFactory.getLogger(HiveSamlAuthTokenGenerator.class);
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/10/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java:
http://gerrit.cloudera.org:8080/#/c/16833/10/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java@31
PS10, Line 31: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlGroupNameFilter.java
line too long (132 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/10/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java:
http://gerrit.cloudera.org:8080/#/c/16833/10/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java@20
PS10, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (139 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/10/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java:
http://gerrit.cloudera.org:8080/#/c/16833/10/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java@32
PS10, Line 32: // slightly modified copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/10/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java@32
PS10, Line 32: // slightly modified copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (158 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/10/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
File fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java:
http://gerrit.cloudera.org:8080/#/c/16833/10/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@42
PS10, Line 42: // modified version of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSaml2Client.java
line too long (144 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/10/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@65
PS10, Line 65: //TODO handle the replayCache as described in http://www.pac4j.org/docs/clients/saml.html
line too long (93 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/10/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@146
PS10, Line 146: // This is done to keep original structure by Vihang + keep ImpalaSamlClient as the only
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/10/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@191
PS10, Line 191: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpServlet.java
line too long (129 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/10/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@192
PS10, Line 192: private String doSamlAuth(WrappedWebContext webContext) throws HttpSamlAuthenticationException {
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/10/fe/src/main/java/org/apache/impala/service/BackendConfig.java
File fe/src/main/java/org/apache/impala/service/BackendConfig.java:
http://gerrit.cloudera.org:8080/#/c/16833/10/fe/src/main/java/org/apache/impala/service/BackendConfig.java@245
PS10, Line 245: public String getSaml2GroupAttibuteName() { return backendCfg_.saml2_group_attribute_name; }
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/10/fe/src/main/java/org/apache/impala/service/JniFrontend.java
File fe/src/main/java/org/apache/impala/service/JniFrontend.java:
http://gerrit.cloudera.org:8080/#/c/16833/10/fe/src/main/java/org/apache/impala/service/JniFrontend.java@688
PS10, Line 688:
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/10/tests/custom_cluster/test_saml2_sso.py
File tests/custom_cluster/test_saml2_sso.py:
http://gerrit.cloudera.org:8080/#/c/16833/10/tests/custom_cluster/test_saml2_sso.py@20
PS10, Line 20: import logging
flake8: F401 'logging' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/10/tests/custom_cluster/test_saml2_sso.py@28
PS10, Line 28: from tests.common.impala_service import ImpaladService
flake8: F401 'tests.common.impala_service.ImpaladService' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/10/tests/custom_cluster/test_saml2_sso.py@31
PS10, Line 31: from tests.shell.util import run_impala_shell_cmd, run_impala_shell_cmd_no_expect, \
flake8: F401 'tests.shell.util.run_impala_shell_cmd_no_expect' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/10/tests/custom_cluster/test_saml2_sso.py@31
PS10, Line 31: from tests.shell.util import run_impala_shell_cmd, run_impala_shell_cmd_no_expect, \
flake8: F401 'tests.shell.util.ImpalaShell' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/10/tests/custom_cluster/test_saml2_sso.py@34
PS10, Line 34: class NoRedirection(urllib2.HTTPErrorProcessor):
flake8: E302 expected 2 blank lines, found 1
http://gerrit.cloudera.org:8080/#/c/16833/10/tests/custom_cluster/test_saml2_sso.py@41
PS10, Line 41: class TestClientSsl(CustomClusterTestSuite):
flake8: E303 too many blank lines (3)
http://gerrit.cloudera.org:8080/#/c/16833/10/tests/custom_cluster/test_saml2_sso.py@46
PS10, Line 46:
flake8: W291 trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/10/tests/custom_cluster/test_saml2_sso.py@46
PS10, Line 46: SP_URL = "https://localhost:8443/simplesaml/saml2/idp/SSOService.php"
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/10/tests/custom_cluster/test_saml2_sso.py@82
PS10, Line 82: r
flake8: F841 local variable 'relay_state' is assigned to but never used
http://gerrit.cloudera.org:8080/#/c/16833/10/tests/custom_cluster/test_saml2_sso.py@97
PS10, Line 97: d
flake8: F841 local variable 'decoded_saml_req' is assigned to but never used
http://gerrit.cloudera.org:8080/#/c/16833/10/tests/custom_cluster/test_saml2_sso.py@103
PS10, Line 103:
flake8: W293 blank line contains whitespace
http://gerrit.cloudera.org:8080/#/c/16833/10/tests/custom_cluster/test_saml2_sso.py@103
PS10, Line 103:
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/10/tests/custom_cluster/test_saml2_sso.py@104
PS10, Line 104:
flake8: W391 blank line at end of file
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 10
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Jan 2021 22:54:53 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: [WiP] SAML implementation in Impala
......................................................................
Patch Set 15:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/8025/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 15
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Tue, 19 Jan 2021 21:46:46 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Thomas Tauber-Marshall (Code Review)" <ge...@cloudera.org>.
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: [WiP] SAML implementation in Impala
......................................................................
Patch Set 11:
(11 comments)
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/common/global-flags.cc
File be/src/common/global-flags.cc:
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/common/global-flags.cc@349
PS11, Line 349: keystorp
typo
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/common/global-flags.cc@349
PS11, Line 349: saml2_keystore_password
Its probably better to make this 'saml2_keystore_password_cmd', similar to how we do flags like 'ssl_private_key_password_cmd', to allow users to keep passwords out of configs/logs.
You can also tag it as sensitive so that it will be redacted, see 'ldap_bind_password_cmd' for an example.
And of course the same for 'saml2_private_key_password'
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/common/global-flags.cc@361
PS11, Line 361: hs2-http port.
Might be worth specifically saying this is the --hs2_http_port flag.
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/common/global-flags.cc@363
PS11, Line 363: DEFINE_bool(saml2_want_assertations_signed, true,
Is there any situation where it would be reasonable to set this to 'false' other than just for testing? If not, might be worth explicitly saying 'just for testing' in the description.
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/common/global-flags.cc@372
PS11, Line 372: saml2_callback_roken_ttl
What's the units on this, seconds?
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/transport/THttpServer.h
File be/src/transport/THttpServer.h:
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/transport/THttpServer.h@151
PS11, Line 151: ot
typo
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/transport/THttpServer.cpp
File be/src/transport/THttpServer.cpp:
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/transport/THttpServer.cpp@132
PS11, Line 132: wrapped_request_->headers["Authorization"] = auth_value_;
Any reason to have two copies of this string, rather than just having 'headersDone()' inspect the value from wrapped_request_?
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/transport/THttpServer.cpp@219
PS11, Line 219: readWholeBody_
As noted elsewhere, I find the name 'readWholeBody_' to be confusing. Maybe just 'isSamlResp_', or if you want to keep it more general than just for SAML something like 'needBodyForAuth'.
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/transport/THttpServer.cpp@232
PS11, Line 232: authorized
This will always be 'false' here
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/transport/THttpTransport.cpp
File be/src/transport/THttpTransport.cpp:
http://gerrit.cloudera.org:8080/#/c/16833/11/be/src/transport/THttpTransport.cpp@104
PS11, Line 104: if (readWholeBody_) {
Are we guaranteed that the saml response won't be chunked? I guess presumably the saml response is pretty small and therefore shouldn't ever be chunked, but might be worth checking for this and at least logging an error in that case.
Or better, modify it to work in the chunked case too, just in case, eg. by calling bodyDone() here is readHeaders_ is true.
I also don't think its necessary to have the 'readWholeBody_' variable in THttpTransport - we can just always call bodyDone() when the body is done, and the logic in THttpServer::bodyDone can decide if it actually needs to process the body, especially since I find the name 'readWholeBody' confusing (it brings up the question - don't we always read the whole body?)
http://gerrit.cloudera.org:8080/#/c/16833/11/common/thrift/Frontend.thrift
File common/thrift/Frontend.thrift:
http://gerrit.cloudera.org:8080/#/c/16833/11/common/thrift/Frontend.thrift@988
PS11, Line 988: struct TSamlAuthnResponse {
Needs some comments, at least for the overall struct, not necessarily every param, here and below
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 11
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Thu, 14 Jan 2021 21:47:52 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
Patch Set 25:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/8146/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 25
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 Feb 2021 17:18:35 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
Patch Set 23:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6881/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 23
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 10 Feb 2021 15:21:56 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Hello Thomas Tauber-Marshall, Vihang Karajgaonkar, Joe McDonnell, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16833
to look at the new patch set (#12).
Change subject: [WiP] SAML implementation in Impala
......................................................................
[WiP] SAML implementation in Impala
The bulk of the SAML2 related code is done on Java side because:
- There is already a POC in Hive that could be reused.
- The only SAML lib for c++ seems to be OpenSaml, which is seemed
quite hard to use and a heavy dependency.
Doing authentication in Java needed some plumbing, as the hs2-http
port is listened to in c++ and http related processing happens in
THttpServer/THttpTransport, which is not a "real" web server, just
a simple http implementation that processes the headers and passes
content to the thrift service.
- Http headers (and in one case body) are inspected and if it is
SAML related, the http request is wrapped in TWrappedHttpRequest
and sent to the Frontend. The Frontend processes it and returns
a TWrappedHttpResponse with the info to return to the client.
- After the last SAML message (with the bearer token) we generate
an auth cookie in c++ (which can be validated in c++), so later
requests in the session don't need to call to Java.
State of implementation:
- The java side is more or less ok, will be updated when the Hive
implementation changes. I would do a proper cleanup / documentation
once the Hive code is more final.
- The c++ side probsbluneeds a lot of cleanup.
- Compatibility with other auth mechanisms should be decided:
- Whether other clients should be able to auth with ldap/kerberos
is not clear yet.
- Only the first step of the auth workflow have automatic tests.
Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
---
M be/src/common/global-flags.cc
M be/src/rpc/auth-provider.h
M be/src/rpc/authentication-test.cc
M be/src/rpc/authentication.cc
M be/src/rpc/authentication.h
M be/src/rpc/hs2-http-test.cc
M be/src/rpc/thrift-server.h
M be/src/service/frontend.cc
M be/src/service/frontend.h
M be/src/service/impala-server.cc
M be/src/transport/THttpServer.cpp
M be/src/transport/THttpServer.h
M be/src/transport/THttpTransport.cpp
M be/src/transport/THttpTransport.h
M be/src/util/backend-gflag-util.cc
M bin/rat_exclude_files.txt
M common/thrift/BackendGflags.thrift
M common/thrift/Frontend.thrift
M common/thrift/metrics.json
M fe/pom.xml
A fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlHttpServlet.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlUtils.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlAuthenticationException.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlNoGroupsMatchedException.java
A fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
A fe/src/main/java/org/apache/impala/authentication/saml/NullSessionStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/WrappedWebContext.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
M java/pom.xml
A testdata/authentication/saml2_sso.jks
A testdata/authentication/saml2_sso_metadata.xml
A tests/custom_cluster/test_saml2_sso.py
39 files changed, 1,885 insertions(+), 50 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/16833/12
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 12
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: [WiP] SAML implementation in Impala
......................................................................
Patch Set 17:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/8081/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 17
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Fri, 05 Feb 2021 01:18:12 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
Patch Set 20:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/8108/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 20
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Tue, 09 Feb 2021 22:31:38 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Hello Thomas Tauber-Marshall, Vihang Karajgaonkar, Joe McDonnell, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16833
to look at the new patch set (#22).
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
IMPALA-10496: SAML implementation in Impala
The bulk of the SAML2 related code is done on Java side because:
- There is already a POC in Hive that could be reused.
- The only SAML lib for c++ seems to be OpenSaml, which is seemed
quite hard to use and a heavy dependency.
Doing authentication in Java needed some plumbing, as the hs2-http
port is listened to in c++ and http related processing happens in
THttpServer/THttpTransport, which is not a "real" web server, just
a simple http implementation that processes the headers and passes
content to the thrift service.
- Http headers (and in one case body) are inspected and if it is
SAML related, the http request is wrapped in TWrappedHttpRequest
and sent to the Frontend. The Frontend processes it and returns
a TWrappedHttpResponse with the info to return to the client.
- After the last SAML message (with the bearer token) we generate
an auth cookie in c++ (which can be validated in c++), so later
requests in the session don't need to call to Java.
State of implementation:
- The java side is more or less ok, will be updated when the Hive
implementation changes. I would do a proper cleanup / documentation
once the Hive code is more final.
- Compatibility with other auth mechanisms should be decided:
- Whether other clients should be able to auth with ldap/kerberos
is not clear yet.
Testing:
- Added EE tests that use Python's urllib2 to sent SAML
requests to Impala. Impala works slightly differently
during tests (saml2_ee_test_mode=true).
Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
---
M be/src/common/global-flags.cc
M be/src/rpc/auth-provider.h
M be/src/rpc/authentication-test.cc
M be/src/rpc/authentication.cc
M be/src/rpc/authentication.h
M be/src/rpc/hs2-http-test.cc
M be/src/rpc/thrift-server.h
M be/src/service/frontend.cc
M be/src/service/frontend.h
M be/src/service/impala-server.cc
M be/src/transport/THttpServer.cpp
M be/src/transport/THttpServer.h
M be/src/transport/THttpTransport.cpp
M be/src/transport/THttpTransport.h
M be/src/util/backend-gflag-util.cc
M bin/rat_exclude_files.txt
M common/thrift/BackendGflags.thrift
M common/thrift/Frontend.thrift
M common/thrift/metrics.json
M fe/pom.xml
A fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlHttpServlet.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlUtils.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlAuthenticationException.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlNoGroupsMatchedException.java
A fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
A fe/src/main/java/org/apache/impala/authentication/saml/NullSessionStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/WrappedWebContext.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
M java/pom.xml
A testdata/authentication/saml2_sso.jks
A testdata/authentication/saml2_sso_metadata.xml
A tests/custom_cluster/test_saml2_sso.py
39 files changed, 2,142 insertions(+), 49 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/16833/22
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 22
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Hello Thomas Tauber-Marshall, Vihang Karajgaonkar, Joe McDonnell, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16833
to look at the new patch set (#11).
Change subject: [WiP] SAML implementation in Impala
......................................................................
[WiP] SAML implementation in Impala
The bulk of the SAML2 related code is done on Java side because:
- There is already a POC in Hive that could be reused.
- The only SAML lib for c++ seems to be OpenSaml, which is seemed
quite hard to use and a heavy dependency.
Doing authentication in Java needed some plumbing, as the hs2-http
port is listened to in c++ and http related processing happens in
THttpServer/THttpTransport, which is not a "real" web server, just
a simple http implementation that processes the headers and passes
content to the thrift service.
- Http headers (and in one case body) are inspected and if it is
SAML related, the http request is wrapped in TWrappedHttpRequest
and sent to the Frontend. The Frontend processes it and returns
a TWrappedHttpResponse with the info to return to the client.
- After the last SAML message (with the bearer token) we generate
an auth cookie in c++ (which can be validated in c++), so later
requests in the session don't need to call to Java.
State of implementation:
- The java side is more or less ok, will be updated when the Hive
implementation changes. I would do a proper cleanup / documentation
once the Hive code is more final.
- The c++ side probsbluneeds a lot of cleanup.
- Compatibility with other auth mechanisms should be decided:
- Whether other clients should be able to auth with ldap/kerberos
is not clear yet.
- Only the first step of the auth workflow have automatic tests.
Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
---
M be/src/common/global-flags.cc
M be/src/rpc/auth-provider.h
M be/src/rpc/authentication-test.cc
M be/src/rpc/authentication.cc
M be/src/rpc/authentication.h
M be/src/rpc/hs2-http-test.cc
M be/src/rpc/thrift-server.h
M be/src/service/frontend.cc
M be/src/service/frontend.h
M be/src/service/impala-server.cc
M be/src/transport/THttpServer.cpp
M be/src/transport/THttpServer.h
M be/src/transport/THttpTransport.cpp
M be/src/transport/THttpTransport.h
M be/src/util/backend-gflag-util.cc
M bin/rat_exclude_files.txt
M common/thrift/BackendGflags.thrift
M common/thrift/Frontend.thrift
M common/thrift/metrics.json
M fe/pom.xml
A fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlHttpServlet.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlUtils.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlAuthenticationException.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlNoGroupsMatchedException.java
A fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
A fe/src/main/java/org/apache/impala/authentication/saml/NullSessionStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/WrappedWebContext.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
M java/pom.xml
A testdata/authentication/saml2_sso.jks
A testdata/authentication/saml2_sso_metadata.xml
A tests/custom_cluster/test_saml2_sso.py
39 files changed, 1,883 insertions(+), 51 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/16833/11
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 11
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: [WiP] SAML implementation in Impala
......................................................................
Patch Set 13:
(27 comments)
http://gerrit.cloudera.org:8080/#/c/16833/13/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/13/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java@20
PS13, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/AuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/13/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/13/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@32
PS13, Line 32: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlAuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/13/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@52
PS13, Line 52: private static final Logger LOG = LoggerFactory.getLogger(HiveSamlAuthTokenGenerator.class);
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/13/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java:
http://gerrit.cloudera.org:8080/#/c/16833/13/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java@31
PS13, Line 31: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlGroupNameFilter.java
line too long (132 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/13/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java:
http://gerrit.cloudera.org:8080/#/c/16833/13/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java@20
PS13, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (139 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/13/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java:
http://gerrit.cloudera.org:8080/#/c/16833/13/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java@32
PS13, Line 32: // slightly modified copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/13/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java@32
PS13, Line 32: // slightly modified copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (158 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/13/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
File fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java:
http://gerrit.cloudera.org:8080/#/c/16833/13/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@42
PS13, Line 42: // modified version of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSaml2Client.java
line too long (144 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/13/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@65
PS13, Line 65: //TODO handle the replayCache as described in http://www.pac4j.org/docs/clients/saml.html
line too long (93 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/13/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@146
PS13, Line 146: // This is done to keep original structure by Vihang + keep ImpalaSamlClient as the only
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/13/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@191
PS13, Line 191: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpServlet.java
line too long (129 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/13/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@192
PS13, Line 192: private String doSamlAuth(WrappedWebContext webContext) throws HttpSamlAuthenticationException {
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/13/fe/src/main/java/org/apache/impala/service/BackendConfig.java
File fe/src/main/java/org/apache/impala/service/BackendConfig.java:
http://gerrit.cloudera.org:8080/#/c/16833/13/fe/src/main/java/org/apache/impala/service/BackendConfig.java@237
PS13, Line 237: public String getSaml2GroupAttibuteName() { return backendCfg_.saml2_group_attribute_name; }
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/13/fe/src/main/java/org/apache/impala/service/JniFrontend.java
File fe/src/main/java/org/apache/impala/service/JniFrontend.java:
http://gerrit.cloudera.org:8080/#/c/16833/13/fe/src/main/java/org/apache/impala/service/JniFrontend.java@708
PS13, Line 708:
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/13/tests/custom_cluster/test_saml2_sso.py
File tests/custom_cluster/test_saml2_sso.py:
http://gerrit.cloudera.org:8080/#/c/16833/13/tests/custom_cluster/test_saml2_sso.py@20
PS13, Line 20: import logging
flake8: F401 'logging' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/13/tests/custom_cluster/test_saml2_sso.py@28
PS13, Line 28: from tests.common.impala_service import ImpaladService
flake8: F401 'tests.common.impala_service.ImpaladService' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/13/tests/custom_cluster/test_saml2_sso.py@31
PS13, Line 31: from tests.shell.util import run_impala_shell_cmd, run_impala_shell_cmd_no_expect, \
flake8: F401 'tests.shell.util.run_impala_shell_cmd_no_expect' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/13/tests/custom_cluster/test_saml2_sso.py@31
PS13, Line 31: from tests.shell.util import run_impala_shell_cmd, run_impala_shell_cmd_no_expect, \
flake8: F401 'tests.shell.util.ImpalaShell' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/13/tests/custom_cluster/test_saml2_sso.py@34
PS13, Line 34: class NoRedirection(urllib2.HTTPErrorProcessor):
flake8: E302 expected 2 blank lines, found 1
http://gerrit.cloudera.org:8080/#/c/16833/13/tests/custom_cluster/test_saml2_sso.py@41
PS13, Line 41: class TestClientSsl(CustomClusterTestSuite):
flake8: E303 too many blank lines (3)
http://gerrit.cloudera.org:8080/#/c/16833/13/tests/custom_cluster/test_saml2_sso.py@46
PS13, Line 46:
flake8: W291 trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/13/tests/custom_cluster/test_saml2_sso.py@46
PS13, Line 46: SP_URL = "https://localhost:8443/simplesaml/saml2/idp/SSOService.php"
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/13/tests/custom_cluster/test_saml2_sso.py@82
PS13, Line 82: r
flake8: F841 local variable 'relay_state' is assigned to but never used
http://gerrit.cloudera.org:8080/#/c/16833/13/tests/custom_cluster/test_saml2_sso.py@97
PS13, Line 97: d
flake8: F841 local variable 'decoded_saml_req' is assigned to but never used
http://gerrit.cloudera.org:8080/#/c/16833/13/tests/custom_cluster/test_saml2_sso.py@103
PS13, Line 103:
flake8: W293 blank line contains whitespace
http://gerrit.cloudera.org:8080/#/c/16833/13/tests/custom_cluster/test_saml2_sso.py@103
PS13, Line 103:
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/13/tests/custom_cluster/test_saml2_sso.py@104
PS13, Line 104:
flake8: W391 blank line at end of file
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 13
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Tue, 19 Jan 2021 20:54:15 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Thomas Tauber-Marshall (Code Review)" <ge...@cloudera.org>.
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
Patch Set 27:
Oh, looks like you ran the verify job without 'dry run' set, so when I +2ed it right before the job completed it ended up submitted it even though I had one more outstanding comment. That's my bad.
I think that's fine, though, and of course no need to revert it just to fix that one minor thing.
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 27
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 Feb 2021 22:59:35 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: [WiP] SAML implementation in Impala
......................................................................
Patch Set 17:
(20 comments)
http://gerrit.cloudera.org:8080/#/c/16833/17/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/17/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java@20
PS17, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/AuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/17/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/17/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@32
PS17, Line 32: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlAuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/17/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@52
PS17, Line 52: private static final Logger LOG = LoggerFactory.getLogger(HiveSamlAuthTokenGenerator.class);
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/17/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java:
http://gerrit.cloudera.org:8080/#/c/16833/17/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java@31
PS17, Line 31: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlGroupNameFilter.java
line too long (132 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/17/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java:
http://gerrit.cloudera.org:8080/#/c/16833/17/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java@20
PS17, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (139 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/17/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java:
http://gerrit.cloudera.org:8080/#/c/16833/17/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java@32
PS17, Line 32: // slightly modified copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (157 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/17/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
File fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java:
http://gerrit.cloudera.org:8080/#/c/16833/17/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@43
PS17, Line 43: // modified version of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSaml2Client.java
line too long (144 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/17/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@69
PS17, Line 69: //TODO handle the replayCache as described in http://www.pac4j.org/docs/clients/saml.html
line too long (93 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/17/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@150
PS17, Line 150: // This is done to keep original structure by Vihang + keep ImpalaSamlClient as the only
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/17/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@195
PS17, Line 195: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpServlet.java
line too long (129 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/17/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@196
PS17, Line 196: private String doSamlAuth(WrappedWebContext webContext) throws HttpSamlAuthenticationException {
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/17/fe/src/main/java/org/apache/impala/service/BackendConfig.java
File fe/src/main/java/org/apache/impala/service/BackendConfig.java:
http://gerrit.cloudera.org:8080/#/c/16833/17/fe/src/main/java/org/apache/impala/service/BackendConfig.java@237
PS17, Line 237: public String getSaml2GroupAttibuteName() { return backendCfg_.saml2_group_attribute_name; }
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/17/tests/custom_cluster/test_saml2_sso.py
File tests/custom_cluster/test_saml2_sso.py:
http://gerrit.cloudera.org:8080/#/c/16833/17/tests/custom_cluster/test_saml2_sso.py@20
PS17, Line 20: import logging
flake8: F401 'logging' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/17/tests/custom_cluster/test_saml2_sso.py@28
PS17, Line 28: from tests.common.impala_service import ImpaladService
flake8: F401 'tests.common.impala_service.ImpaladService' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/17/tests/custom_cluster/test_saml2_sso.py@31
PS17, Line 31: from tests.shell.util import run_impala_shell_cmd, run_impala_shell_cmd_no_expect, \
flake8: F401 'tests.shell.util.run_impala_shell_cmd_no_expect' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/17/tests/custom_cluster/test_saml2_sso.py@31
PS17, Line 31: from tests.shell.util import run_impala_shell_cmd, run_impala_shell_cmd_no_expect, \
flake8: F401 'tests.shell.util.ImpalaShell' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/17/tests/custom_cluster/test_saml2_sso.py@34
PS17, Line 34: class NoRedirection(urllib2.HTTPErrorProcessor):
flake8: E302 expected 2 blank lines, found 1
http://gerrit.cloudera.org:8080/#/c/16833/17/tests/custom_cluster/test_saml2_sso.py@41
PS17, Line 41: class TestClientSsl(CustomClusterTestSuite):
flake8: E303 too many blank lines (3)
http://gerrit.cloudera.org:8080/#/c/16833/17/tests/custom_cluster/test_saml2_sso.py@82
PS17, Line 82: r
flake8: F841 local variable 'relay_state' is assigned to but never used
http://gerrit.cloudera.org:8080/#/c/16833/17/tests/custom_cluster/test_saml2_sso.py@97
PS17, Line 97: d
flake8: F841 local variable 'decoded_saml_req' is assigned to but never used
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 17
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Fri, 05 Feb 2021 01:10:55 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
IMPALA-10496: SAML implementation in Impala
The bulk of the SAML2 related code is done on Java side because:
- There is already an implementation for Hive on review (HIVE-24543).
- The only SAML lib for c++ seems to be OpenSaml, which is seemed
quite hard to use and a heavy dependency.
Doing authentication in Java needed some plumbing, as the hs2-http
port is listened to in c++ and http related processing happens in
THttpServer/THttpTransport, which is not a "real" web server, just
a simple http implementation that processes the headers and passes
content to the thrift service.
- Http headers (and in one case body) are inspected and if it is
SAML related, the http request is wrapped in TWrappedHttpRequest
and sent to the Frontend. The Frontend processes it and returns
a TWrappedHttpResponse with the info to return to the client.
- After the last SAML message (with the bearer token) we generate
an auth cookie in c++ (which can be validated in c++), so later
requests in the session don't need to call to Java.
SAML auth can work alongside LDAP and Kerberos - for each hs2-http
request the path and the http headers are inspected to decide
whether it is SAML related, and if not, then we fallback to other
auth mechanisms. This "mixed mode" has no tests yet, so I consider it
experimental.
Planned followup work:
- It would be great to import the logic implemented in Hive instead
of copy-pasting most of it. I plan to do this in a followup commit,
as this needs changes on the Hive side too.
- Adding more tests will be much easier once we will have a hs2-http
client that supports SAML. See IMPALA-10496 for Impyla support.
- Currently the debug webserver does not support SAML auth.
Implementing SAML for the webserver is problematic on the statestore
which doesn't have a Frontend.
Testing:
- Added EE tests that use Python's urllib2 to sent SAML
requests to Impala. Impala works slightly differently
during tests (saml2_ee_test_mode=true).
Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Reviewed-on: http://gerrit.cloudera.org:8080/16833
Reviewed-by: Thomas Tauber-Marshall <tm...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/rpc/auth-provider.h
M be/src/rpc/authentication-test.cc
M be/src/rpc/authentication.cc
M be/src/rpc/authentication.h
M be/src/rpc/hs2-http-test.cc
M be/src/rpc/thrift-server.h
M be/src/service/frontend.cc
M be/src/service/frontend.h
M be/src/service/impala-server.cc
M be/src/transport/THttpServer.cpp
M be/src/transport/THttpServer.h
M be/src/transport/THttpTransport.cpp
M be/src/transport/THttpTransport.h
M be/src/util/backend-gflag-util.cc
M bin/rat_exclude_files.txt
M common/thrift/BackendGflags.thrift
M common/thrift/Frontend.thrift
M common/thrift/metrics.json
M fe/pom.xml
A fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlHttpServlet.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlUtils.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlAuthenticationException.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlNoGroupsMatchedException.java
A fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
A fe/src/main/java/org/apache/impala/authentication/saml/NullSessionStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/WrappedWebContext.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
M java/pom.xml
A testdata/authentication/saml2_sso.jks
A testdata/authentication/saml2_sso_metadata.xml
A tests/custom_cluster/test_saml2_sso.py
38 files changed, 2,202 insertions(+), 53 deletions(-)
Approvals:
Thomas Tauber-Marshall: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 27
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: [WiP] SAML implementation in Impala
......................................................................
Patch Set 14:
(26 comments)
http://gerrit.cloudera.org:8080/#/c/16833/14/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/14/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java@20
PS14, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/AuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/14/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/14/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@32
PS14, Line 32: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlAuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/14/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@52
PS14, Line 52: private static final Logger LOG = LoggerFactory.getLogger(HiveSamlAuthTokenGenerator.class);
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/14/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java:
http://gerrit.cloudera.org:8080/#/c/16833/14/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java@31
PS14, Line 31: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlGroupNameFilter.java
line too long (132 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/14/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java:
http://gerrit.cloudera.org:8080/#/c/16833/14/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java@20
PS14, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (139 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/14/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java:
http://gerrit.cloudera.org:8080/#/c/16833/14/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java@32
PS14, Line 32: // slightly modified copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/14/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java@32
PS14, Line 32: // slightly modified copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (158 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/14/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
File fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java:
http://gerrit.cloudera.org:8080/#/c/16833/14/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@42
PS14, Line 42: // modified version of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSaml2Client.java
line too long (144 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/14/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@65
PS14, Line 65: //TODO handle the replayCache as described in http://www.pac4j.org/docs/clients/saml.html
line too long (93 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/14/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@146
PS14, Line 146: // This is done to keep original structure by Vihang + keep ImpalaSamlClient as the only
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/14/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@191
PS14, Line 191: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpServlet.java
line too long (129 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/14/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@192
PS14, Line 192: private String doSamlAuth(WrappedWebContext webContext) throws HttpSamlAuthenticationException {
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/14/fe/src/main/java/org/apache/impala/service/BackendConfig.java
File fe/src/main/java/org/apache/impala/service/BackendConfig.java:
http://gerrit.cloudera.org:8080/#/c/16833/14/fe/src/main/java/org/apache/impala/service/BackendConfig.java@237
PS14, Line 237: public String getSaml2GroupAttibuteName() { return backendCfg_.saml2_group_attribute_name; }
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/14/tests/custom_cluster/test_saml2_sso.py
File tests/custom_cluster/test_saml2_sso.py:
http://gerrit.cloudera.org:8080/#/c/16833/14/tests/custom_cluster/test_saml2_sso.py@20
PS14, Line 20: import logging
flake8: F401 'logging' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/14/tests/custom_cluster/test_saml2_sso.py@28
PS14, Line 28: from tests.common.impala_service import ImpaladService
flake8: F401 'tests.common.impala_service.ImpaladService' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/14/tests/custom_cluster/test_saml2_sso.py@31
PS14, Line 31: from tests.shell.util import run_impala_shell_cmd, run_impala_shell_cmd_no_expect, \
flake8: F401 'tests.shell.util.run_impala_shell_cmd_no_expect' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/14/tests/custom_cluster/test_saml2_sso.py@31
PS14, Line 31: from tests.shell.util import run_impala_shell_cmd, run_impala_shell_cmd_no_expect, \
flake8: F401 'tests.shell.util.ImpalaShell' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/14/tests/custom_cluster/test_saml2_sso.py@34
PS14, Line 34: class NoRedirection(urllib2.HTTPErrorProcessor):
flake8: E302 expected 2 blank lines, found 1
http://gerrit.cloudera.org:8080/#/c/16833/14/tests/custom_cluster/test_saml2_sso.py@41
PS14, Line 41: class TestClientSsl(CustomClusterTestSuite):
flake8: E303 too many blank lines (3)
http://gerrit.cloudera.org:8080/#/c/16833/14/tests/custom_cluster/test_saml2_sso.py@46
PS14, Line 46:
flake8: W291 trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/14/tests/custom_cluster/test_saml2_sso.py@46
PS14, Line 46: SP_URL = "https://localhost:8443/simplesaml/saml2/idp/SSOService.php"
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/14/tests/custom_cluster/test_saml2_sso.py@82
PS14, Line 82: r
flake8: F841 local variable 'relay_state' is assigned to but never used
http://gerrit.cloudera.org:8080/#/c/16833/14/tests/custom_cluster/test_saml2_sso.py@97
PS14, Line 97: d
flake8: F841 local variable 'decoded_saml_req' is assigned to but never used
http://gerrit.cloudera.org:8080/#/c/16833/14/tests/custom_cluster/test_saml2_sso.py@103
PS14, Line 103:
flake8: W293 blank line contains whitespace
http://gerrit.cloudera.org:8080/#/c/16833/14/tests/custom_cluster/test_saml2_sso.py@103
PS14, Line 103:
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/16833/14/tests/custom_cluster/test_saml2_sso.py@104
PS14, Line 104:
flake8: W391 blank line at end of file
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 14
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Tue, 19 Jan 2021 21:13:07 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: [WiP] SAML implementation in Impala
......................................................................
Patch Set 10:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/7991/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 10
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Jan 2021 23:15:51 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: [WiP] SAML implementation in Impala
......................................................................
Patch Set 18:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/8093/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 18
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Mon, 08 Feb 2021 21:23:15 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: [WiP] SAML implementation in Impala
......................................................................
Patch Set 16:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/8080/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 16
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Fri, 05 Feb 2021 01:00:04 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Hello Thomas Tauber-Marshall, Vihang Karajgaonkar, Joe McDonnell, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16833
to look at the new patch set (#15).
Change subject: [WiP] SAML implementation in Impala
......................................................................
[WiP] SAML implementation in Impala
The bulk of the SAML2 related code is done on Java side because:
- There is already a POC in Hive that could be reused.
- The only SAML lib for c++ seems to be OpenSaml, which is seemed
quite hard to use and a heavy dependency.
Doing authentication in Java needed some plumbing, as the hs2-http
port is listened to in c++ and http related processing happens in
THttpServer/THttpTransport, which is not a "real" web server, just
a simple http implementation that processes the headers and passes
content to the thrift service.
- Http headers (and in one case body) are inspected and if it is
SAML related, the http request is wrapped in TWrappedHttpRequest
and sent to the Frontend. The Frontend processes it and returns
a TWrappedHttpResponse with the info to return to the client.
- After the last SAML message (with the bearer token) we generate
an auth cookie in c++ (which can be validated in c++), so later
requests in the session don't need to call to Java.
State of implementation:
- The java side is more or less ok, will be updated when the Hive
implementation changes. I would do a proper cleanup / documentation
once the Hive code is more final.
- The c++ side probsbluneeds a lot of cleanup.
- Compatibility with other auth mechanisms should be decided:
- Whether other clients should be able to auth with ldap/kerberos
is not clear yet.
- Only the first step of the auth workflow have automatic tests.
Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
---
M be/src/common/global-flags.cc
M be/src/rpc/auth-provider.h
M be/src/rpc/authentication-test.cc
M be/src/rpc/authentication.cc
M be/src/rpc/authentication.h
M be/src/rpc/hs2-http-test.cc
M be/src/rpc/thrift-server.h
M be/src/service/frontend.cc
M be/src/service/frontend.h
M be/src/service/impala-server.cc
M be/src/transport/THttpServer.cpp
M be/src/transport/THttpServer.h
M be/src/transport/THttpTransport.cpp
M be/src/transport/THttpTransport.h
M be/src/util/backend-gflag-util.cc
M bin/rat_exclude_files.txt
M common/thrift/BackendGflags.thrift
M common/thrift/Frontend.thrift
M common/thrift/metrics.json
M fe/pom.xml
A fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlHttpServlet.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlUtils.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlAuthenticationException.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlNoGroupsMatchedException.java
A fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
A fe/src/main/java/org/apache/impala/authentication/saml/NullSessionStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/WrappedWebContext.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
M java/pom.xml
A testdata/authentication/saml2_sso.jks
A testdata/authentication/saml2_sso_metadata.xml
A tests/custom_cluster/test_saml2_sso.py
39 files changed, 1,886 insertions(+), 50 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/16833/15
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 15
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: [WiP] SAML implementation in Impala
......................................................................
Patch Set 14:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/8024/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 14
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Tue, 19 Jan 2021 21:34:41 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: [WiP] SAML implementation in Impala
......................................................................
Patch Set 18:
(20 comments)
http://gerrit.cloudera.org:8080/#/c/16833/18/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/18/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java@20
PS18, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/AuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/18/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/18/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@32
PS18, Line 32: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlAuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/18/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@52
PS18, Line 52: private static final Logger LOG = LoggerFactory.getLogger(HiveSamlAuthTokenGenerator.class);
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/18/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java:
http://gerrit.cloudera.org:8080/#/c/16833/18/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java@31
PS18, Line 31: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlGroupNameFilter.java
line too long (132 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/18/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java:
http://gerrit.cloudera.org:8080/#/c/16833/18/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java@20
PS18, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (139 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/18/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java:
http://gerrit.cloudera.org:8080/#/c/16833/18/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java@32
PS18, Line 32: // slightly modified copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (157 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/18/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
File fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java:
http://gerrit.cloudera.org:8080/#/c/16833/18/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@43
PS18, Line 43: // modified version of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSaml2Client.java
line too long (144 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/18/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@69
PS18, Line 69: //TODO handle the replayCache as described in http://www.pac4j.org/docs/clients/saml.html
line too long (93 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/18/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@150
PS18, Line 150: // This is done to keep original structure by Vihang + keep ImpalaSamlClient as the only
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/18/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@195
PS18, Line 195: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpServlet.java
line too long (129 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/18/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@196
PS18, Line 196: private String doSamlAuth(WrappedWebContext webContext) throws HttpSamlAuthenticationException {
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/18/fe/src/main/java/org/apache/impala/service/BackendConfig.java
File fe/src/main/java/org/apache/impala/service/BackendConfig.java:
http://gerrit.cloudera.org:8080/#/c/16833/18/fe/src/main/java/org/apache/impala/service/BackendConfig.java@237
PS18, Line 237: public String getSaml2GroupAttibuteName() { return backendCfg_.saml2_group_attribute_name; }
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/18/tests/custom_cluster/test_saml2_sso.py
File tests/custom_cluster/test_saml2_sso.py:
http://gerrit.cloudera.org:8080/#/c/16833/18/tests/custom_cluster/test_saml2_sso.py@20
PS18, Line 20: import logging
flake8: F401 'logging' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/18/tests/custom_cluster/test_saml2_sso.py@28
PS18, Line 28: from tests.common.impala_service import ImpaladService
flake8: F401 'tests.common.impala_service.ImpaladService' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/18/tests/custom_cluster/test_saml2_sso.py@31
PS18, Line 31: from tests.shell.util import run_impala_shell_cmd, run_impala_shell_cmd_no_expect, \
flake8: F401 'tests.shell.util.run_impala_shell_cmd_no_expect' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/18/tests/custom_cluster/test_saml2_sso.py@31
PS18, Line 31: from tests.shell.util import run_impala_shell_cmd, run_impala_shell_cmd_no_expect, \
flake8: F401 'tests.shell.util.ImpalaShell' imported but unused
http://gerrit.cloudera.org:8080/#/c/16833/18/tests/custom_cluster/test_saml2_sso.py@34
PS18, Line 34: class NoRedirection(urllib2.HTTPErrorProcessor):
flake8: E302 expected 2 blank lines, found 1
http://gerrit.cloudera.org:8080/#/c/16833/18/tests/custom_cluster/test_saml2_sso.py@41
PS18, Line 41: class TestClientSsl(CustomClusterTestSuite):
flake8: E303 too many blank lines (3)
http://gerrit.cloudera.org:8080/#/c/16833/18/tests/custom_cluster/test_saml2_sso.py@82
PS18, Line 82: r
flake8: F841 local variable 'relay_state' is assigned to but never used
http://gerrit.cloudera.org:8080/#/c/16833/18/tests/custom_cluster/test_saml2_sso.py@97
PS18, Line 97: d
flake8: F841 local variable 'decoded_saml_req' is assigned to but never used
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 18
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Mon, 08 Feb 2021 21:00:59 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] [WiP] SAML implementation in Impala
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Hello Thomas Tauber-Marshall, Vihang Karajgaonkar, Joe McDonnell, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16833
to look at the new patch set (#13).
Change subject: [WiP] SAML implementation in Impala
......................................................................
[WiP] SAML implementation in Impala
The bulk of the SAML2 related code is done on Java side because:
- There is already a POC in Hive that could be reused.
- The only SAML lib for c++ seems to be OpenSaml, which is seemed
quite hard to use and a heavy dependency.
Doing authentication in Java needed some plumbing, as the hs2-http
port is listened to in c++ and http related processing happens in
THttpServer/THttpTransport, which is not a "real" web server, just
a simple http implementation that processes the headers and passes
content to the thrift service.
- Http headers (and in one case body) are inspected and if it is
SAML related, the http request is wrapped in TWrappedHttpRequest
and sent to the Frontend. The Frontend processes it and returns
a TWrappedHttpResponse with the info to return to the client.
- After the last SAML message (with the bearer token) we generate
an auth cookie in c++ (which can be validated in c++), so later
requests in the session don't need to call to Java.
State of implementation:
- The java side is more or less ok, will be updated when the Hive
implementation changes. I would do a proper cleanup / documentation
once the Hive code is more final.
- The c++ side probsbluneeds a lot of cleanup.
- Compatibility with other auth mechanisms should be decided:
- Whether other clients should be able to auth with ldap/kerberos
is not clear yet.
- Only the first step of the auth workflow have automatic tests.
Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
---
M be/src/common/global-flags.cc
M be/src/rpc/auth-provider.h
M be/src/rpc/authentication-test.cc
M be/src/rpc/authentication.cc
M be/src/rpc/authentication.h
M be/src/rpc/hs2-http-test.cc
M be/src/rpc/thrift-server.h
M be/src/service/frontend.cc
M be/src/service/frontend.h
M be/src/service/impala-server.cc
M be/src/transport/THttpServer.cpp
M be/src/transport/THttpServer.h
M be/src/transport/THttpTransport.cpp
M be/src/transport/THttpTransport.h
M be/src/util/backend-gflag-util.cc
M bin/rat_exclude_files.txt
M common/thrift/BackendGflags.thrift
M common/thrift/Frontend.thrift
M common/thrift/metrics.json
M fe/pom.xml
A fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlHttpServlet.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlUtils.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlAuthenticationException.java
A fe/src/main/java/org/apache/impala/authentication/saml/HttpSamlNoGroupsMatchedException.java
A fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
A fe/src/main/java/org/apache/impala/authentication/saml/NullSessionStore.java
A fe/src/main/java/org/apache/impala/authentication/saml/WrappedWebContext.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
M java/pom.xml
A testdata/authentication/saml2_sso.jks
A testdata/authentication/saml2_sso_metadata.xml
A tests/custom_cluster/test_saml2_sso.py
39 files changed, 1,889 insertions(+), 50 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/16833/13
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 13
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
[Impala-ASF-CR] IMPALA-10496: SAML implementation in Impala
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16833 )
Change subject: IMPALA-10496: SAML implementation in Impala
......................................................................
Patch Set 25:
(16 comments)
http://gerrit.cloudera.org:8080/#/c/16833/25/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/25/fe/src/main/java/org/apache/impala/authentication/saml/AuthTokenGenerator.java@20
PS25, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/AuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/25/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java:
http://gerrit.cloudera.org:8080/#/c/16833/25/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@32
PS25, Line 32: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlAuthTokenGenerator.java
line too long (135 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/25/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlAuthTokenGenerator.java@52
PS25, Line 52: private static final Logger LOG = LoggerFactory.getLogger(HiveSamlAuthTokenGenerator.class);
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/25/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java:
http://gerrit.cloudera.org:8080/#/c/16833/25/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlGroupNameFilter.java@31
PS25, Line 31: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlGroupNameFilter.java
line too long (132 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/25/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java:
http://gerrit.cloudera.org:8080/#/c/16833/25/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateInfo.java@20
PS25, Line 20: // copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (139 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/25/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java
File fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java:
http://gerrit.cloudera.org:8080/#/c/16833/25/fe/src/main/java/org/apache/impala/authentication/saml/HiveSamlRelayStateStore.java@32
PS25, Line 32: // slightly modified copy of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSamlRelayStateInfo.java
line too long (157 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/25/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java
File fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java:
http://gerrit.cloudera.org:8080/#/c/16833/25/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@43
PS25, Line 43: // modified version of https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/auth/saml/HiveSaml2Client.java
line too long (144 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/25/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@69
PS25, Line 69: //TODO handle the replayCache as described in http://www.pac4j.org/docs/clients/saml.html
line too long (93 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/25/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@147
PS25, Line 147: // This is done to keep original structure by Vihang + keep ImpalaSamlClient as the only
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/25/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@187
PS25, Line 187: // https://github.com/vihangk1/hive/blob/master_saml/service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpServlet.java
line too long (129 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/25/fe/src/main/java/org/apache/impala/authentication/saml/ImpalaSamlClient.java@188
PS25, Line 188: private String doSamlAuth(WrappedWebContext webContext) throws HttpSamlAuthenticationException {
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/25/fe/src/main/java/org/apache/impala/service/BackendConfig.java
File fe/src/main/java/org/apache/impala/service/BackendConfig.java:
http://gerrit.cloudera.org:8080/#/c/16833/25/fe/src/main/java/org/apache/impala/service/BackendConfig.java@237
PS25, Line 237: public String getSaml2GroupAttibuteName() { return backendCfg_.saml2_group_attribute_name; }
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/16833/25/tests/custom_cluster/test_saml2_sso.py
File tests/custom_cluster/test_saml2_sso.py:
http://gerrit.cloudera.org:8080/#/c/16833/25/tests/custom_cluster/test_saml2_sso.py@34
PS25, Line 34: class NoRedirection(urllib2.HTTPErrorProcessor):
flake8: E302 expected 2 blank lines, found 1
http://gerrit.cloudera.org:8080/#/c/16833/25/tests/custom_cluster/test_saml2_sso.py@304
PS25, Line 304: a
flake8: E501 line too long (96 > 90 characters)
http://gerrit.cloudera.org:8080/#/c/16833/25/tests/custom_cluster/test_saml2_sso.py@307
PS25, Line 307: a
flake8: E501 line too long (113 > 90 characters)
http://gerrit.cloudera.org:8080/#/c/16833/25/tests/custom_cluster/test_saml2_sso.py@310
PS25, Line 310: :
flake8: E501 line too long (98 > 90 characters)
--
To view, visit http://gerrit.cloudera.org:8080/16833
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia0c026cba1b90e7ff6ec5ae49be78b0d1edd8dfa
Gerrit-Change-Number: 16833
Gerrit-PatchSet: 25
Gerrit-Owner: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 Feb 2021 16:59:11 +0000
Gerrit-HasComments: Yes