RE: How to use Livy Client API with SPNEGO

["Partridge, Lucas (GE Aviation)" <Lu...@ge.com>]

I’m not sure this answers your question directly but I found getting a Livy Java client on Windows to talk successfully to a Kerberos-enabled cluster to be a major pain in the neck. However these links might help you:
https://stackoverflow.com/questions/46909048/livy-rest-api-get-requests-work-but-post-requests-fail-with-401-authentication
https://stackoverflow.com/questions/45957379/curl-on-windows-gssexception-defective-token-detected-mechanism-level-gsshe
Good luck!
Lucas.

From: Joel Folkerts <jo...@gmail.com>
Sent: 30 April 2019 18:21
To: user@livy.incubator.apache.org
Subject: EXT: How to use Livy Client API with SPNEGO

I am attempting to authenticate against a Kerberos-enabled cluster suing the Livy API:

Setting up Kerberos properties:

Properties props = new Properties();
props.setProperty("spnego.enabled", "true");
props.setProperty("auth.login.config", "C:\\PATH_TO_\\jaas-client.conf");
props.setProperty("krb5.debug", "true");
props.setProperty("krb5.conf", "C:\\Windows\\krb5.ini");

Building Livy client with Kerberos properties

this.livyClient = new LivyClientBuilder()
        .setURI(this.config.getLivyURI())
        .setAll(props)
        .build();
I receive the following exception:

NEGOTIATE authentication error: No valid credentials provided (Mechanism level: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt))

Looking closer at the Livy source code, it appears that the enum value "Entry.SPNEGO_ENABLED" needs to be set to false; however, I don't see a way to access the enum Entry object prior to the check on line 66:

https://github.com/cloudera/livy/blob/9ae24d08738652ba5fd817780711d01b110d74a9/client-http/src/main/java/com/cloudera/livy/client/http/HttpConf.java#L66

Any help would be greatly appreciated!

Re: How to use Livy Client API with SPNEGO

[Joel Folkerts <jo...@gmail.com>]

Thank Lucas - I appreciate the links! At first glance, they appear to be
discussing the REST API (
https://livy.incubator.apache.org/docs/latest/rest-api.html) whereas I am
using the Programmatic API (
https://livy.incubator.apache.org/docs/latest/programmatic-api.html). That
said, I will review closely in hopes that they may lead me to a missing
configuration setting.

Following my initial email yesterday, I forked the repo and inserted a
simple config parser POC:
https://github.com/apexxs/incubator-livy/commit/09feac91fc094e5f4587e7e2e9a9b40101b60794.
This allows the HttpConf class to analyze its passed Config entries and
update any matching static Enum values. By no means production worthy but
wanted to reach the rest of the constructor to set up the Kerberos client
session values. As it turns out, the previously unreached code in the
constructor merely sets a few System properties that I could have set in my
code, namely:

System.setProperty("java.security.auth.login.config",
get(Entry.AUTH_LOGIN_CONFIG));
System.setProperty("java.security.krb5.conf", get(Entry.KRB5_CONF));
System.setProperty(
  "sun.security.krb5.debug",
String.valueOf(getBoolean(Entry.KRB5_DEBUG_ENABLED)));
// This is needed to get Kerberos credentials from the environment, instead of
// requiring the application to manually obtain the credentials.
System.setProperty("javax.security.auth.useSubjectCredsOnly", "false");

Despite this, I am still seeing the same Kerberos client exception. I
suspect that it's now an issue with my keytaband/or jaas-client.conf file.

On Wed, May 1, 2019 at 5:38 AM Partridge, Lucas (GE Aviation) <
Lucas.Partridge@ge.com> wrote:

> I’m not sure this answers your question directly but I found getting a
> Livy Java client on Windows to talk successfully to a Kerberos-enabled
> cluster to be a major pain in the neck. However these links might help you:
>
>
> https://stackoverflow.com/questions/46909048/livy-rest-api-get-requests-work-but-post-requests-fail-with-401-authentication
>
>
> https://stackoverflow.com/questions/45957379/curl-on-windows-gssexception-defective-token-detected-mechanism-level-gsshe
>
> Good luck!
>
> Lucas.
>
>
>
> *From:* Joel Folkerts <jo...@gmail.com>
> *Sent:* 30 April 2019 18:21
> *To:* user@livy.incubator.apache.org
> *Subject:* EXT: How to use Livy Client API with SPNEGO
>
>
>
> I am attempting to authenticate against a Kerberos-enabled cluster suing
> the Livy API:
>
>
>
> Setting up Kerberos properties:
>
> *Properties *props = *new *Properties();
> props.setProperty("spnego.enabled", "true");
> props.setProperty("auth.login.config", "C:*\\*PATH_TO_*\\*jaas-client.conf");
> props.setProperty("krb5.debug", "true");
> props.setProperty("krb5.conf", "C:*\\*Windows*\\*krb5.ini");
>
>
>
> Building Livy client with Kerberos properties
>
> *this*.*livyClient *= *new *LivyClientBuilder()
>         .setURI(*this*.*config*.getLivyURI())
>         .setAll(props)
>         .build();
>
> I receive the following exception:
>
>
>
> *NEGOTIATE authentication error: No valid credentials provided (Mechanism
> level: No valid credentials provided (Mechanism level: Failed to find any
> Kerberos tgt))*
>
>
>
> Looking closer at the Livy source code, it appears that the enum value
> "Entry.SPNEGO_ENABLED" needs to be set to false; however, I don't see a way
> to access the enum Entry object prior to the check on line 66:
>
>
>
>
> https://github.com/cloudera/livy/blob/9ae24d08738652ba5fd817780711d01b110d74a9/client-http/src/main/java/com/cloudera/livy/client/http/HttpConf.java#L66
>
>
>
>
> Any help would be greatly appreciated!
>
>
>