You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by "Peter Wicks (pwicks)" <pw...@micron.com> on 2019/04/30 16:21:14 UTC
Kicking the Kerberos out of org.apache.nifi.hadoop
I was thinking of moving the non-Hadoop specific Kerberos classes out of org.apache.nifi.hadoop into a more central location. There are some good utility classes, such as KerberosKeytabCredentials, that it would be nice to have centralized into a non-Hadoop specific library.
Would it be appropriate to co-locate these with the KerberosCredentialsService interface, or is there a better home in a different location?
Thanks,
Peter
Re: [EXT] Re: Kicking the Kerberos out of org.apache.nifi.hadoop
Posted by Bryan Bende <bb...@gmail.com>.
Ah I see, yea the stuff I have in nifi-security-utils currently
doesn't handle the http part, its more about how to get the
credentials and obtain a logged in user with in the current JVM.
The code in the Solr bundle and in hadoop-utils is handling the http
negotiation and we can probably refactor to make that reusable. That
code depends on org.apache.httpcomponents-httpclient so I don't think
we should put that in nifi-security-utils because it then brings in a
new dependency to everything using nifi-security-utils.
I would suggest under nifi-extension-utils, create a new module like
nifi-kerberos-http-utils, and move the relevant code from
nifi-hadoop-utils there, then make nifi-hadoop-utils depend on that,
and also make standard processors depend on that so you can use it in
InvokeHttp.
On Tue, Apr 30, 2019 at 1:04 PM Peter Wicks (pwicks) <pw...@micron.com> wrote:
>
> Thanks Bryan, I was reviewing this earlier. Some context on what I'm actually trying to do:
>
> I'm trying to add Kerberos support to InvokeHTTP, but didn't really want to add a dependency to the org.apache.nifi.hadoop library. The Livy controller service, which I was using as an example for this, uses two classes from org.apache.nifi.hadoop to build an Http Connection that uses Kerberos by referencing KerberosKeytabCredentials and KerberosKeytabSPNegoAuthSchemeProvider.
>
> I also looked at the Solr method, but couldn't understand how it worked... I just took a second look and realized that most of the magic happens in the onTrigger through the KerberosAction, and not in KerberosHttpClientConfigurer, which was a bit misleading, but now makes more sense.
>
> I'll take a shot at doing it this way, maybe I can move `KerberosHttpClientConfigurer` to nifi-security-utils?
>
> Thanks,
> Peter
>
>
> -----Original Message-----
> From: Bryan Bende <bb...@gmail.com>
> Sent: Tuesday, April 30, 2019 10:33 AM
> To: dev@nifi.apache.org
> Subject: [EXT] Re: Kicking the Kerberos out of org.apache.nifi.hadoop
>
> I created a mini-kerberos framework here in nifi-security-utils that does not depend on Hadoop:
>
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fnifi%2Ftree%2Fmaster%2Fnifi-commons%2Fnifi-security-utils%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fnifi%2Fsecurity%2Fkrb&data=02%7C01%7Cpwicks%40micron.com%7C68620b45804641b6970b08d6cd898a9e%7Cf38a5ecd28134862b11bac1d563c806f%7C0%7C0%7C636922387953451036&sdata=u35yNAbJ2J9Wv9MGc%2FkXX10ds%2BgyXFzdIEZpvd35IZU%3D&reserved=0
>
> The first use-case was for the Solr processors and I believe this code is also now used in the DBCPConnectionPool.
>
> We could consider moving code there if it makes sense, but maybe see if you can achieve what you need to do with what is there already.
>
> I think that other Hadoop spnego stuff was added around the same time as part of a different effort, but likely a bit of overlap.
>
> On Tue, Apr 30, 2019 at 12:21 PM Peter Wicks (pwicks) <pw...@micron.com> wrote:
> >
> > I was thinking of moving the non-Hadoop specific Kerberos classes out of org.apache.nifi.hadoop into a more central location. There are some good utility classes, such as KerberosKeytabCredentials, that it would be nice to have centralized into a non-Hadoop specific library.
> >
> > Would it be appropriate to co-locate these with the KerberosCredentialsService interface, or is there a better home in a different location?
> >
> > Thanks,
> > Peter
RE: [EXT] Re: Kicking the Kerberos out of org.apache.nifi.hadoop
Posted by "Peter Wicks (pwicks)" <pw...@micron.com>.
Thanks Bryan, I was reviewing this earlier. Some context on what I'm actually trying to do:
I'm trying to add Kerberos support to InvokeHTTP, but didn't really want to add a dependency to the org.apache.nifi.hadoop library. The Livy controller service, which I was using as an example for this, uses two classes from org.apache.nifi.hadoop to build an Http Connection that uses Kerberos by referencing KerberosKeytabCredentials and KerberosKeytabSPNegoAuthSchemeProvider.
I also looked at the Solr method, but couldn't understand how it worked... I just took a second look and realized that most of the magic happens in the onTrigger through the KerberosAction, and not in KerberosHttpClientConfigurer, which was a bit misleading, but now makes more sense.
I'll take a shot at doing it this way, maybe I can move `KerberosHttpClientConfigurer` to nifi-security-utils?
Thanks,
Peter
-----Original Message-----
From: Bryan Bende <bb...@gmail.com>
Sent: Tuesday, April 30, 2019 10:33 AM
To: dev@nifi.apache.org
Subject: [EXT] Re: Kicking the Kerberos out of org.apache.nifi.hadoop
I created a mini-kerberos framework here in nifi-security-utils that does not depend on Hadoop:
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fnifi%2Ftree%2Fmaster%2Fnifi-commons%2Fnifi-security-utils%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fnifi%2Fsecurity%2Fkrb&data=02%7C01%7Cpwicks%40micron.com%7C68620b45804641b6970b08d6cd898a9e%7Cf38a5ecd28134862b11bac1d563c806f%7C0%7C0%7C636922387953451036&sdata=u35yNAbJ2J9Wv9MGc%2FkXX10ds%2BgyXFzdIEZpvd35IZU%3D&reserved=0
The first use-case was for the Solr processors and I believe this code is also now used in the DBCPConnectionPool.
We could consider moving code there if it makes sense, but maybe see if you can achieve what you need to do with what is there already.
I think that other Hadoop spnego stuff was added around the same time as part of a different effort, but likely a bit of overlap.
On Tue, Apr 30, 2019 at 12:21 PM Peter Wicks (pwicks) <pw...@micron.com> wrote:
>
> I was thinking of moving the non-Hadoop specific Kerberos classes out of org.apache.nifi.hadoop into a more central location. There are some good utility classes, such as KerberosKeytabCredentials, that it would be nice to have centralized into a non-Hadoop specific library.
>
> Would it be appropriate to co-locate these with the KerberosCredentialsService interface, or is there a better home in a different location?
>
> Thanks,
> Peter
Re: Kicking the Kerberos out of org.apache.nifi.hadoop
Posted by Bryan Bende <bb...@gmail.com>.
I created a mini-kerberos framework here in nifi-security-utils that
does not depend on Hadoop:
https://github.com/apache/nifi/tree/master/nifi-commons/nifi-security-utils/src/main/java/org/apache/nifi/security/krb
The first use-case was for the Solr processors and I believe this code
is also now used in the DBCPConnectionPool.
We could consider moving code there if it makes sense, but maybe see
if you can achieve what you need to do with what is there already.
I think that other Hadoop spnego stuff was added around the same time
as part of a different effort, but likely a bit of overlap.
On Tue, Apr 30, 2019 at 12:21 PM Peter Wicks (pwicks) <pw...@micron.com> wrote:
>
> I was thinking of moving the non-Hadoop specific Kerberos classes out of org.apache.nifi.hadoop into a more central location. There are some good utility classes, such as KerberosKeytabCredentials, that it would be nice to have centralized into a non-Hadoop specific library.
>
> Would it be appropriate to co-locate these with the KerberosCredentialsService interface, or is there a better home in a different location?
>
> Thanks,
> Peter