Multiple nifi.kerberos.krb5.file ("There Can Be Only One!)

[Michael Dyer <mi...@trapezoid.com>]

I'm trying to set up a single NiFi server that can connect to two HDFS
clusters, each with it's own Kerberos realm.

According to the NiFi docs:

"At this time, only a single krb5 file is allowed to be specified per NiFi
instance"

Is there a workaround that would allow me to connect to both clusters?

I've tried merging the two krb5.conf files, but I'm not able to get past
this error message (after disabling default_realm)

Caused by: java.lang.IllegalArgumentException: Illegal principal name
xxx@YYYY.ORG:
org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule:
No rules applied to xxx@YYYY.ORG

Re: Multiple nifi.kerberos.krb5.file ("There Can Be Only One!)

[Arpit Gupta <ar...@apache.org>]

Hi Michael

A single krb5.conf should work. Have you defined domain_realm that maps hostname patterns to realms? For example http://web.mit.edu/Kerberos/krb5-1.4/krb5-1.4.1/doc/krb5-admin/domain_realm.html <http://web.mit.edu/Kerberos/krb5-1.4/krb5-1.4.1/doc/krb5-admin/domain_realm.html>.

Also core-site.xml for each of these clusters probably has unique entries so you should make sure core-site.xml being used is for the appropriate cluster. Core site should have an entry for a property hadoop.security.auth_to_local that provides rules on how principal names are converted to short names. More info here http://hortonworks.com/blog/fine-tune-your-apache-hadoop-security-settings/ <http://hortonworks.com/blog/fine-tune-your-apache-hadoop-security-settings/>

--
Arpit

> On Jul 18, 2016, at 2:05 PM, Michael Dyer <mi...@trapezoid.com> wrote:
> 
> I'm trying to set up a single NiFi server that can connect to two HDFS clusters, each with it's own Kerberos realm.  
> 
> According to the NiFi docs:
> 
> "At this time, only a single krb5 file is allowed to be specified per NiFi instance"
> 
> Is there a workaround that would allow me to connect to both clusters?  
> 
> I've tried merging the two krb5.conf files, but I'm not able to get past this error message (after disabling default_realm)
> 
> Caused by: java.lang.IllegalArgumentException: Illegal principal name xxx@YYYY.ORG <ma...@YYYY.ORG>: org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No rules applied to xxx@YYYY.ORG <ma...@YYYY.ORG>
> 
> 

Re: Multiple nifi.kerberos.krb5.file ("There Can Be Only One!)

[Bryan Rosander <br...@gmail.com>]

Hey Michael,

Your best bet will probably be to use the site-to-site functionality of
NiFi in order to bridge the Kerberos (and potentially Hadoop distribution
version) gap.  Configure one instance of NiFi to talk to each cluster and
have them exchange data over site-to-site.

These instances of NiFi could probably still reside on the same machine or
vm if desired.

https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#site-to-site

Thanks,
Bryan

On Mon, Jul 18, 2016 at 5:05 PM, Michael Dyer <mi...@trapezoid.com>
wrote:

> I'm trying to set up a single NiFi server that can connect to two HDFS
> clusters, each with it's own Kerberos realm.
>
> According to the NiFi docs:
>
> "At this time, only a single krb5 file is allowed to be specified per NiFi
> instance"
>
> Is there a workaround that would allow me to connect to both clusters?
>
> I've tried merging the two krb5.conf files, but I'm not able to get past
> this error message (after disabling default_realm)
>
> Caused by: java.lang.IllegalArgumentException: Illegal principal name
> xxx@YYYY.ORG:
> org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule:
> No rules applied to xxx@YYYY.ORG
>
>
>