You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by James Srinivasan <ja...@gmail.com> on 2019/06/12 20:05:52 UTC
Kerberos Ticket Renewal (when not updating Hadoop user)
Hi all,
I'm finally getting around to fixing up some deprecation issues with
our use of Kerberos with Accumulo and GeoMesa
(https://github.com/locationtech/geomesa/). Because I didn't know any
better at the time, I used the KerberosToken ctor specifying that the
Hadoop user should be replaced. Combined with a thread to periodically
renew the ticket (calling
UserGroupInformation.getCurrentUser.checkTGTAndReloginFromKeytab()),
this has worked nicely for us.
However, there are some unfortunate side effects of updating the
Hadoop user - for instance, subsequent HDFS operations use the new
user, who may not have the same permissions as the original user in a
Zeppelin-type notebook environment. Plus the replaceCurrentUser param
is deprecated and removed in Accumulo 2.0. So I'm keen on not
replacing the Hadoop user, but how do I handle ticket renewal?
Thanks very much,
James
Re: Kerberos Ticket Renewal (when not updating Hadoop user)
Posted by James Srinivasan <ja...@gmail.com>.
Itym more things to worry about (eg race conditions)!
Will definitely be useful when I get round to PRs for Accumulo NiFi
processors.
On Thu, 13 Jun 2019, 19:36 Jeff, <jt...@gmail.com> wrote:
> James,
>
> No worries! At least you now have another point of reference. :)
>
> On Thu, Jun 13, 2019 at 1:11 PM James Srinivasan <
> james.srinivasan@gmail.com> wrote:
>
>> Err, my bad - meant to send this to the Accumulo list!
>>
>> Sorry!
>>
>> On Thu, 13 Jun 2019, 18:07 Jeff, <jt...@gmail.com> wrote:
>>
>>> Hello James,
>>>
>>> For our Hadoop processors, we generally don't do any explicit
>>> relogins/TGT renewal. It's handled implicitly by the Hadoop libs. PR 2360
>>> [1] is the primary change-set to allow this in NiFi, and several NiFI JIRAs
>>> (mainly NIFI-3472 [2]) are referenced in that pull request if you are
>>> interested in doing further reading.
>>>
>>> HiveConnectionPool (across various Hive versions) is the only component
>>> that comes to mind where we explicitly try to do a relogin. NIFI-5134 [3]
>>> contains more information on that.
>>>
>>> Hope this information helps!
>>>
>>> [1] https://github.com/apache/nifi/pull/2360
>>> [2] https://issues.apache.org/jira/browse/NIFI-3472
>>> [3] https://issues.apache.org/jira/browse/NIFI-5134
>>>
>>> On Wed, Jun 12, 2019 at 4:06 PM James Srinivasan <
>>> james.srinivasan@gmail.com> wrote:
>>>
>>>> Hi all,
>>>>
>>>> I'm finally getting around to fixing up some deprecation issues with
>>>> our use of Kerberos with Accumulo and GeoMesa
>>>> (https://github.com/locationtech/geomesa/). Because I didn't know any
>>>> better at the time, I used the KerberosToken ctor specifying that the
>>>> Hadoop user should be replaced. Combined with a thread to periodically
>>>> renew the ticket (calling
>>>> UserGroupInformation.getCurrentUser.checkTGTAndReloginFromKeytab()),
>>>> this has worked nicely for us.
>>>>
>>>> However, there are some unfortunate side effects of updating the
>>>> Hadoop user - for instance, subsequent HDFS operations use the new
>>>> user, who may not have the same permissions as the original user in a
>>>> Zeppelin-type notebook environment. Plus the replaceCurrentUser param
>>>> is deprecated and removed in Accumulo 2.0. So I'm keen on not
>>>> replacing the Hadoop user, but how do I handle ticket renewal?
>>>>
>>>> Thanks very much,
>>>>
>>>> James
>>>>
>>>
Re: Kerberos Ticket Renewal (when not updating Hadoop user)
Posted by Jeff <jt...@gmail.com>.
James,
No worries! At least you now have another point of reference. :)
On Thu, Jun 13, 2019 at 1:11 PM James Srinivasan <ja...@gmail.com>
wrote:
> Err, my bad - meant to send this to the Accumulo list!
>
> Sorry!
>
> On Thu, 13 Jun 2019, 18:07 Jeff, <jt...@gmail.com> wrote:
>
>> Hello James,
>>
>> For our Hadoop processors, we generally don't do any explicit
>> relogins/TGT renewal. It's handled implicitly by the Hadoop libs. PR 2360
>> [1] is the primary change-set to allow this in NiFi, and several NiFI JIRAs
>> (mainly NIFI-3472 [2]) are referenced in that pull request if you are
>> interested in doing further reading.
>>
>> HiveConnectionPool (across various Hive versions) is the only component
>> that comes to mind where we explicitly try to do a relogin. NIFI-5134 [3]
>> contains more information on that.
>>
>> Hope this information helps!
>>
>> [1] https://github.com/apache/nifi/pull/2360
>> [2] https://issues.apache.org/jira/browse/NIFI-3472
>> [3] https://issues.apache.org/jira/browse/NIFI-5134
>>
>> On Wed, Jun 12, 2019 at 4:06 PM James Srinivasan <
>> james.srinivasan@gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> I'm finally getting around to fixing up some deprecation issues with
>>> our use of Kerberos with Accumulo and GeoMesa
>>> (https://github.com/locationtech/geomesa/). Because I didn't know any
>>> better at the time, I used the KerberosToken ctor specifying that the
>>> Hadoop user should be replaced. Combined with a thread to periodically
>>> renew the ticket (calling
>>> UserGroupInformation.getCurrentUser.checkTGTAndReloginFromKeytab()),
>>> this has worked nicely for us.
>>>
>>> However, there are some unfortunate side effects of updating the
>>> Hadoop user - for instance, subsequent HDFS operations use the new
>>> user, who may not have the same permissions as the original user in a
>>> Zeppelin-type notebook environment. Plus the replaceCurrentUser param
>>> is deprecated and removed in Accumulo 2.0. So I'm keen on not
>>> replacing the Hadoop user, but how do I handle ticket renewal?
>>>
>>> Thanks very much,
>>>
>>> James
>>>
>>
Re: Kerberos Ticket Renewal (when not updating Hadoop user)
Posted by James Srinivasan <ja...@gmail.com>.
Err, my bad - meant to send this to the Accumulo list!
Sorry!
On Thu, 13 Jun 2019, 18:07 Jeff, <jt...@gmail.com> wrote:
> Hello James,
>
> For our Hadoop processors, we generally don't do any explicit relogins/TGT
> renewal. It's handled implicitly by the Hadoop libs. PR 2360 [1] is the
> primary change-set to allow this in NiFi, and several NiFI JIRAs (mainly
> NIFI-3472 [2]) are referenced in that pull request if you are interested in
> doing further reading.
>
> HiveConnectionPool (across various Hive versions) is the only component
> that comes to mind where we explicitly try to do a relogin. NIFI-5134 [3]
> contains more information on that.
>
> Hope this information helps!
>
> [1] https://github.com/apache/nifi/pull/2360
> [2] https://issues.apache.org/jira/browse/NIFI-3472
> [3] https://issues.apache.org/jira/browse/NIFI-5134
>
> On Wed, Jun 12, 2019 at 4:06 PM James Srinivasan <
> james.srinivasan@gmail.com> wrote:
>
>> Hi all,
>>
>> I'm finally getting around to fixing up some deprecation issues with
>> our use of Kerberos with Accumulo and GeoMesa
>> (https://github.com/locationtech/geomesa/). Because I didn't know any
>> better at the time, I used the KerberosToken ctor specifying that the
>> Hadoop user should be replaced. Combined with a thread to periodically
>> renew the ticket (calling
>> UserGroupInformation.getCurrentUser.checkTGTAndReloginFromKeytab()),
>> this has worked nicely for us.
>>
>> However, there are some unfortunate side effects of updating the
>> Hadoop user - for instance, subsequent HDFS operations use the new
>> user, who may not have the same permissions as the original user in a
>> Zeppelin-type notebook environment. Plus the replaceCurrentUser param
>> is deprecated and removed in Accumulo 2.0. So I'm keen on not
>> replacing the Hadoop user, but how do I handle ticket renewal?
>>
>> Thanks very much,
>>
>> James
>>
>
Re: Kerberos Ticket Renewal (when not updating Hadoop user)
Posted by Jeff <jt...@gmail.com>.
Hello James,
For our Hadoop processors, we generally don't do any explicit relogins/TGT
renewal. It's handled implicitly by the Hadoop libs. PR 2360 [1] is the
primary change-set to allow this in NiFi, and several NiFI JIRAs (mainly
NIFI-3472 [2]) are referenced in that pull request if you are interested in
doing further reading.
HiveConnectionPool (across various Hive versions) is the only component
that comes to mind where we explicitly try to do a relogin. NIFI-5134 [3]
contains more information on that.
Hope this information helps!
[1] https://github.com/apache/nifi/pull/2360
[2] https://issues.apache.org/jira/browse/NIFI-3472
[3] https://issues.apache.org/jira/browse/NIFI-5134
On Wed, Jun 12, 2019 at 4:06 PM James Srinivasan <ja...@gmail.com>
wrote:
> Hi all,
>
> I'm finally getting around to fixing up some deprecation issues with
> our use of Kerberos with Accumulo and GeoMesa
> (https://github.com/locationtech/geomesa/). Because I didn't know any
> better at the time, I used the KerberosToken ctor specifying that the
> Hadoop user should be replaced. Combined with a thread to periodically
> renew the ticket (calling
> UserGroupInformation.getCurrentUser.checkTGTAndReloginFromKeytab()),
> this has worked nicely for us.
>
> However, there are some unfortunate side effects of updating the
> Hadoop user - for instance, subsequent HDFS operations use the new
> user, who may not have the same permissions as the original user in a
> Zeppelin-type notebook environment. Plus the replaceCurrentUser param
> is deprecated and removed in Accumulo 2.0. So I'm keen on not
> replacing the Hadoop user, but how do I handle ticket renewal?
>
> Thanks very much,
>
> James
>