[jira] [Created] (SLIDER-304) Handle HDFS token expiry in the AM

["Jonathan Maron (JIRA)" <ji...@apache.org>]

Jonathan Maron created SLIDER-304:
-------------------------------------

             Summary: Handle HDFS token expiry in the AM
                 Key: SLIDER-304
                 URL: https://issues.apache.org/jira/browse/SLIDER-304
             Project: Slider
          Issue Type: Bug
          Components: appmaster, core
            Reporter: Jonathan Maron
            Assignee: Jonathan Maron


For a long-running application, the application master should take care of renewing (or replacing) the HDFS delegation tokens to allow continuous operation.  There are a number of possible approaches:

1)  HBase-like - HBase appears to simply retrieve a delegation token for each FS interaction (UGI.getCurrent().getToken() or FS.getDelegationToken).  Given a secure cluster and an authenticated user initiated app this is a simple approach.  I am not certain about the cost of RPC token retrieval code for a remote FS - the AM based interactions are constrained to app initialization and some updates (i.e. there aren't many)

2)  Extend  the model in 1 by only replacing the token based on an expiry time.

3)  Launch a delegation token renewer/replacer thread/task.

I currently would lean towards 1 or 2 with an eye towards implementing 3 if the need presented itself during testing etc.



--
This message was sent by Atlassian JIRA
(v6.2#6252)