[jira] [Commented] (SLIDER-304) Handle HDFS token expiry in the AM

["ASF subversion and git services (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SLIDER-304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14109878#comment-14109878 ] 

ASF subversion and git services commented on SLIDER-304:
--------------------------------------------------------

Commit 0bd6c6cf93b098358c252355e2fbde7e87bd2fe7 in incubator-slider's branch refs/heads/feature/SLIDER-304_hdfs_token_renewal from [~jmaron]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-slider.git;h=0bd6c6c ]

SLIDER-304 initial token renewal code for review


> Handle HDFS token expiry in the AM
> ----------------------------------
>
>                 Key: SLIDER-304
>                 URL: https://issues.apache.org/jira/browse/SLIDER-304
>             Project: Slider
>          Issue Type: Bug
>          Components: appmaster, core
>            Reporter: Jonathan Maron
>            Assignee: Jonathan Maron
>
> For a long-running application, the application master should take care of renewing (or replacing) the HDFS delegation tokens to allow continuous operation.  There are a number of possible approaches:
> 1)  HBase-like - HBase appears to simply retrieve a delegation token for each FS interaction (UGI.getCurrent().getToken() or FS.getDelegationToken).  Given a secure cluster and an authenticated user initiated app this is a simple approach.  I am not certain about the cost of RPC token retrieval code for a remote FS - the AM based interactions are constrained to app initialization and some updates (i.e. there aren't many)
> 2)  Extend  the model in 1 by only replacing the token based on an expiry time.
> 3)  Launch a delegation token renewer/replacer thread/task.
> I currently would lean towards 1 or 2 with an eye towards implementing 3 if the need presented itself during testing etc.



--
This message was sent by Atlassian JIRA
(v6.2#6252)