You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@hbase.apache.org by "Nick Dimiduk (JIRA)" <ji...@apache.org> on 2016/03/31 17:38:25 UTC

[jira] [Commented] (HBASE-15570) renewable delegation tokens for long-lived spark applications

    [ https://issues.apache.org/jira/browse/HBASE-15570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15220050#comment-15220050 ] 

Nick Dimiduk commented on HBASE-15570:
--------------------------------------

Is this specific to the Spark integration? Does it apply to OutputFormat implementations as well -- ie, Spark/Flink Streaming writing through OutputFormat.

> renewable delegation tokens for long-lived spark applications
> -------------------------------------------------------------
>
>                 Key: HBASE-15570
>                 URL: https://issues.apache.org/jira/browse/HBASE-15570
>             Project: HBase
>          Issue Type: Improvement
>          Components: spark
>            Reporter: Sean Busbey
>            Assignee: Sean Busbey
>
> Right now our spark integration works on secure clusters by getting delegation tokens and sending them to the executors. Unfortunately, applications that need to run for longer than the delegation token lifetime (by default 7 days) will fail.
> In particular, this is an issue for Spark Streaming applications. Since they expect to run indefinitely, we should have a means for renewing the delegation tokens.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)