[jira] [Comment Edited] (OOZIE-3035) HDFS HA and log aggregation: getting HDFS delegation token from YARN renewer within JavaActionExecutor

["Peter Bacsko (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OOZIE-3035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16126491#comment-16126491 ] 

Peter Bacsko edited comment on OOZIE-3035 at 8/14/17 10:10 PM:
---------------------------------------------------------------

# I can see too many nested if-else. If it's possible, we should reduce those to the bare minimum. Is it necessary to check if {{credentials}} is empty? 
# If we need the token and don't have it, shouldn't we bail out and throw an exception? That would be much better than continuing execution and seeing a problem later on.
# Do we need a token in a non-secure cluster?


was (Author: pbacsko):
# I can see too many nested if-else. If it's possible, we should reduce those to the bare minimum. Is it necessary to check if {{credentials}} is empty? 
# If we need the token and don't have it, shouldn't we bail out and throw an exception? That would be much better than continuing execution and seeing a problem later on.
# Do we need a token in an non-secure cluster?

> HDFS HA and log aggregation: getting HDFS delegation token from YARN renewer within JavaActionExecutor
> ------------------------------------------------------------------------------------------------------
>
>                 Key: OOZIE-3035
>                 URL: https://issues.apache.org/jira/browse/OOZIE-3035
>             Project: Oozie
>          Issue Type: Bug
>    Affects Versions: 4.3.0
>         Environment: * [*Hadoop 3 alpha 4*|https://github.com/apache/hadoop/tree/branch-3.0.0-alpha4]
> * [*HDFS HA*|https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithNFS.html]
> * log aggregation turned on
>            Reporter: Andras Piros
>            Assignee: Andras Piros
>             Fix For: 5.0.0
>
>         Attachments: OOZIE-3035.001.patch
>
>
> In a secure environment, when both HDFS HA and log aggregation are turned on, {{JavaActionExecutor}} is not able to call {{YarnClient#submitApplication}} since {{HDFS_DELEGATION_TOKEN}} is missing.
> In those cases we need to get {{HDFS_DELEGATION_TOKEN}} from YARN:
> * get YARN renewer via {{Master#getMasterPrincipal}}
> * get {{HDFS_DELEGATION_TOKEN}} via {{DFSClient#getDelegationToken}}
> * add {{HDFS_DELEGATION_TOKEN}} to {{Credentials}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)