[jira] [Comment Edited] (TEZ-1433) Invalid credentials can be used when a DAG is submitted to a session which has timed out

["Bikas Saha (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-1433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14144118#comment-14144118 ] 

Bikas Saha edited comment on TEZ-1433 at 9/23/14 12:44 AM:
-----------------------------------------------------------

This refactoring is not calling any extra stuff beyond the existing code. Its just using local object where it was earlier modifying the dag. for the tokens themselves, after the following call the HDFS tokens from the session get cached in the dagCredentials and after that TokenCache.obtainTokens...() will find HDFS token already present and will not fetch it again from the NN.

{code}+    Credentials dagCredentials = new Credentials();
     // All session creds are required for the DAG.
     dagCredentials.mergeAll(sessionCredentials);{code}

Please let me know if the new flow is not using the dagCredentials object everywhere (and thus missing the cache) or if there are any more comments. thanks!


was (Author: bikassaha):
This refactoring is not calling any extra stuff beyond the existing code. Its just using local object where it was earlier modifying the dag. for the tokens themselves, after the following call the HDFS tokens from the session get cached in the dagCredentials and after that TokenCache.obtainTokens...() will find HDFS token already present and will not fetch it again from the NN.

{code}+    Credentials dagCredentials = new Credentials();
     // All session creds are required for the DAG.
     dagCredentials.mergeAll(sessionCredentials);{code}

> Invalid credentials can be used when a DAG is submitted to a session which has timed out
> ----------------------------------------------------------------------------------------
>
>                 Key: TEZ-1433
>                 URL: https://issues.apache.org/jira/browse/TEZ-1433
>             Project: Apache Tez
>          Issue Type: Bug
>    Affects Versions: 0.4.0
>            Reporter: Siddharth Seth
>            Assignee: Bikas Saha
>         Attachments: TEZ-1433-v1.patch, TEZ-1433.2.patch, TEZ-1433.3.patch, TEZ-1433.4.patch
>
>
> When a DAG is submitted to a session which has timed out, and the same DAG is then submitted to a new session - credentials associated with the old session can end up getting used.
> Before we know that the session is no longer valid, the DAG is modified to add local resources and credentials.
> On the next submission, since the DAG already has tokens (for HDFS for example) from the old session, the tokens are not updated.
> Meanwhile, the old token would end up being cancelled by the RM - since the applicaiton associated with the previous session has finished.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)