[jira] [Commented] (TEZ-3362) Delete intermediate data at DAG level for Shuffle Handler

["Eric Badger (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15458757#comment-15458757 ] 

Eric Badger commented on TEZ-3362:
----------------------------------

Hi [~kshukla], thanks for the patch! I have just one comment before [~jeagles] gives his review.

{noformat}
+  public void dagDelete(DAG dag, JobTokenSecretManager jobTokenSecretManager)
+      throws IOException, InterruptedException {
+    for(NodeId nodeId : taskSchedulerManager.getPerDagNodeMap().get(dag.getName())) {
+      URL baseURL = ShuffleUtils.constructBaseURIForShuffleHandlerDagComplete(nodeId.getHost(), 13563,
{noformat}
Shouldn't we be using DEFAULT_SHUFFLE_PORT instead of the hardcoded value 13563?

> Delete intermediate data at DAG level for Shuffle Handler
> ---------------------------------------------------------
>
>                 Key: TEZ-3362
>                 URL: https://issues.apache.org/jira/browse/TEZ-3362
>             Project: Apache Tez
>          Issue Type: Sub-task
>            Reporter: Jonathan Eagles
>            Assignee: Kuhu Shukla
>         Attachments: TEZ-3362.001.patch
>
>
> Applications like hive that use tez in session mode need the ability to delete intermediate data after a DAG completes and while the application continues to run.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)