[jira] [Commented] (TEZ-3408) Allow Task Output Files to reside in DAG specific directories for Custom Shuffle Handler

["Jonathan Eagles (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-3408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419415#comment-15419415 ] 

Jonathan Eagles commented on TEZ-3408:
--------------------------------------

Unused import
{code:title=TestShuffleHandler.java}
import org.apache.tez.runtime.library.common.shuffle.ShuffleUtils;
{code}

Since SimpleFetchedInputAllocator is a private class. Just modify the constructor to always take a dagId. Then you can removed the constructor helper getSimpleFetchedInputAllocator and keep the logic from polluting UnorderedKVInput

TODO comment addressed (file a jira) or removed
{code:title=TezTaskOutputFiles}
//TODO : Add dag based logic
{code}

Comments in TezTaskOutputFiles don't reflect the new
{code:title=TezTaskOutputFiles}
  /*
   * ${appDir}/output/${uniqueId}
   */
  private Path getAttemptOutputDir() {
{code}

> Allow Task Output Files to reside in DAG specific directories for Custom Shuffle Handler
> ----------------------------------------------------------------------------------------
>
>                 Key: TEZ-3408
>                 URL: https://issues.apache.org/jira/browse/TEZ-3408
>             Project: Apache Tez
>          Issue Type: Sub-task
>            Reporter: Kuhu Shukla
>            Assignee: Kuhu Shukla
>         Attachments: TEZ-3408.001.patch, TEZ-3408.002.patch
>
>
> When configured to use Custom Tez Shuffle Handler, the task output files can be in DAG specific output directories,for example: \{appDir\}/dag_123/output/$\{uniqueId\}/file.out
> This is one step towards multiple files feature.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)