[jira] [Commented] (TEZ-3340) Add support for YARN Shared Cache

["Siddharth Seth (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15375802#comment-15375802 ] 

Siddharth Seth commented on TEZ-3340:
-------------------------------------

Tez does not upload the jars on it's own.
However, this is something that we want to change (not sure if a jira exists yet). Having to manually upload the jar to HDFS is an avoidable step; it does get rid of multiple copies of the same jar all over the dist cache, but there's other approaches to avoiding that.

> Add support for YARN Shared Cache
> ---------------------------------
>
>                 Key: TEZ-3340
>                 URL: https://issues.apache.org/jira/browse/TEZ-3340
>             Project: Apache Tez
>          Issue Type: Improvement
>            Reporter: Ming Ma
>
> YARN provides shared cache in functionality YARN-1492. According to [~ctrezzo] most of the YARN functionality is in hadoop 2.8 and frameworks can start to use it. MR adds the support via MAPREDUCE-5951.
> Can anyone confirm if Tez supports the upload of application DAG jar and dependent lib jars from client machine to HDFS as part of Tez app submission? From my test, that doesn't seem to happen. Instead Tez expects applications to upload the jars to HDFS beforehand and then set the tez.aux.uris to the HDFS locations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)