[jira] [Updated] (TEZ-1534) Make client side configs available to AM and tasks

["Siddharth Seth (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TEZ-1534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Siddharth Seth updated TEZ-1534:
--------------------------------
    Attachment: TEZ-1534.1.txt

Patch allows additional parameters to be filtered through - selective list from properties used in hadoop-common, hdfs and yarn.
There's a unstable, semi-private conf property which can be used to change this if something is missed out.

Based on an offline discussion with [~hitesh] - not removing HADOOP_CLASSPATH from the AM/task classpath. If these configs exist, they'll be loaded and overwritten by values from the client.
The main reason is in case new Configuration() instances are created by hadoop-code - they'll have some configuration to work with.

[~hitesh] - please review.

> Make client side configs available to AM and tasks
> --------------------------------------------------
>
>                 Key: TEZ-1534
>                 URL: https://issues.apache.org/jira/browse/TEZ-1534
>             Project: Apache Tez
>          Issue Type: Improvement
>            Reporter: Siddharth Seth
>            Assignee: Siddharth Seth
>         Attachments: TEZ-1534.1.txt
>
>
> Configs from the client (specifically the ones provided to TezClient, along with YARN additions) should be shipped over to the cluster (AM and tasks), instead of AM/tasks depending on configs present on cluster nodes.
> These configs will primarily be used for Tez components like RPC servers, clients etc - and not by the Processor / Input / Output - which should be sending over fully configured payloads in any case.
> Tez should continue to run without core-site, hdfs-site, yarn-site etc in the classpath on cluster nodes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)