[jira] [Commented] (TEZ-1348) Setup configs required for local mode automatically, instead of relying on changes to tez-site

["Siddharth Seth (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-1348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14275767#comment-14275767 ] 

Siddharth Seth commented on TEZ-1348:
-------------------------------------

Some of the properties which are typically setup for local mode.
tez.runtime.optimize.local.fetch
fs.defaultFS
tez.local.mode
dag.scheduler.class

> Setup configs required for local mode automatically, instead of relying on changes to tez-site
> ----------------------------------------------------------------------------------------------
>
>                 Key: TEZ-1348
>                 URL: https://issues.apache.org/jira/browse/TEZ-1348
>             Project: Apache Tez
>          Issue Type: Sub-task
>            Reporter: Siddharth Seth
>            Priority: Critical
>
> In TEZ-717, I incorrect thought setting fs.defaultFS programmatically in tez-site would work for local mode.
> Currently the requirement is that tez-site.xml must have fs.defaultFS set to file:///.
> While that works, it doesn't allow for seamless execution in either local-mode or on a cluster.
> The main issue here is that when Inputs / Outputs are configured - they use a version of configuration which reads tez-site, and do not use the configuration from the client itself (which is correct behaviour).
> Not sure what a good way to fix this is 
> 1) It may be possible to override this value each time an instance of Configuration/TezConfiguration is created. One possible way would be to statically add a default resource to Configuration the moment a local client is created.
> 2) Provide information in the contexts on whether this is local or not. This is fairly ugly, and would get in the way of running mixed mode tasks.
> Anyone have other suggestions ?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)