You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "Siddharth Seth (JIRA)" <ji...@apache.org> on 2019/04/26 19:24:00 UTC
[jira] [Updated] (TEZ-1348) Allow Tez local mode to run against
filesystems other than local FS
[ https://issues.apache.org/jira/browse/TEZ-1348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated TEZ-1348:
--------------------------------
Fix Version/s: (was: 0.10.0)
0.10.1
> Allow Tez local mode to run against filesystems other than local FS
> -------------------------------------------------------------------
>
> Key: TEZ-1348
> URL: https://issues.apache.org/jira/browse/TEZ-1348
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Siddharth Seth
> Assignee: Todd Lipcon
> Priority: Critical
> Fix For: 0.10.1
>
> Attachments: tez-1348.patch, tez-1348.patch, tez-1348.txt
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> In TEZ-717, I incorrect thought setting fs.defaultFS programmatically in tez-site would work for local mode.
> Currently the requirement is that tez-site.xml must have fs.defaultFS set to file:///.
> While that works, it doesn't allow for seamless execution in either local-mode or on a cluster.
> The main issue here is that when Inputs / Outputs are configured - they use a version of configuration which reads tez-site, and do not use the configuration from the client itself (which is correct behaviour).
> Not sure what a good way to fix this is
> 1) It may be possible to override this value each time an instance of Configuration/TezConfiguration is created. One possible way would be to statically add a default resource to Configuration the moment a local client is created.
> 2) Provide information in the contexts on whether this is local or not. This is fairly ugly, and would get in the way of running mixed mode tasks.
> Anyone have other suggestions ?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)