You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tez.apache.org by "Jacob Tolar (JIRA)" <ji...@apache.org> on 2017/10/02 22:23:01 UTC
[jira] [Created] (TEZ-3848) Tez Local mode doesn't localize
distributed cache files
Jacob Tolar created TEZ-3848:
--------------------------------
Summary: Tez Local mode doesn't localize distributed cache files
Key: TEZ-3848
URL: https://issues.apache.org/jira/browse/TEZ-3848
Project: Apache Tez
Issue Type: Bug
Reporter: Jacob Tolar
Tez doesn't symlink LocalResources into place in LocalContainerLauncher.
In YARN mode, Yarn takes care of this when it launches the container. But in local mode, if you're depending on a file existing in the distributed cache, it's never symlinked into place (so you're out of luck).
We test our pig scripts in local mode and have some tools to set up the distributed cache the same way it would work in production. This works fine in Mapreduce mode but are unable to use Pig + Tez local mode for testing due to this problem.
I have a fix working and will submit a PR once I rebase it.
[~jeagles] [~wlauer@yahoo-inc.com]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)