[jira] [Commented] (TEZ-3894) Tez intermediate outputs implicitly rely on permissive umask for shuffle

["Tarek Abouzeid (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-3894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17163589#comment-17163589 ] 

Tarek Abouzeid commented on TEZ-3894:
-------------------------------------

Hi,

an update to this ticket, in Hortonworks HDP, the umask settings for TEZ was being fetched from the HDFS service umask setting where it was 077, changing it to 022 fixed the problem.

Best Regards, 

> Tez intermediate outputs implicitly rely on permissive umask for shuffle
> ------------------------------------------------------------------------
>
>                 Key: TEZ-3894
>                 URL: https://issues.apache.org/jira/browse/TEZ-3894
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Jason Darrell Lowe
>            Assignee: Jason Darrell Lowe
>            Priority: Major
>             Fix For: 0.9.2
>
>         Attachments: TEZ-3894.001.patch
>
>
> Tez does not explicitly set the permissions of intermediate output files for shuffle. In a secure cluster the shuffle service is running as a different user than the task, so the output files require group readability in order to serve up the data during the shuffle phase. If the umask is too restrictive (e.g.: 077) then the task's file.out and file.out.index permissions can be too restrictive to allow the shuffle handler to access them.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)