You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Mithun Radhakrishnan (JIRA)" <ji...@apache.org> on 2017/10/12 20:47:00 UTC
[jira] [Updated] (HIVE-17791) Temp dirs under the staging directory
should honour `inheritPerms`
[ https://issues.apache.org/jira/browse/HIVE-17791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mithun Radhakrishnan updated HIVE-17791:
----------------------------------------
Attachment: HIVE-17791.1.patch
> Temp dirs under the staging directory should honour `inheritPerms`
> ------------------------------------------------------------------
>
> Key: HIVE-17791
> URL: https://issues.apache.org/jira/browse/HIVE-17791
> Project: Hive
> Issue Type: Bug
> Components: Authorization
> Reporter: Mithun Radhakrishnan
> Assignee: Chris Drome
> Attachments: HIVE-17791.1.patch
>
>
> For [~cdrome]:
> CLI creates two levels of staging directories but calls setPermissions on the top-level directory only if {{hive.warehouse.subdir.inherit.perms=true}}.
> The top-level directory, {{/user/cdrome/hive/words_text_dist/dt=c/.hive-staging_hive_2016-07-15_08-44-22_082_5534649671389063929-1}} is created the first time {{Context.getExternalTmpPath}} is called.
> The child directory, {{/user/cdrome/hive/words_text_dist/dt=c/.hive-staging_hive_2016-07-15_08-44-22_082_5534649671389063929-1/_tmp.-ext-10000}} is created when {{TezTask.execute}} is called at line 164:
> {code:java}
> DAG dag = build(jobConf, work, scratchDir, appJarLr, additionalLr, ctx);
> {code}
> This calls {{DagUtils.createVertex}}, which calls {{Utilities.createTmpDirs}}:
> {code:java}
> 3770 private static void createTmpDirs(Configuration conf,
> 3771 List<Operator<? extends OperatorDesc>> ops) throws IOException {
> 3772
> 3773 while (!ops.isEmpty()) {
> 3774 Operator<? extends OperatorDesc> op = ops.remove(0);
> 3775
> 3776 if (op instanceof FileSinkOperator) {
> 3777 FileSinkDesc fdesc = ((FileSinkOperator) op).getConf();
> 3778 Path tempDir = fdesc.getDirName();
> 3779
> 3780 if (tempDir != null) {
> 3781 Path tempPath = Utilities.toTempPath(tempDir);
> 3782 FileSystem fs = tempPath.getFileSystem(conf);
> 3783 fs.mkdirs(tempPath); // <------ HERE!
> 3784 }
> 3785 }
> 3786
> 3787 if (op.getChildOperators() != null) {
> 3788 ops.addAll(op.getChildOperators());
> 3789 }
> 3790 }
> 3791 }
> {code}
> It turns out that {{inheritPerms}} is no longer part of {{master}}. I'll rebase this for {{branch-2}}, and {{branch-2.2}}. {{master}} will have to wait till the issues around {{StorageBasedAuthProvider}}, directory permissions, etc. are sorted out.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)