You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2022/12/24 13:35:00 UTC
[jira] [Work logged] (HIVE-26887) Make sure dirPath has the correct permissions
[ https://issues.apache.org/jira/browse/HIVE-26887?focusedWorklogId=835576&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-835576 ]
ASF GitHub Bot logged work on HIVE-26887:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 24/Dec/22 13:34
Start Date: 24/Dec/22 13:34
Worklog Time Spent: 10m
Work Description: skysiders opened a new pull request, #3894:
URL: https://github.com/apache/hive/pull/3894
In the QueryResultsCache function of class QueryResultsCache, there is the following code segment
```java
private QueryResultsCache(HiveConf configuration) throws IOException {
......
FileSystem fs = cacheDirPath.getFileSystem(conf);
FsPermission fsPermission = new FsPermission("700");
fs.mkdirs(cacheDirPath, fsPermission);
......
}
```
It can be seen that the function will use the mkdirs to create cacheDirPath, and the parameters passed in include the path variable cacheDirPath and a permission 700. But we haven't confirmed whether the permission is correctly assigned to the file.
The above question is raised because there are two mkdir functions of hadoop,`mkdirs(Path f, FsPermission permission)` with `mkdirs(FileSystem fs, Path dir, FsPermission permission)`and the first one is used here. The permissions of this function will be affected by the underlying umask. Although 700 here will hardly be affected by umask, but I think from a rigorous point of view, we should have one more permission check and permission grant here. So I judge whether the file permission is correctly granted by detecting the influence of umask on the permission, and if not, give it the correct permission through setPermission.
```java
if (!permission.equals(permission.applyUMask(FsPermission.getUMask(conf)))) {
fs.setPermission(dir, permission);
}
```
And I find same issue in other three methods here.
In class Context
```java
private Path getScratchDir(String scheme, String authority,
boolean mkdir, String scratchDir) {
......
FileSystem fs = dirPath.getFileSystem(conf);
dirPath = new Path(fs.makeQualified(dirPath).toString());
FsPermission fsPermission = new FsPermission(scratchDirPermission);
if (!fs.mkdirs(dirPath, fsPermission)) {
throw new RuntimeException("Cannot make directory: "
+ dirPath.toString());
......
}
```
In class SessionState
```java
static void createPath(HiveConf conf, Path path, String permission, boolean isLocal,
boolean isCleanUp) throws IOException {
FsPermission fsPermission = new FsPermission(permission);
FileSystem fs;
......
if (!fs.mkdirs(path, fsPermission)) {
throw new IOException("Failed to create directory " + path + " on fs " + fs.getUri());
}
......
}
```
and in class TezSessionState
```java
private Path createTezDir(String sessionId, String suffix) throws IOException {
......
Path tezDir = new Path(hdfsScratchDir, TEZ_DIR);
FileSystem fs = tezDir.getFileSystem(conf);
FsPermission fsPermission = new FsPermission(HiveConf.getVar(conf, HiveConf.ConfVars.SCRATCHDIRPERMISSION));
fs.mkdirs(tezDir, fsPermission);
......
}
```
Issue Time Tracking
-------------------
Worklog Id: (was: 835576)
Remaining Estimate: 0h
Time Spent: 10m
> Make sure dirPath has the correct permissions
> ---------------------------------------------
>
> Key: HIVE-26887
> URL: https://issues.apache.org/jira/browse/HIVE-26887
> Project: Hive
> Issue Type: Improvement
> Reporter: Zhang Dongsheng
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In the QueryResultsCache function of class QueryResultsCache, there is the following code segment
> {code:java}
> private QueryResultsCache(HiveConf configuration) throws IOException {
> ......
> FileSystem fs = cacheDirPath.getFileSystem(conf);
> FsPermission fsPermission = new FsPermission("700");
> fs.mkdirs(cacheDirPath, fsPermission);
> ......
> }
> {code}
> It can be seen that the function will use the mkdirs to create cacheDirPath, and the parameters passed in include the path variable cacheDirPath and a permission 700. But we haven't confirmed whether the permission is correctly assigned to the file.
> The above question is raised because there are two mkdir functions of hadoop,
> {code:java}
> mkdirs(Path f, FsPermission permission)
> {code}
> and
> {code:java}
> mkdirs(FileSystem fs, Path dir, FsPermission permission)
> {code}
> and the first one is used here. The permissions of this function will be affected by the underlying umask. Although 700 here will hardly be affected by umask, but I think from a rigorous point of view, we should have one more permission check and permission grant here.
> And I find same issue in other three methods here.
> In class Context
> {code:java}
> private Path getScratchDir(String scheme, String authority,
> boolean mkdir, String scratchDir) {
> ......
> FileSystem fs = dirPath.getFileSystem(conf);
> dirPath = new Path(fs.makeQualified(dirPath).toString());
> FsPermission fsPermission = new FsPermission(scratchDirPermission);
> if (!fs.mkdirs(dirPath, fsPermission)) {
> throw new RuntimeException("Cannot make directory: "
> + dirPath.toString());
> ......
> }
> {code}
> In class SessionState
> {code:java}
> static void createPath(HiveConf conf, Path path, String permission, boolean isLocal,
> boolean isCleanUp) throws IOException {
> FsPermission fsPermission = new FsPermission(permission);
> FileSystem fs;
> ......
> if (!fs.mkdirs(path, fsPermission)) {
> throw new IOException("Failed to create directory " + path + " on fs " + fs.getUri());
> }
> ......
> }
> {code}
> and in class TezSessionState
> {code:java}
> private Path createTezDir(String sessionId, String suffix) throws IOException {
> ......
> Path tezDir = new Path(hdfsScratchDir, TEZ_DIR);
> FileSystem fs = tezDir.getFileSystem(conf);
> FsPermission fsPermission = new FsPermission(HiveConf.getVar(conf, HiveConf.ConfVars.SCRATCHDIRPERMISSION));
> fs.mkdirs(tezDir, fsPermission);
> ......
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)