You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Ted Yu (JIRA)" <ji...@apache.org> on 2017/03/01 03:17:45 UTC
[jira] [Updated] (HBASE-17710) HBase in standalone mode creates
directories with 777 permission
[ https://issues.apache.org/jira/browse/HBASE-17710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-17710:
---------------------------
Status: Patch Available (was: Open)
> HBase in standalone mode creates directories with 777 permission
> ----------------------------------------------------------------
>
> Key: HBASE-17710
> URL: https://issues.apache.org/jira/browse/HBASE-17710
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 1.1.2
> Environment: HDP-2.5.3
> Reporter: Toshihiro Suzuki
> Assignee: Ted Yu
> Attachments: 17710.v1.txt
>
>
> HBase in standalone mode creates directories with 777 permission in hbase.rootdir.
> Ambari metrics collector defaults to standalone mode.
> {code}
> # find /var/lib/ambari-metrics-collector/hbase -perm 777 -type d -exec ls -ld {} \;
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17 /var/lib/ambari-metrics-collector/hbase/data/hbase/namespace/d0cca53847904f4b4add1caa0ce3a9af/info
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17 /var/lib/ambari-metrics-collector/hbase/data/hbase/backup/cbceb8fccd968b4b4583365d4dc6e377/meta
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17 /var/lib/ambari-metrics-collector/hbase/data/hbase/backup/cbceb8fccd968b4b4583365d4dc6e377/session
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/SYSTEM.CATALOG/2f4ce2294cd21cecb58fd1aca5646144/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/SYSTEM.SEQUENCE/0eb67274ece8a4a26cfeeef2c6d4cd37/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/SYSTEM.SEQUENCE/aef86710a4005f98e2dc90675f2eb325/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/SYSTEM.STATS/5b1d955e255e55979621214a7e4083b8/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/SYSTEM.FUNCTION/32c033735cf144bac5637de23f7f7dd0/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/METRICS_METADATA/e420dfa799742fe4516ad1e4deefb793/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/HOSTED_APPS_METADATA/110be63e2a9994121fc5b48d663daf2c/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/CONTAINER_METRICS/a103719f87e8430635abf51a7fe98637/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/cdb1d032beb90e350ce309e5d383c78e/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/294deab47187494e845a5199702b4d04/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/1a263b4fe068ef2db5ba1c3e45553354/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/48f94dfb0161d8a28f645d2e1a473235/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_HOURLY/6d096ac3e70e54dd4a8612e17cfc4b11/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_DAILY/e81850d62da64c8d1c67be309f136e23/0
> drwxrwxrwx. 2 ams hadoop 45 Mar 1 02:21 /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_AGGREGATE/b43ff796de887197834ad62fdb612b59/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:21 /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_AGGREGATE/b43ff796de887197834ad62fdb612b59/.tmp
> drwxrwxrwx. 2 ams hadoop 45 Mar 1 02:21 /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_AGGREGATE/c8eadeb7dead8fda9729b8e9b10c4929/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:21 /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_AGGREGATE/c8eadeb7dead8fda9729b8e9b10c4929/.tmp
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_AGGREGATE_MINUTE/ca9f9754ae9ae4cdc3e1b0523eecc390/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:18 /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_AGGREGATE_HOURLY/8412e8a8aec5d6307943fac78ce14c7a/0
> drwxrwxrwx. 2 ams hadoop 6 Mar 1 02:18 /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_AGGREGATE_DAILY/7c3358aba91ea0d76ddd8bc3ceb2d578/0
> {code}
> My analysis is as follows:
> FileSystem.mkdirs(Path f) method creates a directory with permission 777. Because HFileSystem which inherits FileSystem doesn't override the method, when we call HFileSystem.mkdirs(Path f), it tries to create a directory with permission 777.
> I've found that HFileSystem.mkdirs(Path f) gets called in the following areas:
> https://github.com/hortonworks/hbase/blob/HDP-2.5.3.0-37-tag/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java#L686
> https://github.com/hortonworks/hbase/blob/HDP-2.5.3.0-37-tag/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionFileSystem.java#L968
> At these times, the call path is as follows ("fs" is an instance of the abstract FileSystem class, being a field / member variable of FilterFileSystem):
> HFileSystem.mkdirs(Path f) -> FileSystem.mkdirs(Path f) -> FilterFileSystem.mkdirs(Path f, FsPermission permission) -> fs.mkdirs(Path f, FsPermission permission)
> For example:
> * In HRegionFileSystem.createDir(Path dir), fs.mkdirs(dir) is called.
> * Here "fs" is an instance of HFileSystem.
> * HFileSystem who inherits from FileSystem does not have an overridden implementation of mkdirs(dir).
> * Therefore, the parent FileSystem.mkdirs(Path f) is called.
> * FileSystem.mkdirs(Path f) calls FileSystem.mkdirs(f, FsPermission.getDirDefault()).
> ** FsPermission.getDirDefault() returns a default permission of 777 via this line:
> *** return new FsPermission((short)00777);
> * FileSystem.mkdirs(f, FsPermission.getDirDefault()) is abstract, so FilterFileSystem.mkdirs(Path f, FsPermission permission) gets called.
> * FilterFileSystem.mkdirs(Path f, FsPermission permission) calls fs.mkdirs(f, permission).
> * "fs" at this point can be either RawLocalFileSystem or DistributedFileSystem depending on whether we are in standalone mode or distributed mode.
> RawLocalFileSystem.mkdirs creates directories with permission 777. On the other hand, DistributedFileSystem.mkdirs applies the umask to the 777 permission (https://hadoop.apache.org/docs/r2.7.3/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html#Changes_to_the_File_System_API). By default, the umask (fs.permissions.umask-mode) is 0022, so the permission of the directory becomes 755 after application of the 0022 umask.
> In distributed mode directories with 777 are not created in HDFS, but in standalone mode directories with 777 are created.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)