You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Namit Jain (JIRA)" <ji...@apache.org> on 2010/09/23 23:04:32 UTC
[jira] Created: (HIVE-1667) Store the group of the owner of the
table in metastore
Store the group of the owner of the table in metastore
------------------------------------------------------
Key: HIVE-1667
URL: https://issues.apache.org/jira/browse/HIVE-1667
Project: Hadoop Hive
Issue Type: New Feature
Reporter: Namit Jain
Currently, the group of the owner of the table is not stored in the metastore.
Secondly, if you create a table, the table's owner group is set to the group for the parent. It is not read from the UGI passed in.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1667) Store the group of the owner of the
table in metastore
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-1667:
-----------------------------------
Attachment: hive-1667.patch
I am interested in this issue, specifically the second part of it and curious to know how its gonna be implemented. I am uploading a diff file illustrating my thinking on it and couple of open questions for it. Would be great if some one can take a look and comment on it.
> Store the group of the owner of the table in metastore
> ------------------------------------------------------
>
> Key: HIVE-1667
> URL: https://issues.apache.org/jira/browse/HIVE-1667
> Project: Hadoop Hive
> Issue Type: New Feature
> Reporter: Namit Jain
> Attachments: hive-1667.patch
>
>
> Currently, the group of the owner of the table is not stored in the metastore.
> Secondly, if you create a table, the table's owner group is set to the group for the parent. It is not read from the UGI passed in.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1667) Store the group of the owner of the
table in metastore
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916734#action_12916734 ]
Namit Jain commented on HIVE-1667:
----------------------------------
There may be a superuser, say root, who is the owner of the parent dir (which will be warehouse dir).
If I create a table T, the group should be my group, not root. - The current BSD semantics force the group to be root
> Store the group of the owner of the table in metastore
> ------------------------------------------------------
>
> Key: HIVE-1667
> URL: https://issues.apache.org/jira/browse/HIVE-1667
> Project: Hadoop Hive
> Issue Type: New Feature
> Reporter: Namit Jain
> Attachments: hive-1667.patch
>
>
> Currently, the group of the owner of the table is not stored in the metastore.
> Secondly, if you create a table, the table's owner group is set to the group for the parent. It is not read from the UGI passed in.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1667) Store the group of the owner of the
table in metastore
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916649#action_12916649 ]
Ashutosh Chauhan commented on HIVE-1667:
----------------------------------------
I am having second thought on this. What exactly is the usecase for storing group name? HDFS follows BSD semantics for group resolution which means that when table is created in Hive which results in mkdirs() call on HDFS, table dir will be created with the group as group of parent dir (which will be warehouse dir) Now its possible that user creating the table is not a part of parent dir's group. By storing primary group of user in table's metadata in metastore, you now are in inconsistent state. You have stored group name of the table as user's primary group while on the other hand group of the dir on HDFS is different and is the group of parent's dir.
In my opinion to avoid such divergences, HDFS should be considered as primary source of truth w.r.t, permissions and group on both logical table as well as physical dirs.
Whenever user asks Hive to set permissions or group we should directly try to do chmod and chgrp on physical dirs and whenever Hive need user and group information it should query filesystem. Thoughts?
> Store the group of the owner of the table in metastore
> ------------------------------------------------------
>
> Key: HIVE-1667
> URL: https://issues.apache.org/jira/browse/HIVE-1667
> Project: Hadoop Hive
> Issue Type: New Feature
> Reporter: Namit Jain
> Attachments: hive-1667.patch
>
>
> Currently, the group of the owner of the table is not stored in the metastore.
> Secondly, if you create a table, the table's owner group is set to the group for the parent. It is not read from the UGI passed in.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1667) Store the group of the owner of the
table in metastore
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12915516#action_12915516 ]
Namit Jain commented on HIVE-1667:
----------------------------------
Is it true that the first group is always the primary group (like Unix) ?
If org.apache.hadoop.security.UserGroupInformation guarantees that, your approach seems correct.
One more thing to check is what happens in case of creation of a partition via insert overwrite ?
The temp. folder is moved to the final folder, so you may have to fix there also.
> Store the group of the owner of the table in metastore
> ------------------------------------------------------
>
> Key: HIVE-1667
> URL: https://issues.apache.org/jira/browse/HIVE-1667
> Project: Hadoop Hive
> Issue Type: New Feature
> Reporter: Namit Jain
> Attachments: hive-1667.patch
>
>
> Currently, the group of the owner of the table is not stored in the metastore.
> Secondly, if you create a table, the table's owner group is set to the group for the parent. It is not read from the UGI passed in.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1667) Store the group of the owner of the
table in metastore
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12919351#action_12919351 ]
Namit Jain commented on HIVE-1667:
----------------------------------
Not in CREATE TABLE, but Hive should let the user specify the group via some configuration parameter later
> Store the group of the owner of the table in metastore
> ------------------------------------------------------
>
> Key: HIVE-1667
> URL: https://issues.apache.org/jira/browse/HIVE-1667
> Project: Hadoop Hive
> Issue Type: New Feature
> Reporter: Namit Jain
> Attachments: hive-1667.patch
>
>
> Currently, the group of the owner of the table is not stored in the metastore.
> Secondly, if you create a table, the table's owner group is set to the group for the parent. It is not read from the UGI passed in.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1667) Store the group of the owner of the
table in metastore
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12919353#action_12919353 ]
Ashutosh Chauhan commented on HIVE-1667:
----------------------------------------
And similarly for permissions? Let user specify permissions for her table through some config parameter.
> Store the group of the owner of the table in metastore
> ------------------------------------------------------
>
> Key: HIVE-1667
> URL: https://issues.apache.org/jira/browse/HIVE-1667
> Project: Hadoop Hive
> Issue Type: New Feature
> Reporter: Namit Jain
> Attachments: hive-1667.patch
>
>
> Currently, the group of the owner of the table is not stored in the metastore.
> Secondly, if you create a table, the table's owner group is set to the group for the parent. It is not read from the UGI passed in.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1667) Store the group of the owner of the
table in metastore
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12919370#action_12919370 ]
Namit Jain commented on HIVE-1667:
----------------------------------
No, ideally there should be privileges per table for a user/group via a separate API
> Store the group of the owner of the table in metastore
> ------------------------------------------------------
>
> Key: HIVE-1667
> URL: https://issues.apache.org/jira/browse/HIVE-1667
> Project: Hadoop Hive
> Issue Type: New Feature
> Reporter: Namit Jain
> Attachments: hive-1667.patch
>
>
> Currently, the group of the owner of the table is not stored in the metastore.
> Secondly, if you create a table, the table's owner group is set to the group for the parent. It is not read from the UGI passed in.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1667) Store the group of the owner of the
table in metastore
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12919348#action_12919348 ]
Ashutosh Chauhan commented on HIVE-1667:
----------------------------------------
Are you also envisioning a use case where Hive will let user to choose the group at the time of creation of table by letting user specify the group name in CREATE TABLE statement.
> Store the group of the owner of the table in metastore
> ------------------------------------------------------
>
> Key: HIVE-1667
> URL: https://issues.apache.org/jira/browse/HIVE-1667
> Project: Hadoop Hive
> Issue Type: New Feature
> Reporter: Namit Jain
> Attachments: hive-1667.patch
>
>
> Currently, the group of the owner of the table is not stored in the metastore.
> Secondly, if you create a table, the table's owner group is set to the group for the parent. It is not read from the UGI passed in.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.