You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Devaraj Das (JIRA)" <ji...@apache.org> on 2011/02/14 12:45:57 UTC
[jira] Created: (HIVE-1988) Make the delegation token issued by the
MetaStore owned by the right user
Make the delegation token issued by the MetaStore owned by the right user
-------------------------------------------------------------------------
Key: HIVE-1988
URL: https://issues.apache.org/jira/browse/HIVE-1988
Project: Hive
Issue Type: Bug
Components: Metastore, Security, Server Infrastructure
Affects Versions: 0.7.0
Reporter: Devaraj Das
Fix For: 0.7.0
The 'owner' of any delegation token issued by the MetaStore is set to the requesting user. When a delegation token is asked by the user himself during a job submission, this is fine. However, in the case where the token is requested for by services (e.g., Oozie), on behalf of the user, the token's owner is set to the user the service is running as. Later on, when the token is used by a MapReduce task, the MetaStore treats the incoming request as coming from Oozie and does operations as Oozie. This means any new directory creations (e.g., create_table) on the hdfs by the MetaStore will end up with Oozie as the owner.
Also, the MetaStore doesn't check whether a user asking for a token on behalf of some other user, is actually authorized to act on behalf of that other user. We should start using the ProxyUser authorization in the MetaStore (HADOOP-6510's APIs).
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1988) Make the delegation token issued by
the MetaStore owned by the right user
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016277#comment-13016277 ]
Amareshwari Sriramadasu commented on HIVE-1988:
-----------------------------------------------
Running tests. Will commit once tests pass.
> Make the delegation token issued by the MetaStore owned by the right user
> -------------------------------------------------------------------------
>
> Key: HIVE-1988
> URL: https://issues.apache.org/jira/browse/HIVE-1988
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Security, Server Infrastructure
> Affects Versions: 0.7.0
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.8.0
>
> Attachments: hive-1988-3.patch, hive-1988-5.1.patch, hive-1988.patch
>
>
> The 'owner' of any delegation token issued by the MetaStore is set to the requesting user. When a delegation token is asked by the user himself during a job submission, this is fine. However, in the case where the token is requested for by services (e.g., Oozie), on behalf of the user, the token's owner is set to the user the service is running as. Later on, when the token is used by a MapReduce task, the MetaStore treats the incoming request as coming from Oozie and does operations as Oozie. This means any new directory creations (e.g., create_table) on the hdfs by the MetaStore will end up with Oozie as the owner.
> Also, the MetaStore doesn't check whether a user asking for a token on behalf of some other user, is actually authorized to act on behalf of that other user. We should start using the ProxyUser authorization in the MetaStore (HADOOP-6510's APIs).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1988) Make the delegation token issued by
the MetaStore owned by the right user
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HIVE-1988:
------------------------------
Attachment: hive-1988-5.1.patch
The same patch that i uploaded on reviewboard..
> Make the delegation token issued by the MetaStore owned by the right user
> -------------------------------------------------------------------------
>
> Key: HIVE-1988
> URL: https://issues.apache.org/jira/browse/HIVE-1988
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Security, Server Infrastructure
> Affects Versions: 0.7.0
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.8.0
>
> Attachments: hive-1988-3.patch, hive-1988-5.1.patch, hive-1988.patch
>
>
> The 'owner' of any delegation token issued by the MetaStore is set to the requesting user. When a delegation token is asked by the user himself during a job submission, this is fine. However, in the case where the token is requested for by services (e.g., Oozie), on behalf of the user, the token's owner is set to the user the service is running as. Later on, when the token is used by a MapReduce task, the MetaStore treats the incoming request as coming from Oozie and does operations as Oozie. This means any new directory creations (e.g., create_table) on the hdfs by the MetaStore will end up with Oozie as the owner.
> Also, the MetaStore doesn't check whether a user asking for a token on behalf of some other user, is actually authorized to act on behalf of that other user. We should start using the ProxyUser authorization in the MetaStore (HADOOP-6510's APIs).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1988) Make the delegation token issued by
the MetaStore owned by the right user
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13012279#comment-13012279 ]
Devaraj Das commented on HIVE-1988:
-----------------------------------
I updated the reviewboard https://reviews.apache.org/r/528/ with some changes. The main change is: The methods getDelegationToken/renewDelegationToken now check for the authentication method being KERBEROS. If not, then it refuses to give a delegation token. This takes care of a security hole, where, if a delegation token has been compromised, the malicious user in possession of the token could use it to authenticate itself with the metastore, and get a new delegation token. This process could go forever (and hence the malicious user could access the metastore without ever going through a kerberos authentication). Making the handing out of delegation tokens based on a prior kerberos authentication limits this.
Also, the patch on reviewboard doesn't have generated code. I will upload it once someone takes a look at the patch and gives feedback.
> Make the delegation token issued by the MetaStore owned by the right user
> -------------------------------------------------------------------------
>
> Key: HIVE-1988
> URL: https://issues.apache.org/jira/browse/HIVE-1988
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Security, Server Infrastructure
> Affects Versions: 0.7.0
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.8.0
>
> Attachments: hive-1988-3.patch, hive-1988.patch
>
>
> The 'owner' of any delegation token issued by the MetaStore is set to the requesting user. When a delegation token is asked by the user himself during a job submission, this is fine. However, in the case where the token is requested for by services (e.g., Oozie), on behalf of the user, the token's owner is set to the user the service is running as. Later on, when the token is used by a MapReduce task, the MetaStore treats the incoming request as coming from Oozie and does operations as Oozie. This means any new directory creations (e.g., create_table) on the hdfs by the MetaStore will end up with Oozie as the owner.
> Also, the MetaStore doesn't check whether a user asking for a token on behalf of some other user, is actually authorized to act on behalf of that other user. We should start using the ProxyUser authorization in the MetaStore (HADOOP-6510's APIs).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1988) Make the delegation token issued by
the MetaStore owned by the right user
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016241#comment-13016241 ]
jiraposter@reviews.apache.org commented on HIVE-1988:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/528/#review394
-----------------------------------------------------------
Ship it!
+1
- Amareshwari
On 2011-04-05 21:24:34, Devaraj Das wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/528/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-04-05 21:24:34)
bq.
bq.
bq. Review request for hive.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Fixes to some security issues discussed in HIVE-1988
bq.
bq.
bq. This addresses bug HIVE-1988.
bq. https://issues.apache.org/jira/browse/HIVE-1988
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. http://svn.apache.org/repos/asf/hive/trunk/metastore/if/hive_metastore.thrift 1089155
bq. http://svn.apache.org/repos/asf/hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 1089155
bq. http://svn.apache.org/repos/asf/hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 1089155
bq. http://svn.apache.org/repos/asf/hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 1089155
bq. http://svn.apache.org/repos/asf/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 1089155
bq. http://svn.apache.org/repos/asf/hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php 1089155
bq. http://svn.apache.org/repos/asf/hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 1089155
bq. http://svn.apache.org/repos/asf/hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 1089155
bq. http://svn.apache.org/repos/asf/hive/trunk/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 1089155
bq. http://svn.apache.org/repos/asf/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1089155
bq. http://svn.apache.org/repos/asf/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 1089155
bq. http://svn.apache.org/repos/asf/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 1089155
bq. http://svn.apache.org/repos/asf/hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/DelegationTokenSecretManager.java 1089155
bq. http://svn.apache.org/repos/asf/hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 1089155
bq. http://svn.apache.org/repos/asf/hive/trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java 1089155
bq. http://svn.apache.org/repos/asf/hive/trunk/shims/src/test/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java 1089155
bq.
bq. Diff: https://reviews.apache.org/r/528/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. New unit test added and that passes. All unit tests passed.
bq.
bq.
bq. Thanks,
bq.
bq. Devaraj
bq.
bq.
> Make the delegation token issued by the MetaStore owned by the right user
> -------------------------------------------------------------------------
>
> Key: HIVE-1988
> URL: https://issues.apache.org/jira/browse/HIVE-1988
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Security, Server Infrastructure
> Affects Versions: 0.7.0
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.8.0
>
> Attachments: hive-1988-3.patch, hive-1988.patch
>
>
> The 'owner' of any delegation token issued by the MetaStore is set to the requesting user. When a delegation token is asked by the user himself during a job submission, this is fine. However, in the case where the token is requested for by services (e.g., Oozie), on behalf of the user, the token's owner is set to the user the service is running as. Later on, when the token is used by a MapReduce task, the MetaStore treats the incoming request as coming from Oozie and does operations as Oozie. This means any new directory creations (e.g., create_table) on the hdfs by the MetaStore will end up with Oozie as the owner.
> Also, the MetaStore doesn't check whether a user asking for a token on behalf of some other user, is actually authorized to act on behalf of that other user. We should start using the ProxyUser authorization in the MetaStore (HADOOP-6510's APIs).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1988) Make the delegation token issued by
the MetaStore owned by the right user
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015794#comment-13015794 ]
jiraposter@reviews.apache.org commented on HIVE-1988:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/528/#review386
-----------------------------------------------------------
http://svn.apache.org/repos/asf/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
<https://reviews.apache.org/r/528/#comment734>
HadoopShims.isSecureShimImpl() is not called anywhere else. Shall we remove it if not required anymore?
http://svn.apache.org/repos/asf/hive/trunk/shims/src/test/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java
<https://reviews.apache.org/r/528/#comment735>
Do you want to move this into setup(), as it is common in both testcases?
http://svn.apache.org/repos/asf/hive/trunk/shims/src/test/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java
<https://reviews.apache.org/r/528/#comment736>
code looks duplicated. Can it be refactored by passing group names to a method?
- Amareshwari
On 2011-03-29 10:26:38, Devaraj Das wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/528/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-03-29 10:26:38)
bq.
bq.
bq. Review request for hive.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Fixes to some security issues discussed in HIVE-1988
bq.
bq.
bq. This addresses bug HIVE-1988.
bq. https://issues.apache.org/jira/browse/HIVE-1988
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. http://svn.apache.org/repos/asf/hive/trunk/metastore/if/hive_metastore.thrift 1085623
bq. http://svn.apache.org/repos/asf/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1085623
bq. http://svn.apache.org/repos/asf/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 1085623
bq. http://svn.apache.org/repos/asf/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 1085623
bq. http://svn.apache.org/repos/asf/hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/DelegationTokenSecretManager.java 1085623
bq. http://svn.apache.org/repos/asf/hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 1085623
bq. http://svn.apache.org/repos/asf/hive/trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java 1085623
bq. http://svn.apache.org/repos/asf/hive/trunk/shims/src/test/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java 1085623
bq.
bq. Diff: https://reviews.apache.org/r/528/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. New unit test added and that passes. All unit tests passed.
bq.
bq.
bq. Thanks,
bq.
bq. Devaraj
bq.
bq.
> Make the delegation token issued by the MetaStore owned by the right user
> -------------------------------------------------------------------------
>
> Key: HIVE-1988
> URL: https://issues.apache.org/jira/browse/HIVE-1988
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Security, Server Infrastructure
> Affects Versions: 0.7.0
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.8.0
>
> Attachments: hive-1988-3.patch, hive-1988.patch
>
>
> The 'owner' of any delegation token issued by the MetaStore is set to the requesting user. When a delegation token is asked by the user himself during a job submission, this is fine. However, in the case where the token is requested for by services (e.g., Oozie), on behalf of the user, the token's owner is set to the user the service is running as. Later on, when the token is used by a MapReduce task, the MetaStore treats the incoming request as coming from Oozie and does operations as Oozie. This means any new directory creations (e.g., create_table) on the hdfs by the MetaStore will end up with Oozie as the owner.
> Also, the MetaStore doesn't check whether a user asking for a token on behalf of some other user, is actually authorized to act on behalf of that other user. We should start using the ProxyUser authorization in the MetaStore (HADOOP-6510's APIs).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1988) Make the delegation token issued by
the MetaStore owned by the right user
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HIVE-1988:
------------------------------
Fix Version/s: 0.8.0
Status: Patch Available (was: Open)
> Make the delegation token issued by the MetaStore owned by the right user
> -------------------------------------------------------------------------
>
> Key: HIVE-1988
> URL: https://issues.apache.org/jira/browse/HIVE-1988
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Security, Server Infrastructure
> Affects Versions: 0.7.0
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.8.0
>
> Attachments: hive-1988-3.patch, hive-1988.patch
>
>
> The 'owner' of any delegation token issued by the MetaStore is set to the requesting user. When a delegation token is asked by the user himself during a job submission, this is fine. However, in the case where the token is requested for by services (e.g., Oozie), on behalf of the user, the token's owner is set to the user the service is running as. Later on, when the token is used by a MapReduce task, the MetaStore treats the incoming request as coming from Oozie and does operations as Oozie. This means any new directory creations (e.g., create_table) on the hdfs by the MetaStore will end up with Oozie as the owner.
> Also, the MetaStore doesn't check whether a user asking for a token on behalf of some other user, is actually authorized to act on behalf of that other user. We should start using the ProxyUser authorization in the MetaStore (HADOOP-6510's APIs).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1988) Make the delegation token issued by
the MetaStore owned by the right user
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016129#comment-13016129 ]
jiraposter@reviews.apache.org commented on HIVE-1988:
-----------------------------------------------------
bq. On 2011-04-05 07:52:15, Amareshwari Sriramadasu wrote:
bq. > http://svn.apache.org/repos/asf/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 152
bq. > <https://reviews.apache.org/r/528/diff/2/?file=14844#file14844line152>
bq. >
bq. > HadoopShims.isSecureShimImpl() is not called anywhere else. Shall we remove it if not required anymore?
I suggest we leave it there. This seems like a useful method, and I am actually using it in another patch.
bq. On 2011-04-05 07:52:15, Amareshwari Sriramadasu wrote:
bq. > http://svn.apache.org/repos/asf/hive/trunk/shims/src/test/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java, lines 144-156
bq. > <https://reviews.apache.org/r/528/diff/2/?file=14850#file14850line144>
bq. >
bq. > Do you want to move this into setup(), as it is common in both testcases?
Done
bq. On 2011-04-05 07:52:15, Amareshwari Sriramadasu wrote:
bq. > http://svn.apache.org/repos/asf/hive/trunk/shims/src/test/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java, lines 192-209
bq. > <https://reviews.apache.org/r/528/diff/2/?file=14850#file14850line192>
bq. >
bq. > code looks duplicated. Can it be refactored by passing group names to a method?
Done
- Devaraj
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/528/#review386
-----------------------------------------------------------
On 2011-03-29 10:26:38, Devaraj Das wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/528/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-03-29 10:26:38)
bq.
bq.
bq. Review request for hive.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Fixes to some security issues discussed in HIVE-1988
bq.
bq.
bq. This addresses bug HIVE-1988.
bq. https://issues.apache.org/jira/browse/HIVE-1988
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. http://svn.apache.org/repos/asf/hive/trunk/metastore/if/hive_metastore.thrift 1085623
bq. http://svn.apache.org/repos/asf/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1085623
bq. http://svn.apache.org/repos/asf/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 1085623
bq. http://svn.apache.org/repos/asf/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 1085623
bq. http://svn.apache.org/repos/asf/hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/DelegationTokenSecretManager.java 1085623
bq. http://svn.apache.org/repos/asf/hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 1085623
bq. http://svn.apache.org/repos/asf/hive/trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java 1085623
bq. http://svn.apache.org/repos/asf/hive/trunk/shims/src/test/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java 1085623
bq.
bq. Diff: https://reviews.apache.org/r/528/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. New unit test added and that passes. All unit tests passed.
bq.
bq.
bq. Thanks,
bq.
bq. Devaraj
bq.
bq.
> Make the delegation token issued by the MetaStore owned by the right user
> -------------------------------------------------------------------------
>
> Key: HIVE-1988
> URL: https://issues.apache.org/jira/browse/HIVE-1988
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Security, Server Infrastructure
> Affects Versions: 0.7.0
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.8.0
>
> Attachments: hive-1988-3.patch, hive-1988.patch
>
>
> The 'owner' of any delegation token issued by the MetaStore is set to the requesting user. When a delegation token is asked by the user himself during a job submission, this is fine. However, in the case where the token is requested for by services (e.g., Oozie), on behalf of the user, the token's owner is set to the user the service is running as. Later on, when the token is used by a MapReduce task, the MetaStore treats the incoming request as coming from Oozie and does operations as Oozie. This means any new directory creations (e.g., create_table) on the hdfs by the MetaStore will end up with Oozie as the owner.
> Also, the MetaStore doesn't check whether a user asking for a token on behalf of some other user, is actually authorized to act on behalf of that other user. We should start using the ProxyUser authorization in the MetaStore (HADOOP-6510's APIs).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1988) Make the delegation token issued by
the MetaStore owned by the right user
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13011538#comment-13011538 ]
Devaraj Das commented on HIVE-1988:
-----------------------------------
This patch also does some refactoring of the audit logging part. It moves a class to do with audit logging to the 20S shim code.
> Make the delegation token issued by the MetaStore owned by the right user
> -------------------------------------------------------------------------
>
> Key: HIVE-1988
> URL: https://issues.apache.org/jira/browse/HIVE-1988
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Security, Server Infrastructure
> Affects Versions: 0.7.0
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.8.0
>
> Attachments: hive-1988-3.patch, hive-1988.patch
>
>
> The 'owner' of any delegation token issued by the MetaStore is set to the requesting user. When a delegation token is asked by the user himself during a job submission, this is fine. However, in the case where the token is requested for by services (e.g., Oozie), on behalf of the user, the token's owner is set to the user the service is running as. Later on, when the token is used by a MapReduce task, the MetaStore treats the incoming request as coming from Oozie and does operations as Oozie. This means any new directory creations (e.g., create_table) on the hdfs by the MetaStore will end up with Oozie as the owner.
> Also, the MetaStore doesn't check whether a user asking for a token on behalf of some other user, is actually authorized to act on behalf of that other user. We should start using the ProxyUser authorization in the MetaStore (HADOOP-6510's APIs).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-1988) Make the delegation token issued by
the MetaStore owned by the right user
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das reassigned HIVE-1988:
---------------------------------
Assignee: Devaraj Das
> Make the delegation token issued by the MetaStore owned by the right user
> -------------------------------------------------------------------------
>
> Key: HIVE-1988
> URL: https://issues.apache.org/jira/browse/HIVE-1988
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Security, Server Infrastructure
> Affects Versions: 0.7.0
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.8.0
>
> Attachments: hive-1988-3.patch, hive-1988.patch
>
>
> The 'owner' of any delegation token issued by the MetaStore is set to the requesting user. When a delegation token is asked by the user himself during a job submission, this is fine. However, in the case where the token is requested for by services (e.g., Oozie), on behalf of the user, the token's owner is set to the user the service is running as. Later on, when the token is used by a MapReduce task, the MetaStore treats the incoming request as coming from Oozie and does operations as Oozie. This means any new directory creations (e.g., create_table) on the hdfs by the MetaStore will end up with Oozie as the owner.
> Also, the MetaStore doesn't check whether a user asking for a token on behalf of some other user, is actually authorized to act on behalf of that other user. We should start using the ProxyUser authorization in the MetaStore (HADOOP-6510's APIs).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1988) Make the delegation token issued by
the MetaStore owned by the right user
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HIVE-1988:
------------------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed this.
Thanks Devaraj!
> Make the delegation token issued by the MetaStore owned by the right user
> -------------------------------------------------------------------------
>
> Key: HIVE-1988
> URL: https://issues.apache.org/jira/browse/HIVE-1988
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Security, Server Infrastructure
> Affects Versions: 0.7.0
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.8.0
>
> Attachments: hive-1988-3.patch, hive-1988-5.1.patch, hive-1988.patch
>
>
> The 'owner' of any delegation token issued by the MetaStore is set to the requesting user. When a delegation token is asked by the user himself during a job submission, this is fine. However, in the case where the token is requested for by services (e.g., Oozie), on behalf of the user, the token's owner is set to the user the service is running as. Later on, when the token is used by a MapReduce task, the MetaStore treats the incoming request as coming from Oozie and does operations as Oozie. This means any new directory creations (e.g., create_table) on the hdfs by the MetaStore will end up with Oozie as the owner.
> Also, the MetaStore doesn't check whether a user asking for a token on behalf of some other user, is actually authorized to act on behalf of that other user. We should start using the ProxyUser authorization in the MetaStore (HADOOP-6510's APIs).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1988) Make the delegation token issued by
the MetaStore owned by the right user
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HIVE-1988:
------------------------------
Attachment: hive-1988-3.patch
https://reviews.apache.org/r/528/ is the reviewboard page.
> Make the delegation token issued by the MetaStore owned by the right user
> -------------------------------------------------------------------------
>
> Key: HIVE-1988
> URL: https://issues.apache.org/jira/browse/HIVE-1988
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Security, Server Infrastructure
> Affects Versions: 0.7.0
> Reporter: Devaraj Das
> Fix For: 0.8.0
>
> Attachments: hive-1988-3.patch, hive-1988.patch
>
>
> The 'owner' of any delegation token issued by the MetaStore is set to the requesting user. When a delegation token is asked by the user himself during a job submission, this is fine. However, in the case where the token is requested for by services (e.g., Oozie), on behalf of the user, the token's owner is set to the user the service is running as. Later on, when the token is used by a MapReduce task, the MetaStore treats the incoming request as coming from Oozie and does operations as Oozie. This means any new directory creations (e.g., create_table) on the hdfs by the MetaStore will end up with Oozie as the owner.
> Also, the MetaStore doesn't check whether a user asking for a token on behalf of some other user, is actually authorized to act on behalf of that other user. We should start using the ProxyUser authorization in the MetaStore (HADOOP-6510's APIs).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1988) Make the delegation token issued by
the MetaStore owned by the right user
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015795#comment-13015795 ]
Amareshwari Sriramadasu commented on HIVE-1988:
-----------------------------------------------
Changes look good overall. I updated the review board with some minor comments. You can upload the next patch with generated code.
> Make the delegation token issued by the MetaStore owned by the right user
> -------------------------------------------------------------------------
>
> Key: HIVE-1988
> URL: https://issues.apache.org/jira/browse/HIVE-1988
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Security, Server Infrastructure
> Affects Versions: 0.7.0
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.8.0
>
> Attachments: hive-1988-3.patch, hive-1988.patch
>
>
> The 'owner' of any delegation token issued by the MetaStore is set to the requesting user. When a delegation token is asked by the user himself during a job submission, this is fine. However, in the case where the token is requested for by services (e.g., Oozie), on behalf of the user, the token's owner is set to the user the service is running as. Later on, when the token is used by a MapReduce task, the MetaStore treats the incoming request as coming from Oozie and does operations as Oozie. This means any new directory creations (e.g., create_table) on the hdfs by the MetaStore will end up with Oozie as the owner.
> Also, the MetaStore doesn't check whether a user asking for a token on behalf of some other user, is actually authorized to act on behalf of that other user. We should start using the ProxyUser authorization in the MetaStore (HADOOP-6510's APIs).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1988) Make the delegation token issued by the
MetaStore owned by the right user
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-1988:
-----------------------------------
Attachment: hive-1988.patch
WIP. Attaching here for reference.
> Make the delegation token issued by the MetaStore owned by the right user
> -------------------------------------------------------------------------
>
> Key: HIVE-1988
> URL: https://issues.apache.org/jira/browse/HIVE-1988
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Security, Server Infrastructure
> Affects Versions: 0.7.0
> Reporter: Devaraj Das
> Fix For: 0.7.0
>
> Attachments: hive-1988.patch
>
>
> The 'owner' of any delegation token issued by the MetaStore is set to the requesting user. When a delegation token is asked by the user himself during a job submission, this is fine. However, in the case where the token is requested for by services (e.g., Oozie), on behalf of the user, the token's owner is set to the user the service is running as. Later on, when the token is used by a MapReduce task, the MetaStore treats the incoming request as coming from Oozie and does operations as Oozie. This means any new directory creations (e.g., create_table) on the hdfs by the MetaStore will end up with Oozie as the owner.
> Also, the MetaStore doesn't check whether a user asking for a token on behalf of some other user, is actually authorized to act on behalf of that other user. We should start using the ProxyUser authorization in the MetaStore (HADOOP-6510's APIs).
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1988) Make the delegation token issued by
the MetaStore owned by the right user
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016128#comment-13016128 ]
jiraposter@reviews.apache.org commented on HIVE-1988:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/528/
-----------------------------------------------------------
(Updated 2011-04-05 21:24:34.129643)
Review request for hive.
Changes
-------
Addressed Amareshwari's comments.
Summary
-------
Fixes to some security issues discussed in HIVE-1988
This addresses bug HIVE-1988.
https://issues.apache.org/jira/browse/HIVE-1988
Diffs (updated)
-----
http://svn.apache.org/repos/asf/hive/trunk/metastore/if/hive_metastore.thrift 1089155
http://svn.apache.org/repos/asf/hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 1089155
http://svn.apache.org/repos/asf/hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 1089155
http://svn.apache.org/repos/asf/hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 1089155
http://svn.apache.org/repos/asf/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 1089155
http://svn.apache.org/repos/asf/hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php 1089155
http://svn.apache.org/repos/asf/hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 1089155
http://svn.apache.org/repos/asf/hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 1089155
http://svn.apache.org/repos/asf/hive/trunk/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 1089155
http://svn.apache.org/repos/asf/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1089155
http://svn.apache.org/repos/asf/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 1089155
http://svn.apache.org/repos/asf/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 1089155
http://svn.apache.org/repos/asf/hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/DelegationTokenSecretManager.java 1089155
http://svn.apache.org/repos/asf/hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 1089155
http://svn.apache.org/repos/asf/hive/trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java 1089155
http://svn.apache.org/repos/asf/hive/trunk/shims/src/test/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java 1089155
Diff: https://reviews.apache.org/r/528/diff
Testing
-------
New unit test added and that passes. All unit tests passed.
Thanks,
Devaraj
> Make the delegation token issued by the MetaStore owned by the right user
> -------------------------------------------------------------------------
>
> Key: HIVE-1988
> URL: https://issues.apache.org/jira/browse/HIVE-1988
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Security, Server Infrastructure
> Affects Versions: 0.7.0
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.8.0
>
> Attachments: hive-1988-3.patch, hive-1988.patch
>
>
> The 'owner' of any delegation token issued by the MetaStore is set to the requesting user. When a delegation token is asked by the user himself during a job submission, this is fine. However, in the case where the token is requested for by services (e.g., Oozie), on behalf of the user, the token's owner is set to the user the service is running as. Later on, when the token is used by a MapReduce task, the MetaStore treats the incoming request as coming from Oozie and does operations as Oozie. This means any new directory creations (e.g., create_table) on the hdfs by the MetaStore will end up with Oozie as the owner.
> Also, the MetaStore doesn't check whether a user asking for a token on behalf of some other user, is actually authorized to act on behalf of that other user. We should start using the ProxyUser authorization in the MetaStore (HADOOP-6510's APIs).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira