You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sentry.apache.org by Sravya Tirukkovalur <sr...@cloudera.com> on 2014/04/21 21:32:50 UTC

Review Request 20515: Not able to read policy files on HDFS (Regression)

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20515/
-----------------------------------------------------------

Review request for sentry, Brock Noland and Prasad Mujumdar.


Bugs: sentry-176
    https://issues.apache.org/jira/browse/sentry-176


Repository: sentry


Description
-------

This bug surfaced on real cluster deployment of Sentry trunk. It was a rather small fix where hadoop configuration parameters were not being used, hence file system uri part was defaulting to file:///. As part of this fix, I also made some changes to allow end to end tests to use sentry-provider file on HDFS. Earlier all the tests were using sentry-provider.ini file on local file system, even though we were using miniDFS to store hive warehouse. Changed TestCrossDbOps to use this so that, we have some coverage where sentry policy file is on HDFS.


Diffs
-----

  sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/conf/HiveAuthzConf.java e162bbd6acbbc4f88ec8bb0d89aec4d6b98cfd44 
  sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/PolicyFile.java 0189f85e8503059dd626ec4775e65741b4b6b8ea 
  sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/PolicyFiles.java a908ec317c2f644cece80d0db0a05c75da62e14a 
  sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/SimpleFileProviderBackend.java e7f69ac36f61abe5c16ffe975aab29b504ca8412 
  sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/AbstractTestWithHiveServer.java 0258c2df1556d693fca7a932a47b6ab40421b93d 
  sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/AbstractTestWithStaticConfiguration.java 64444077eea7d6df67bc79a52bec122086aeecca 
  sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/TestCrossDbOps.java 836cb277da24b44089ed2faf01e36de23ffa9066 
  sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/TestPerDatabasePolicyFile.java 805d60f078a4f033186d237728d9d08349c7fdae 
  sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/TestServerConfiguration.java 6bb39bc1a3a2f1e978558c34d9823c5a8c25d8b7 
  sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/fs/AbstractDFS.java 145584d26bdcd3e2e166dd9df1cf65617387794f 
  sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/fs/ClusterDFS.java d5db8110dace40557972336b1bb4ccfbe894ef60 
  sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/fs/DFS.java 9e9bb276788bdb0155e13b269b3c8d27180d2c5a 
  sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/fs/MiniDFS.java de684a9676f45d19b873994c089dea4a4bf90099 
  sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/hiveserver/HiveServerFactory.java f00efdb4d72ebc218904d42fb9b42518416f413e 

Diff: https://reviews.apache.org/r/20515/diff/


Testing
-------

* Verified the fix on real cluster.
* New tests and existing tests pass.


Thanks,

Sravya Tirukkovalur


Re: Review Request 20515: Not able to read policy files on HDFS (Regression)

Posted by Vamsee Yarlagadda <va...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20515/#review41780
-----------------------------------------------------------

Ship it!


Ship It!

- Vamsee Yarlagadda


On April 21, 2014, 7:32 p.m., Sravya Tirukkovalur wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20515/
> -----------------------------------------------------------
> 
> (Updated April 21, 2014, 7:32 p.m.)
> 
> 
> Review request for sentry, Brock Noland and Prasad Mujumdar.
> 
> 
> Bugs: sentry-176
>     https://issues.apache.org/jira/browse/sentry-176
> 
> 
> Repository: sentry
> 
> 
> Description
> -------
> 
> This bug surfaced on real cluster deployment of Sentry trunk. It was a rather small fix where hadoop configuration parameters were not being used, hence file system uri part was defaulting to file:///. As part of this fix, I also made some changes to allow end to end tests to use sentry-provider file on HDFS. Earlier all the tests were using sentry-provider.ini file on local file system, even though we were using miniDFS to store hive warehouse. Changed TestCrossDbOps to use this so that, we have some coverage where sentry policy file is on HDFS.
> 
> 
> Diffs
> -----
> 
>   sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/conf/HiveAuthzConf.java e162bbd6acbbc4f88ec8bb0d89aec4d6b98cfd44 
>   sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/PolicyFile.java 0189f85e8503059dd626ec4775e65741b4b6b8ea 
>   sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/PolicyFiles.java a908ec317c2f644cece80d0db0a05c75da62e14a 
>   sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/SimpleFileProviderBackend.java e7f69ac36f61abe5c16ffe975aab29b504ca8412 
>   sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/AbstractTestWithHiveServer.java 0258c2df1556d693fca7a932a47b6ab40421b93d 
>   sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/AbstractTestWithStaticConfiguration.java 64444077eea7d6df67bc79a52bec122086aeecca 
>   sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/TestCrossDbOps.java 836cb277da24b44089ed2faf01e36de23ffa9066 
>   sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/TestPerDatabasePolicyFile.java 805d60f078a4f033186d237728d9d08349c7fdae 
>   sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/TestServerConfiguration.java 6bb39bc1a3a2f1e978558c34d9823c5a8c25d8b7 
>   sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/fs/AbstractDFS.java 145584d26bdcd3e2e166dd9df1cf65617387794f 
>   sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/fs/ClusterDFS.java d5db8110dace40557972336b1bb4ccfbe894ef60 
>   sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/fs/DFS.java 9e9bb276788bdb0155e13b269b3c8d27180d2c5a 
>   sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/fs/MiniDFS.java de684a9676f45d19b873994c089dea4a4bf90099 
>   sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/hiveserver/HiveServerFactory.java f00efdb4d72ebc218904d42fb9b42518416f413e 
> 
> Diff: https://reviews.apache.org/r/20515/diff/
> 
> 
> Testing
> -------
> 
> * Verified the fix on real cluster.
> * New tests and existing tests pass.
> 
> 
> Thanks,
> 
> Sravya Tirukkovalur
> 
>


Re: Review Request 20515: Not able to read policy files on HDFS (Regression)

Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20515/#review42509
-----------------------------------------------------------

Ship it!


Ship It!

- Jarek Cecho


On April 21, 2014, 7:32 p.m., Sravya Tirukkovalur wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20515/
> -----------------------------------------------------------
> 
> (Updated April 21, 2014, 7:32 p.m.)
> 
> 
> Review request for sentry, Brock Noland and Prasad Mujumdar.
> 
> 
> Bugs: sentry-176
>     https://issues.apache.org/jira/browse/sentry-176
> 
> 
> Repository: sentry
> 
> 
> Description
> -------
> 
> This bug surfaced on real cluster deployment of Sentry trunk. It was a rather small fix where hadoop configuration parameters were not being used, hence file system uri part was defaulting to file:///. As part of this fix, I also made some changes to allow end to end tests to use sentry-provider file on HDFS. Earlier all the tests were using sentry-provider.ini file on local file system, even though we were using miniDFS to store hive warehouse. Changed TestCrossDbOps to use this so that, we have some coverage where sentry policy file is on HDFS.
> 
> 
> Diffs
> -----
> 
>   sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/conf/HiveAuthzConf.java e162bbd6acbbc4f88ec8bb0d89aec4d6b98cfd44 
>   sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/PolicyFile.java 0189f85e8503059dd626ec4775e65741b4b6b8ea 
>   sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/PolicyFiles.java a908ec317c2f644cece80d0db0a05c75da62e14a 
>   sentry-provider/sentry-provider-file/src/main/java/org/apache/sentry/provider/file/SimpleFileProviderBackend.java e7f69ac36f61abe5c16ffe975aab29b504ca8412 
>   sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/AbstractTestWithHiveServer.java 0258c2df1556d693fca7a932a47b6ab40421b93d 
>   sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/AbstractTestWithStaticConfiguration.java 64444077eea7d6df67bc79a52bec122086aeecca 
>   sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/TestCrossDbOps.java 836cb277da24b44089ed2faf01e36de23ffa9066 
>   sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/TestPerDatabasePolicyFile.java 805d60f078a4f033186d237728d9d08349c7fdae 
>   sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/TestServerConfiguration.java 6bb39bc1a3a2f1e978558c34d9823c5a8c25d8b7 
>   sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/fs/AbstractDFS.java 145584d26bdcd3e2e166dd9df1cf65617387794f 
>   sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/fs/ClusterDFS.java d5db8110dace40557972336b1bb4ccfbe894ef60 
>   sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/fs/DFS.java 9e9bb276788bdb0155e13b269b3c8d27180d2c5a 
>   sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/fs/MiniDFS.java de684a9676f45d19b873994c089dea4a4bf90099 
>   sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/hiveserver/HiveServerFactory.java f00efdb4d72ebc218904d42fb9b42518416f413e 
> 
> Diff: https://reviews.apache.org/r/20515/diff/
> 
> 
> Testing
> -------
> 
> * Verified the fix on real cluster.
> * New tests and existing tests pass.
> 
> 
> Thanks,
> 
> Sravya Tirukkovalur
> 
>