You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sentry.apache.org by Hao Hao <ha...@cloudera.com> on 2016/07/19 07:01:38 UTC
Review Request 50182: Sentry-1371: Rework Sentry start up and Hive
state fetch
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50182/
-----------------------------------------------------------
Review request for sentry, Anne Yu and Sravya Tirukkovalur.
Repository: sentry
Description
-------
Added logic in HMSFollower for hive full update. Block sentry service from start up until full update fetch is complete.
Diffs
-----
sentry-binding/pom.xml 30bca146786a833391fa44c62e036777b694a7b1
sentry-binding/sentry-binding-hive-common/pom.xml 18b422d5a688e636af4e01b382fa3e5677ac884b
sentry-binding/sentry-binding-hive-common/src/main/java/org/apache/sentry/binding/hive/conf/HiveAuthzConf.java
sentry-binding/sentry-binding-hive-conf/pom.xml PRE-CREATION
sentry-binding/sentry-binding-hive-follower/pom.xml PRE-CREATION
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/JSONAlterPartitionMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/JSONAlterTableMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONAddPartitionMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONAlterPartitionMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONAlterTableMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONCreateDatabaseMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONCreateTableMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONDropDatabaseMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONDropPartitionMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONDropTableMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONMessageDeserializer.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONMessageFactory.java
sentry-hdfs/sentry-hdfs-common/pom.xml e767e06c13be2312ccd04e39c518ddef4fbf7744
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastoreCacheInitializer.java 807e4e0fbc10e00dbc716be2befe66795d4936a7
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastorePlugin.java f37596df0d00c9be39e95695dbafde088ad474cb
sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/TestMetastoreCacheInitializer.java 9e6072df23ca404cd500a940a1035dfdc03d6928
sentry-provider/sentry-provider-db/pom.xml b8143ffa3adca9e47e7cb092131d65064d57c86b
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java PRE-CREATION
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryKerberosContext.java f54f161b381088285486a5ca74972f93ee620547
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java 531ab3545e1beb79c18e713f9d97453ee258748d
Diff: https://reviews.apache.org/r/50182/diff/
Testing
-------
Thanks,
Hao Hao
Re: Review Request 50182: Sentry-1371: Rework Sentry start up and
Hive state fetch
Posted by Sravya Tirukkovalur <sr...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50182/#review144572
-----------------------------------------------------------
Ship it!
Ship It!
- Sravya Tirukkovalur
On Aug. 2, 2016, 11:27 p.m., Hao Hao wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50182/
> -----------------------------------------------------------
>
> (Updated Aug. 2, 2016, 11:27 p.m.)
>
>
> Review request for sentry, Anne Yu and Sravya Tirukkovalur.
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Added logic in HMSFollower for hive full update.
> Block sentry service from start up until full update fetch is complete.
> Fetch notification logs which are before failover but did not applied.
>
>
> Diffs
> -----
>
> sentry-hdfs/sentry-hdfs-common/pom.xml e767e06c13be2312ccd04e39c518ddef4fbf7744
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java PRE-CREATION
> sentry-hdfs/sentry-hdfs-common/src/test/java/org/apache/sentry/hdfs/TestFullUpdateInitializer.java PRE-CREATION
> sentry-provider/sentry-provider-db/pom.xml b8143ffa3adca9e47e7cb092131d65064d57c86b
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java PRE-CREATION
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java e93e5b45caa1db61d6c5d69312f66f475dd02ee8
>
> Diff: https://reviews.apache.org/r/50182/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Hao Hao
>
>
Re: Review Request 50182: Sentry-1371: Rework Sentry start up and
Hive state fetch
Posted by Hao Hao <ha...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50182/
-----------------------------------------------------------
(Updated Aug. 2, 2016, 11:27 p.m.)
Review request for sentry, Anne Yu and Sravya Tirukkovalur.
Repository: sentry
Description
-------
Added logic in HMSFollower for hive full update.
Block sentry service from start up until full update fetch is complete.
Fetch notification logs which are before failover but did not applied.
Diffs (updated)
-----
sentry-hdfs/sentry-hdfs-common/pom.xml e767e06c13be2312ccd04e39c518ddef4fbf7744
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java PRE-CREATION
sentry-hdfs/sentry-hdfs-common/src/test/java/org/apache/sentry/hdfs/TestFullUpdateInitializer.java PRE-CREATION
sentry-provider/sentry-provider-db/pom.xml b8143ffa3adca9e47e7cb092131d65064d57c86b
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java PRE-CREATION
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java e93e5b45caa1db61d6c5d69312f66f475dd02ee8
Diff: https://reviews.apache.org/r/50182/diff/
Testing
-------
Thanks,
Hao Hao
Re: Review Request 50182: Sentry-1371: Rework Sentry start up and
Hive state fetch
Posted by Sravya Tirukkovalur <sr...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50182/#review144152
-----------------------------------------------------------
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java (line 39)
<https://reviews.apache.org/r/50182/#comment210171>
Javadoc: Class description?
- Sravya Tirukkovalur
On July 29, 2016, 1:34 a.m., Hao Hao wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50182/
> -----------------------------------------------------------
>
> (Updated July 29, 2016, 1:34 a.m.)
>
>
> Review request for sentry, Anne Yu and Sravya Tirukkovalur.
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Added logic in HMSFollower for hive full update.
> Block sentry service from start up until full update fetch is complete.
> Fetch notification logs which are before failover but did not applied.
>
>
> Diffs
> -----
>
> sentry-hdfs/sentry-hdfs-common/pom.xml e767e06c13be2312ccd04e39c518ddef4fbf7744
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java PRE-CREATION
> sentry-hdfs/sentry-hdfs-common/src/test/java/org/apache/sentry/hdfs/TestFullUpdateInitializer.java PRE-CREATION
> sentry-provider/sentry-provider-db/pom.xml b8143ffa3adca9e47e7cb092131d65064d57c86b
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java PRE-CREATION
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java e93e5b45caa1db61d6c5d69312f66f475dd02ee8
>
> Diff: https://reviews.apache.org/r/50182/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Hao Hao
>
>
Re: Review Request 50182: Sentry-1371: Rework Sentry start up and
Hive state fetch
Posted by Hao Hao <ha...@cloudera.com>.
> On July 29, 2016, 8:03 p.m., Sravya Tirukkovalur wrote:
> > Seems like we need to not block Sentry startup for the sake of HMS full update (to avoid dependencies at startup), but would like to not serve requests ( eventually only for Hive requests) until the full update processing is finished?
Yeah, put it as todo and filed a upstream for tracking this SENTRY-1437.
- Hao
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50182/#review144165
-----------------------------------------------------------
On July 29, 2016, 1:34 a.m., Hao Hao wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50182/
> -----------------------------------------------------------
>
> (Updated July 29, 2016, 1:34 a.m.)
>
>
> Review request for sentry, Anne Yu and Sravya Tirukkovalur.
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Added logic in HMSFollower for hive full update.
> Block sentry service from start up until full update fetch is complete.
> Fetch notification logs which are before failover but did not applied.
>
>
> Diffs
> -----
>
> sentry-hdfs/sentry-hdfs-common/pom.xml e767e06c13be2312ccd04e39c518ddef4fbf7744
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java PRE-CREATION
> sentry-hdfs/sentry-hdfs-common/src/test/java/org/apache/sentry/hdfs/TestFullUpdateInitializer.java PRE-CREATION
> sentry-provider/sentry-provider-db/pom.xml b8143ffa3adca9e47e7cb092131d65064d57c86b
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java PRE-CREATION
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java e93e5b45caa1db61d6c5d69312f66f475dd02ee8
>
> Diff: https://reviews.apache.org/r/50182/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Hao Hao
>
>
Re: Review Request 50182: Sentry-1371: Rework Sentry start up and
Hive state fetch
Posted by Sravya Tirukkovalur <sr...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50182/#review144165
-----------------------------------------------------------
Seems like we need to not block Sentry startup for the sake of HMS full update (to avoid dependencies at startup), but would like to not serve requests ( eventually only for Hive requests) until the full update processing is finished?
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java (line 234)
<https://reviews.apache.org/r/50182/#comment210185>
Seems like we should make a synchronized call to createIntialUpdate anyways as the calling thread needs to block? In which case, is there a reason why we want to sppin up another thread?
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java (lines 257 - 266)
<https://reviews.apache.org/r/50182/#comment210186>
Seems like we should never apply deltas unless full update processing is completed?
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java (lines 269 - 272)
<https://reviews.apache.org/r/50182/#comment210182>
Seems like this needs to be implemented partially in this patch? That is, use fullUpdateComplete? Eventually we should set this value based on notification id stored in Sentry?
- Sravya Tirukkovalur
On July 29, 2016, 1:34 a.m., Hao Hao wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50182/
> -----------------------------------------------------------
>
> (Updated July 29, 2016, 1:34 a.m.)
>
>
> Review request for sentry, Anne Yu and Sravya Tirukkovalur.
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Added logic in HMSFollower for hive full update.
> Block sentry service from start up until full update fetch is complete.
> Fetch notification logs which are before failover but did not applied.
>
>
> Diffs
> -----
>
> sentry-hdfs/sentry-hdfs-common/pom.xml e767e06c13be2312ccd04e39c518ddef4fbf7744
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java PRE-CREATION
> sentry-hdfs/sentry-hdfs-common/src/test/java/org/apache/sentry/hdfs/TestFullUpdateInitializer.java PRE-CREATION
> sentry-provider/sentry-provider-db/pom.xml b8143ffa3adca9e47e7cb092131d65064d57c86b
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java PRE-CREATION
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java e93e5b45caa1db61d6c5d69312f66f475dd02ee8
>
> Diff: https://reviews.apache.org/r/50182/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Hao Hao
>
>
Re: Review Request 50182: Sentry-1371: Rework Sentry start up and
Hive state fetch
Posted by Hao Hao <ha...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50182/
-----------------------------------------------------------
(Updated July 29, 2016, 1:34 a.m.)
Review request for sentry, Anne Yu and Sravya Tirukkovalur.
Repository: sentry
Description
-------
Added logic in HMSFollower for hive full update.
Block sentry service from start up until full update fetch is complete.
Fetch notification logs which are before failover but did not applied.
Diffs (updated)
-----
sentry-hdfs/sentry-hdfs-common/pom.xml e767e06c13be2312ccd04e39c518ddef4fbf7744
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java PRE-CREATION
sentry-hdfs/sentry-hdfs-common/src/test/java/org/apache/sentry/hdfs/TestFullUpdateInitializer.java PRE-CREATION
sentry-provider/sentry-provider-db/pom.xml b8143ffa3adca9e47e7cb092131d65064d57c86b
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java PRE-CREATION
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java e93e5b45caa1db61d6c5d69312f66f475dd02ee8
Diff: https://reviews.apache.org/r/50182/diff/
Testing
-------
Thanks,
Hao Hao
Re: Review Request 50182: Sentry-1371: Rework Sentry start up and
Hive state fetch
Posted by Hao Hao <ha...@cloudera.com>.
> On July 28, 2016, 6:08 p.m., Rahul Sharma wrote:
> > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java, line 169
> > <https://reviews.apache.org/r/50182/diff/4/?file=1449586#file1449586line169>
> >
> > Handle exector shutdown in case of exception?
Relevent to Sentry-1321.
- Hao
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50182/#review143970
-----------------------------------------------------------
On July 29, 2016, 1:34 a.m., Hao Hao wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50182/
> -----------------------------------------------------------
>
> (Updated July 29, 2016, 1:34 a.m.)
>
>
> Review request for sentry, Anne Yu and Sravya Tirukkovalur.
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Added logic in HMSFollower for hive full update.
> Block sentry service from start up until full update fetch is complete.
> Fetch notification logs which are before failover but did not applied.
>
>
> Diffs
> -----
>
> sentry-hdfs/sentry-hdfs-common/pom.xml e767e06c13be2312ccd04e39c518ddef4fbf7744
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java PRE-CREATION
> sentry-hdfs/sentry-hdfs-common/src/test/java/org/apache/sentry/hdfs/TestFullUpdateInitializer.java PRE-CREATION
> sentry-provider/sentry-provider-db/pom.xml b8143ffa3adca9e47e7cb092131d65064d57c86b
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java PRE-CREATION
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java e93e5b45caa1db61d6c5d69312f66f475dd02ee8
>
> Diff: https://reviews.apache.org/r/50182/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Hao Hao
>
>
Re: Review Request 50182: Sentry-1371: Rework Sentry start up and
Hive state fetch
Posted by Rahul Sharma <ra...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50182/#review143970
-----------------------------------------------------------
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java (line 169)
<https://reviews.apache.org/r/50182/#comment209906>
Handle exector shutdown in case of exception?
- Rahul Sharma
On July 21, 2016, 6:22 a.m., Hao Hao wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50182/
> -----------------------------------------------------------
>
> (Updated July 21, 2016, 6:22 a.m.)
>
>
> Review request for sentry, Anne Yu and Sravya Tirukkovalur.
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Added logic in HMSFollower for hive full update.
> Block sentry service from start up until full update fetch is complete.
> Fetch notification logs which are before failover but did not applied.
>
>
> Diffs
> -----
>
> sentry-binding/pom.xml 30bca146786a833391fa44c62e036777b694a7b1
> sentry-binding/sentry-binding-hive-common/pom.xml 18b422d5a688e636af4e01b382fa3e5677ac884b
> sentry-binding/sentry-binding-hive-common/src/main/java/org/apache/sentry/binding/hive/conf/HiveAuthzConf.java
> sentry-binding/sentry-binding-hive-conf/pom.xml PRE-CREATION
> sentry-binding/sentry-binding-hive-follower/pom.xml PRE-CREATION
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/JSONAlterPartitionMessage.java
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/JSONAlterTableMessage.java
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONAddPartitionMessage.java
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONAlterPartitionMessage.java
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONAlterTableMessage.java
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONCreateDatabaseMessage.java
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONCreateTableMessage.java
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONDropDatabaseMessage.java
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONDropPartitionMessage.java
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONDropTableMessage.java
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONMessageDeserializer.java
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONMessageFactory.java
> sentry-hdfs/sentry-hdfs-common/pom.xml e767e06c13be2312ccd04e39c518ddef4fbf7744
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastoreCacheInitializer.java 807e4e0fbc10e00dbc716be2befe66795d4936a7
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastorePlugin.java f37596df0d00c9be39e95695dbafde088ad474cb
> sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/TestMetastoreCacheInitializer.java 9e6072df23ca404cd500a940a1035dfdc03d6928
> sentry-provider/sentry-provider-db/pom.xml b8143ffa3adca9e47e7cb092131d65064d57c86b
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java PRE-CREATION
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryKerberosContext.java f54f161b381088285486a5ca74972f93ee620547
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java 531ab3545e1beb79c18e713f9d97453ee258748d
>
> Diff: https://reviews.apache.org/r/50182/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Hao Hao
>
>
Re: Review Request 50182: Sentry-1371: Rework Sentry start up and
Hive state fetch
Posted by Hao Hao <ha...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50182/
-----------------------------------------------------------
(Updated July 21, 2016, 6:22 a.m.)
Review request for sentry, Anne Yu and Sravya Tirukkovalur.
Repository: sentry
Description (updated)
-------
Added logic in HMSFollower for hive full update.
Block sentry service from start up until full update fetch is complete.
Fetch notification logs which are before failover but did not applied.
Diffs (updated)
-----
sentry-binding/pom.xml 30bca146786a833391fa44c62e036777b694a7b1
sentry-binding/sentry-binding-hive-common/pom.xml 18b422d5a688e636af4e01b382fa3e5677ac884b
sentry-binding/sentry-binding-hive-common/src/main/java/org/apache/sentry/binding/hive/conf/HiveAuthzConf.java
sentry-binding/sentry-binding-hive-conf/pom.xml PRE-CREATION
sentry-binding/sentry-binding-hive-follower/pom.xml PRE-CREATION
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/JSONAlterPartitionMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/JSONAlterTableMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONAddPartitionMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONAlterPartitionMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONAlterTableMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONCreateDatabaseMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONCreateTableMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONDropDatabaseMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONDropPartitionMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONDropTableMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONMessageDeserializer.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONMessageFactory.java
sentry-hdfs/sentry-hdfs-common/pom.xml e767e06c13be2312ccd04e39c518ddef4fbf7744
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastoreCacheInitializer.java 807e4e0fbc10e00dbc716be2befe66795d4936a7
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastorePlugin.java f37596df0d00c9be39e95695dbafde088ad474cb
sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/TestMetastoreCacheInitializer.java 9e6072df23ca404cd500a940a1035dfdc03d6928
sentry-provider/sentry-provider-db/pom.xml b8143ffa3adca9e47e7cb092131d65064d57c86b
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java PRE-CREATION
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryKerberosContext.java f54f161b381088285486a5ca74972f93ee620547
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java 531ab3545e1beb79c18e713f9d97453ee258748d
Diff: https://reviews.apache.org/r/50182/diff/
Testing
-------
Thanks,
Hao Hao
Re: Review Request 50182: Sentry-1371: Rework Sentry start up and
Hive state fetch
Posted by Hao Hao <ha...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50182/
-----------------------------------------------------------
(Updated July 21, 2016, 6:11 a.m.)
Review request for sentry, Anne Yu and Sravya Tirukkovalur.
Repository: sentry
Description
-------
Added logic in HMSFollower for hive full update. Block sentry service from start up until full update fetch is complete.
Diffs (updated)
-----
sentry-binding/pom.xml 30bca146786a833391fa44c62e036777b694a7b1
sentry-binding/sentry-binding-hive-common/pom.xml 18b422d5a688e636af4e01b382fa3e5677ac884b
sentry-binding/sentry-binding-hive-common/src/main/java/org/apache/sentry/binding/hive/conf/HiveAuthzConf.java
sentry-binding/sentry-binding-hive-conf/pom.xml PRE-CREATION
sentry-binding/sentry-binding-hive-follower/pom.xml PRE-CREATION
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/JSONAlterPartitionMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/JSONAlterTableMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONAddPartitionMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONAlterPartitionMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONAlterTableMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONCreateDatabaseMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONCreateTableMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONDropDatabaseMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONDropPartitionMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONDropTableMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONMessageDeserializer.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONMessageFactory.java
sentry-hdfs/sentry-hdfs-common/pom.xml e767e06c13be2312ccd04e39c518ddef4fbf7744
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastoreCacheInitializer.java 807e4e0fbc10e00dbc716be2befe66795d4936a7
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastorePlugin.java f37596df0d00c9be39e95695dbafde088ad474cb
sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/TestMetastoreCacheInitializer.java 9e6072df23ca404cd500a940a1035dfdc03d6928
sentry-provider/sentry-provider-db/pom.xml b8143ffa3adca9e47e7cb092131d65064d57c86b
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java PRE-CREATION
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryKerberosContext.java f54f161b381088285486a5ca74972f93ee620547
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java 531ab3545e1beb79c18e713f9d97453ee258748d
Diff: https://reviews.apache.org/r/50182/diff/
Testing
-------
Thanks,
Hao Hao
Re: Review Request 50182: Sentry-1371: Rework Sentry start up and
Hive state fetch
Posted by Hao Hao <ha...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50182/
-----------------------------------------------------------
(Updated July 21, 2016, 6:01 a.m.)
Review request for sentry, Anne Yu and Sravya Tirukkovalur.
Repository: sentry
Description
-------
Added logic in HMSFollower for hive full update. Block sentry service from start up until full update fetch is complete.
Diffs (updated)
-----
sentry-binding/pom.xml 30bca146786a833391fa44c62e036777b694a7b1
sentry-binding/sentry-binding-hive-common/pom.xml 18b422d5a688e636af4e01b382fa3e5677ac884b
sentry-binding/sentry-binding-hive-common/src/main/java/org/apache/sentry/binding/hive/conf/HiveAuthzConf.java
sentry-binding/sentry-binding-hive-conf/pom.xml PRE-CREATION
sentry-binding/sentry-binding-hive-follower/pom.xml PRE-CREATION
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/JSONAlterPartitionMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/JSONAlterTableMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONAddPartitionMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONAlterPartitionMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONAlterTableMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONCreateDatabaseMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONCreateTableMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONDropDatabaseMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONDropPartitionMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONDropTableMessage.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONMessageDeserializer.java
sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONMessageFactory.java
sentry-hdfs/sentry-hdfs-common/pom.xml e767e06c13be2312ccd04e39c518ddef4fbf7744
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastoreCacheInitializer.java 807e4e0fbc10e00dbc716be2befe66795d4936a7
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastorePlugin.java f37596df0d00c9be39e95695dbafde088ad474cb
sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/TestMetastoreCacheInitializer.java 9e6072df23ca404cd500a940a1035dfdc03d6928
sentry-provider/sentry-provider-db/pom.xml b8143ffa3adca9e47e7cb092131d65064d57c86b
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java PRE-CREATION
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryKerberosContext.java f54f161b381088285486a5ca74972f93ee620547
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java 531ab3545e1beb79c18e713f9d97453ee258748d
Diff: https://reviews.apache.org/r/50182/diff/
Testing
-------
Thanks,
Hao Hao