You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@sentry.apache.org by "Arjun Mishra (JIRA)" <ji...@apache.org> on 2018/10/29 16:58:01 UTC
[jira] [Commented] (SENTRY-2437) When granting privileges a single
transaction per grant causes long delays
[ https://issues.apache.org/jira/browse/SENTRY-2437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16667445#comment-16667445 ]
Arjun Mishra commented on SENTRY-2437:
--------------------------------------
Below is the summary of the impact of having a single transaction. We are seeing improvement of over 20 times faster
{noformat}
Current Sentry Code:
HDFS Sync ON
Start Time : 2018-10-29 08:26:15,519 INFO org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor: TEST RUN: Start grant
Retry(1) : 2018-10-29 08:29:35,663 INFO org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor: TEST RUN: Start grant
Retry(2) : 2018-10-29 08:32:55,815 INFO org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor: TEST RUN: Start grant
End Time(1) : 2018-10-29 08:34:05,662 INFO org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor: TEST RUN: Completed grant
End Time(2) : 2018-10-29 08:34:06,992 INFO org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor: TEST RUN: Completed grant
End Time(3) : 2018-10-29 08:35:25,798 INFO org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor: TEST RUN: Completed grant
Total Time : 554.91 sec. After all retries were completed
HDFS Sync OFF
Start Time : 2018-10-29 08:42:13,944 INFO org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor: TEST RUN: Start grant
Retry(1) : 2018-10-29 08:45:34,141 INFO org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor: TEST RUN: Start grant
Retry(2) : 2018-10-29 08:48:54,283 INFO org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor: TEST RUN: Start grant
End Time(1) : 2018-10-29 08:49:58,434 INFO org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor: TEST RUN: Completed grant
End Time(2) : 2018-10-29 08:50:15,245 INFO org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor: TEST RUN: Completed grant
End Time(3) : 2018-10-29 08:50:18,654 INFO org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor: TEST RUN: Completed grant
Total Time : 489.059 sec. After all retries were completed
Code change with a single transaction:
HDFS Sync ON
Start Time : 2018-10-29 09:46:19,260 INFO org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor: TEST RUN: Start grant
End Time : 2018-10-29 09:46:40,767 INFO org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor: TEST RUN: Completed grant
Total Time : 26.426 sec
IMPROVEMENT : Over 20 times faster
HDFS Sync OFF
Start Time : 2018-10-29 09:53:37,795 INFO org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor: TEST RUN: Start grant
End Time : 2018-10-29 09:53:56,433 INFO org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor: TEST RUN: Completed grant
Total Time : 22.932 sec
IMPROVEMENT : Over 20 times faster
{noformat}
> When granting privileges a single transaction per grant causes long delays
> --------------------------------------------------------------------------
>
> Key: SENTRY-2437
> URL: https://issues.apache.org/jira/browse/SENTRY-2437
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
> Reporter: Arjun Mishra
> Assignee: Arjun Mishra
> Priority: Major
>
> Currently sentry creates a transaction for each TSentryPrivilege object it needs to grant. If the list of privileges is very large creating a single transaction for each significantly affects performance. This is particularly impactful for tables with large columns and if a user grants privileges to many of those columns
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)