[jira] [Commented] (SENTRY-2300) Move Permission Update due to DDL to HMS Post Event Listener

["kalyan kumar kalvagadda (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SENTRY-2300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16609193#comment-16609193 ] 

kalyan kumar kalvagadda commented on SENTRY-2300:
-------------------------------------------------

[~LinaAtAustin]

Conceptually SENTRY-2300 will address the 0.5.sec delay in fetching the notifications but this approach significantly changes the current way of fetching changes to HMS data and the changes are not trivial.  We need to look very closely to understand all the effects of doing so.

*Let’s also understand the limitations with the current approach. These are the issues i see.*
 # There would 0.5 sec delay in fetching the notifications.
 # Missing permissions updates for the HMS changes that happened while Sentry is fetching full snapshot.[As explained in the comment above]

[1] and [2] can be addressed with some trivial fixes.

 

*Here are some of the issues that i see with the approach proposed in SENTRY-2300*
 # When there is a failure in sentry server while processing an event, event is lost for ever. Failures could be any internal failures. This is the not case currently. If there is a failure in persisting the notification, it is not lost it will be fetches in the subsequent attempt. 
 # We still need to depend on HMS notifications to update the Full snapshot with the HMS changes that happened while fetching the snapshot.

> Move Permission Update due to DDL to HMS Post Event Listener
> ------------------------------------------------------------
>
>                 Key: SENTRY-2300
>                 URL: https://issues.apache.org/jira/browse/SENTRY-2300
>             Project: Sentry
>          Issue Type: Bug
>          Components: Sentry
>    Affects Versions: 2.1.0
>            Reporter: Na Li
>            Assignee: Na Li
>            Priority: Major
>
> There was a code in MetastorePlugin that modified Sentry privileges on table Create/Drop and database Create/Drop. As part of Sentry HA work we moved all this logic from Sentry plugin to be driven by notifications which required the extra synchronization between HMS and Sentry.
> It should be possible to do permission changes in the post event listener itself to avoid blocking for Sentry. This requires some experiments though because it may cause strange artifacts since at the time these DDL operations are done Sentry may not be aware of the current state - for example you may try to change permissions of a table that Sentry doesn’t know about, which seems to be OK. 
> This update will have the following benefits:
> {code}
> * HMS waits on Sentry polling HMS update takes 0.5 to 1 second. This update will remove this delay
> * Sentry knows every DDL update, and therefore can update permission correctly. In current approach using notification processing, Sentry could miss updates if full snapshot is fetched from HMS, and permission is not updated correctly. In the case of table rename, when mission DDL update event because of full snapshot, sentry will not move the permissions associated with old table to the new table. And the authorization on queries on the renamed table will fail.
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)