You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@sentry.apache.org by "Alexander Kolbasov (JIRA)" <ji...@apache.org> on 2017/06/05 23:53:12 UTC
[jira] [Comment Edited] (SENTRY-1792) Permissions table may have
holes
[ https://issues.apache.org/jira/browse/SENTRY-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16037796#comment-16037796 ]
Alexander Kolbasov edited comment on SENTRY-1792 at 6/5/17 11:52 PM:
---------------------------------------------------------------------
The possible cause is when multiple threads are updating perm change table (multiple requests for changes are issued through hive, and go to sentry service, and they can be in its own thread), thread_a get changeID_a from DB, processing entry, and then save it as change entry_A in DB. thread_b get changeID_b from DB, processing entry, and then save it as change entry_B in DB. If thread_A gets changeID_b before thread_B gets changeID_b, but finishes after thread_B, then we can see the hole at short time interval. eventually, we don't see the hole. Vamsee and I checked the DB changeID, they are continuous even after several days run.
fix is to extract continuous list from non-continuous list starting with requested ID.
{code}
Examples of the change list. the temporary hole is always at the end of the list
2017-06-05 14:02:42,489 ERROR org.apache.sentry.provider.db.service.persistent.SentryStore: Certain perm delta is missing in SENTRY_PERM_CHANEG table! Current size of elements = 5 and expected size = 7, from changeID: 1078683. The table may get corrupted. perm changeID list: 1078683,1078684,1078685,1078686,1078689
2017-06-05 14:03:01,669 ERROR org.apache.sentry.provider.db.service.persistent.SentryStore: Certain perm delta is missing in SENTRY_PERM_CHANEG table! Current size of elements = 5 and expected size = 6, from changeID: 1078958. The table may get corrupted. perm changeID list: 1078958,1078959,1078960,1078961,1078963
2017-06-05 14:03:14,645 ERROR org.apache.sentry.provider.db.service.persistent.SentryStore: Certain perm delta is missing in SENTRY_PERM_CHANEG table! Current size of elements = 8 and expected size = 9, from changeID: 1079144. The table may get corrupted. perm changeID list: 1079144,1079145,1079146,1079147,1079148,1079149,1079150,1079152
2017-06-05 14:03:26,735 ERROR org.apache.sentry.provider.db.service.persistent.SentryStore: Certain perm delta is missing in SENTRY_PERM_CHANEG table! Current size of elements = 7 and expected size = 8, from changeID: 1079324. The table may get corrupted. perm changeID list: 1079324,1079325,1079326,1079327,1079328,1079329,1079331
{code}
was (Author: linaataustin):
The possible cause is when multiple threads are updating perm change table (multiple requests for changes are issued through hive, and go to sentry service, and they can be in its own thread), thread_a get changeID_a from DB, processing entry, and then save it as change entry_A in DB. thread_b get changeID_b from DB, processing entry, and then save it as change entry_B in DB. If thread_A gets changeID_b before thread_B gets changeID_b, but finishes after thread_B, then we can see the hole at short time interval. eventually, we don't see the hole. Vamsee and I checked the DB changeID, they are continuous even after several days run.
fix is to extract continuous list from non-continuous list starting with requested ID.
Examples of the change list. the temporary hole is always at the end of the list
2017-06-05 14:02:42,489 ERROR org.apache.sentry.provider.db.service.persistent.SentryStore: Certain perm delta is missing in SENTRY_PERM_CHANEG table! Current size of elements = 5 and expected size = 7, from changeID: 1078683. The table may get corrupted. perm changeID list: 1078683,1078684,1078685,1078686,1078689
2017-06-05 14:03:01,669 ERROR org.apache.sentry.provider.db.service.persistent.SentryStore: Certain perm delta is missing in SENTRY_PERM_CHANEG table! Current size of elements = 5 and expected size = 6, from changeID: 1078958. The table may get corrupted. perm changeID list: 1078958,1078959,1078960,1078961,1078963
2017-06-05 14:03:14,645 ERROR org.apache.sentry.provider.db.service.persistent.SentryStore: Certain perm delta is missing in SENTRY_PERM_CHANEG table! Current size of elements = 8 and expected size = 9, from changeID: 1079144. The table may get corrupted. perm changeID list: 1079144,1079145,1079146,1079147,1079148,1079149,1079150,1079152
2017-06-05 14:03:26,735 ERROR org.apache.sentry.provider.db.service.persistent.SentryStore: Certain perm delta is missing in SENTRY_PERM_CHANEG table! Current size of elements = 7 and expected size = 8, from changeID: 1079324. The table may get corrupted. perm changeID list: 1079324,1079325,1079326,1079327,1079328,1079329,1079331
> Permissions table may have holes
> --------------------------------
>
> Key: SENTRY-1792
> URL: https://issues.apache.org/jira/browse/SENTRY-1792
> Project: Sentry
> Issue Type: Sub-task
> Components: Sentry
> Affects Versions: sentry-ha-redesign
> Reporter: Na Li
> Labels: sentry-ha
> Attachments: SENTRY-1792.001-sentry-ha-redesign.patch
>
>
> Sentry server returns full permission snapshot almost every 10 seconds. The desired behavior is to reduce the chance of getting full snapshot significantly.
> From debug message, we can see the returned change list is empty.
> That causes Sentry to send back full snapshot instead of delta changes. It has performance penalty because it takes much longer to get a full snapshot than getting the delta changes.
> {code}
> 2017-06-05 14:00:21,030 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1076942, curSeqNum:1076948 ; type: PermImageRetriever
> 2017-06-05 14:00:31,905 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1076949, curSeqNum:1077064 ; type: PermImageRetriever
> 2017-06-05 14:01:00,041 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1077354, curSeqNum:1077361 ; type: PermImageRetriever
> 2017-06-05 14:01:25,735 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1077630, curSeqNum:1077636 ; type: PermImageRetriever
> 2017-06-05 14:01:42,197 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1077820, curSeqNum:1077826 ; type: PermImageRetriever
> 2017-06-05 14:01:53,491 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1077827, curSeqNum:1077962 ; type: PermImageRetriever
> 2017-06-05 14:02:09,472 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1078147, curSeqNum:1078155 ; type: PermImageRetriever
> 2017-06-05 14:02:21,527 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1078300, curSeqNum:1078305 ; type: PermImageRetriever
> 2017-06-05 14:02:35,761 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1078483, curSeqNum:1078489 ; type: PermImageRetriever
> 2017-06-05 14:02:50,079 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1078683, curSeqNum:1078686 ; type: PermImageRetriever
> 2017-06-05 14:03:09,889 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1078958, curSeqNum:1078961 ; type: PermImageRetriever
> 2017-06-05 14:03:21,641 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1079144, curSeqNum:1079150 ; type: PermImageRetriever
> 2017-06-05 14:03:32,600 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1079324, curSeqNum:1079331 ; type: PermImageRetriever
> 2017-06-05 14:03:46,223 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1079537, curSeqNum:1079546 ; type: PermImageRetriever
> 2017-06-05 14:03:59,870 INFO org.apache.sentry.hdfs.DBUpdateForwarder: #### GetAllUpdatesFrom return full image with isDeltaAvailable:true, isDeltaEmpty:true, reqSeqNum:1079762, curSeqNum:1079769 ; type: PermImageRetriever
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)