You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@sentry.apache.org by "Alexander Kolbasov (JIRA)" <ji...@apache.org> on 2017/04/30 21:46:04 UTC
[jira] [Created] (SENTRY-1740) Deadlock when handling GM privileges
Alexander Kolbasov created SENTRY-1740:
------------------------------------------
Summary: Deadlock when handling GM privileges
Key: SENTRY-1740
URL: https://issues.apache.org/jira/browse/SENTRY-1740
Project: Sentry
Issue Type: Bug
Components: Sentry
Affects Versions: 1.8.0, sentry-ha-redesign
Reporter: Alexander Kolbasov
While running Kafka TestAclsCrud e2e test I noticed a bunch of failures in the log:
{code}
2017-04-30 14:30:45,684 (Timer-1) [ERROR - org.apache.sentry.provider.db.generic.UpdatableCache.revokeAllPrivilegesIfRequired(UpdatableCache.java:157)] Failed to update roles and privileges cache for 40123 times. Revoking all privileges from cache, which will cause all authorization requests to fail.
2017-04-30 14:30:45,685 (pool-19-thread-14) [ERROR - org.apache.sentry.provider.db.generic.service.thrift.SentryGenericPolicyProcessor.requestHandle(SentryGenericPolicyProcessor.java:221)] Unknown error:Iteration request failed : SELECT 'org.apache.sentry.provider.db.service.model.MSentryGMPrivilege' AS NUCLEUS_TYPE,A1."ACTION",A1.COMPONENT_NAME,A1.CREATE_TIME,A1.WITH_GRANT_OPTION,A1.RESOURCE_NAME_0,A1.RESOURCE_NAME_1,A1.RESOURCE_NAME_2,A1.RESOURCE_NAME_3,A1.RESOURCE_TYPE_0,A1.RESOURCE_TYPE_1,A1.RESOURCE_TYPE_2,A1.RESOURCE_TYPE_3,A1."SCOPE",A1.SERVICE_NAME,A1.GM_PRIVILEGE_ID,A0.ROLE_ID FROM SENTRY_ROLE_GM_PRIVILEGE_MAP A0 INNER JOIN SENTRY_GM_PRIVILEGE A1 ON A0.GM_PRIVILEGE_ID = A1.GM_PRIVILEGE_ID WHERE EXISTS (SELECT 'org.apache.sentry.provider.db.service.model.MSentryRole' AS NUCLEUS_TYPE,A0_SUB.ROLE_ID AS DN_DATASTOREID FROM SENTRY_ROLE A0_SUB WHERE A0.ROLE_ID = A0_SUB.ROLE_ID)
javax.jdo.JDODataStoreException: Iteration request failed : SELECT 'org.apache.sentry.provider.db.service.model.MSentryGMPrivilege' AS NUCLEUS_TYPE,A1."ACTION",A1.COMPONENT_NAME,A1.CREATE_TIME,A1.WITH_GRANT_OPTION,A1.RESOURCE_NAME_0,A1.RESOURCE_NAME_1,A1.RESOURCE_NAME_2,A1.RESOURCE_NAME_3,A1.RESOURCE_TYPE_0,A1.RESOURCE_TYPE_1,A1.RESOURCE_TYPE_2,A1.RESOURCE_TYPE_3,A1."SCOPE",A1.SERVICE_NAME,A1.GM_PRIVILEGE_ID,A0.ROLE_ID FROM SENTRY_ROLE_GM_PRIVILEGE_MAP A0 INNER JOIN SENTRY_GM_PRIVILEGE A1 ON A0.GM_PRIVILEGE_ID = A1.GM_PRIVILEGE_ID WHERE EXISTS (SELECT 'org.apache.sentry.provider.db.service.model.MSentryRole' AS NUCLEUS_TYPE,A0_SUB.ROLE_ID AS DN_DATASTOREID FROM SENTRY_ROLE A0_SUB WHERE A0.ROLE_ID = A0_SUB.ROLE_ID)
at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:230)
at org.apache.sentry.provider.db.service.persistent.SentryStore.getAllRoles(SentryStore.java:303)
at org.apache.sentry.provider.db.service.persistent.SentryStore.access$1400(SentryStore.java:101)
at org.apache.sentry.provider.db.service.persistent.SentryStore$29.execute(SentryStore.java:1610)
at org.apache.sentry.provider.db.service.persistent.SentryStore$29.execute(SentryStore.java:1604)
at org.apache.sentry.provider.db.service.persistent.TransactionManager.executeTransaction(TransactionManager.java:114)
at org.apache.sentry.provider.db.service.persistent.SentryStore.getMSentryRolesByGroupName(SentryStore.java:1603)
at org.apache.sentry.provider.db.service.persistent.SentryStore.getTSentryRolesByGroupName(SentryStore.java:1639)
at org.apache.sentry.provider.db.generic.service.persistent.DelegateSentryStore.getRolesByGroups(DelegateSentryStore.java:240)
...
{code}
{code}
java.sql.SQLTransactionRollbackException: A lock could not be obtained due to a deadlock, cycle of locks and waiters is:
Lock : ROW, SENTRY_ROLE_GM_PRIVILEGE_MAP, (1,7)
Waiting XID : {585, S} , SENTRY, SELECT 'org.apache.sentry.provider.db.service.model.MSentryGMPrivilege' AS NUCLEUS_TYPE,A1."ACTION",A1.COMPONENT_NAME,A1.CREATE_TIME,A1.WITH_GRANT_OPTION,A1.RESOURCE_NAME_0,A1.RESOURCE_NAME_1,A1.RESOURCE_NAME_2,A1.RESOURCE_NAME_3,A1.RESOURCE_TYPE_0,A1.RESOURCE_TYPE_1,A1.RESOURCE_TYPE_2,A1.RESOURCE_TYPE_3,A1."SCOPE",A1.SERVICE_NAME,A1.GM_PRIVILEGE_ID,A0.ROLE_ID FROM SENTRY_ROLE_GM_PRIVILEGE_MAP A0 INNER JOIN SENTRY_GM_PRIVILEGE A1 ON A0.GM_PRIVILEGE_ID = A1.GM_PRIVILEGE_ID WHERE EXISTS (SELECT 'org.apache.sentry.provider.db.service.model.MSentryRole' AS NUCLEUS_TYPE,A0_SUB.ROLE_ID AS DN_DATASTOREID FROM SENTRY_ROLE A0_SUB WHERE A0.ROLE_ID = A0_SUB.ROLE_ID)
Granted XID : {575, S}
Lock : ROW, SENTRY_ROLE, (1,7)
Waiting XID : {575, X} , SENTRY, DELETE FROM SENTRY_ROLE WHERE ROLE_ID=?
Granted XID : {575, S} , {583, S} , {585, S}
. The selected victim is XID : 585.
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeQuery(Unknown Source)
at com.jolbox.bonecp.PreparedStatementHandle.executeQuery(PreparedStatementHandle.java:172)
at org.datanucleus.store.rdbms.ParamLoggingPreparedStatement.executeQuery(ParamLoggingPreparedStatement.java:381)
at org.datanucleus.store.rdbms.SQLController.executeStatementQuery(SQLController.java:559)
at org.datanucleus.store.rdbms.query.JDOQLQuery.performExecute(JDOQLQuery.java:724)
at org.datanucleus.store.query.Query.executeQuery(Query.java:1786)
at org.datanucleus.store.query.Query.executeWithArray(Query.java:1672)
at org.datanucleus.store.query.Query.execute(Query.java:1654)
at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:221)
at org.apache.sentry.provider.db.service.persistent.SentryStore.getAllRoles(SentryStore.java:303)
at org.apache.sentry.provider.db.service.persistent.SentryStore.access$1400(SentryStore.java:101)
at org.apache.sentry.provider.db.service.persistent.SentryStore$29.execute(SentryStore.java:1610)
at org.apache.sentry.provider.db.service.persistent.SentryStore$29.execute(SentryStore.java:1604)
at org.apache.sentry.provider.db.service.persistent.TransactionManager.executeTransaction(TransactionManager.java:114)
at org.apache.sentry.provider.db.service.persistent.SentryStore.getMSentryRolesByGroupName(SentryStore.java:1603)
at org.apache.sentry.provider.db.service.persistent.SentryStore.getTSentryRolesByGroupName(SentryStore.java:1639)
at org.apache.sentry.provider.db.generic.service.persistent.DelegateSentryStore.getRolesByGroups(DelegateSentryStore.java:240)
{code}
{code}
Caused by: java.sql.SQLException: A lock could not be obtained due to a deadlock, cycle of locks and waiters is:
Lock : ROW, SENTRY_ROLE_GM_PRIVILEGE_MAP, (1,7)
Waiting XID : {585, S} , SENTRY, SELECT 'org.apache.sentry.provider.db.service.model.MSentryGMPrivilege' AS NUCLEUS_TYPE,A1."ACTION",A1.COMPONENT_NAME,A1.CREATE_TIME,A1.WITH_GRANT_OPTION,A1.RESOURCE_NAME_0,A1.RESOURCE_NAME_1,A1.RESOURCE_NAME_2,A1.RESOURCE_NAME_3,A1.RESOURCE_TYPE_0,A1.RESOURCE_TYPE_1,A1.RESOURCE_TYPE_2,A1.RESOURCE_TYPE_3,A1."SCOPE",A1.SERVICE_NAME,A1.GM_PRIVILEGE_ID,A0.ROLE_ID FROM SENTRY_ROLE_GM_PRIVILEGE_MAP A0 INNER JOIN SENTRY_GM_PRIVILEGE A1 ON A0.GM_PRIVILEGE_ID = A1.GM_PRIVILEGE_ID WHERE EXISTS (SELECT 'org.apache.sentry.provider.db.service.model.MSentryRole' AS NUCLEUS_TYPE,A0_SUB.ROLE_ID AS DN_DATASTOREID FROM SENTRY_ROLE A0_SUB WHERE A0.ROLE_ID = A0_SUB.ROLE_ID)
Granted XID : {575, S}
Lock : ROW, SENTRY_ROLE, (1,7)
Waiting XID : {575, X} , SENTRY, DELETE FROM SENTRY_ROLE WHERE ROLE_ID=?
Granted XID : {575, S} , {583, S} , {585, S}
. The selected victim is XID : 585.
at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
... 38 more
Caused by: ERROR 40001: A lock could not be obtained due to a deadlock, cycle of locks and waiters is:
Lock : ROW, SENTRY_ROLE_GM_PRIVILEGE_MAP, (1,7)
Waiting XID : {585, S} , SENTRY, SELECT 'org.apache.sentry.provider.db.service.model.MSentryGMPrivilege' AS NUCLEUS_TYPE,A1."ACTION",A1.COMPONENT_NAME,A1.CREATE_TIME,A1.WITH_GRANT_OPTION,A1.RESOURCE_NAME_0,A1.RESOURCE_NAME_1,A1.RESOURCE_NAME_2,A1.RESOURCE_NAME_3,A1.RESOURCE_TYPE_0,A1.RESOURCE_TYPE_1,A1.RESOURCE_TYPE_2,A1.RESOURCE_TYPE_3,A1."SCOPE",A1.SERVICE_NAME,A1.GM_PRIVILEGE_ID,A0.ROLE_ID FROM SENTRY_ROLE_GM_PRIVILEGE_MAP A0 INNER JOIN SENTRY_GM_PRIVILEGE A1 ON A0.GM_PRIVILEGE_ID = A1.GM_PRIVILEGE_ID WHERE EXISTS (SELECT 'org.apache.sentry.provider.db.service.model.MSentryRole' AS NUCLEUS_TYPE,A0_SUB.ROLE_ID AS DN_DATASTOREID FROM SENTRY_ROLE A0_SUB WHERE A0.ROLE_ID = A0_SUB.ROLE_ID)
Granted XID : {575, S}
Lock : ROW, SENTRY_ROLE, (1,7)
Waiting XID : {575, X} , SENTRY, DELETE FROM SENTRY_ROLE WHERE ROLE_ID=?
Granted XID : {575, S} , {583, S} , {585, S}
. The selected victim is XID : 585.
{code}
I think there is some problem in handling generic privileges.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)