[jira] [Created] (SENTRY-2491) Sentry High availability unit tests run into deadlock sometimes

["Na Li (JIRA)" <ji...@apache.org>]

Na Li created SENTRY-2491:
-----------------------------

             Summary: Sentry High availability unit tests run into deadlock sometimes
                 Key: SENTRY-2491
                 URL: https://issues.apache.org/jira/browse/SENTRY-2491
             Project: Sentry
          Issue Type: Bug
          Components: Sentry
    Affects Versions: 2.2.0
            Reporter: Na Li
            Assignee: Na Li


In sentry unit tests, we don't create schema before running a test. Instead, we use dataNucleus to create sentry tables when they are accessed. This creates potential deadlock when running test for Sentry HA setup. 

For example, the following shows the event sequence that causes deadlock
1) thread_1 gets shared lock of SYSTABLES in order to read table SENTRY_HMS_NOTIFICATION_ID
2) thread_2 gets shared lock of SYSTABLES in order to read table SENTRY_HMS_NOTIFICATION_ID
3) thread_1 tries to get execution lock to create table SENTRY_HMS_NOTIFICATION_ID,
   and wait for execution lock because thread_2 got shared lock already.
4) thread_2 tries to get execution lock to create table SENTRY_HMS_NOTIFICATION_ID,
   and wait for execution lock because thread_1 got shared lock already.

The solution is to let the instances of sentry service start with delay. Specifically,
let HMS follower threads separate as far as possible, i.e., half of the interval.
      
This deadlock only exists in unit tests, and does not exist in production because schema is created before starting Sentry services. Therefore, there is no table creation after service starts.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)