[jira] [Updated] (HIVE-23236) Remove the global lock from acquireLock

["Marton Bod (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-23236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Marton Bod updated HIVE-23236:
------------------------------
    Description: 
Currently we have a global lock (NEXT_LOCK_ID) when running enqueueLock, because the algorithm in checkLock requires the locks to have a well defined order, and also requires that every lock component is already stored in the RDBMS before checking the locks.

Proposed approach:
 * Enqueue locks without a global S4U lock, using an auto-incremented value instead to get the next lock ID (modify next_lock_id table)
 * Before checking for lock conflicts, issue a S4U for all lock components we are requesting (db/table/partition level) in order to prevent other clients for doing interleaving conflict check (especially late-coming clients holding a lower lockID)
 * Conflict check algorithm to check not just for lower lockIDs, but also higher lockIDs which are already in 'acquired' state

  was:
Currently we have a global lock (NEXT_LOCK_ID) when running enqueueLock, because the algorithm in checkLock requires the locks to have a well defined order, and also requires that every lock component is already stored in the RDBMS before checking the locks.

Proposed approach:
 * Enqueue lock without a global lock, using a sequence for getting the next lock ID
 * Insert the locks as usual, but with an additional timestamp (enqueued_at)
 * Check lock should check all enqueued locks for the given db/table/partition but order should be determined based on the enqueued_at time instead of the lockID
 * If there is no blocking lock, update the state to acquired as usual


> Remove the global lock from acquireLock
> ---------------------------------------
>
>                 Key: HIVE-23236
>                 URL: https://issues.apache.org/jira/browse/HIVE-23236
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Marton Bod
>            Assignee: Marton Bod
>            Priority: Major
>
> Currently we have a global lock (NEXT_LOCK_ID) when running enqueueLock, because the algorithm in checkLock requires the locks to have a well defined order, and also requires that every lock component is already stored in the RDBMS before checking the locks.
> Proposed approach:
>  * Enqueue locks without a global S4U lock, using an auto-incremented value instead to get the next lock ID (modify next_lock_id table)
>  * Before checking for lock conflicts, issue a S4U for all lock components we are requesting (db/table/partition level) in order to prevent other clients for doing interleaving conflict check (especially late-coming clients holding a lower lockID)
>  * Conflict check algorithm to check not just for lower lockIDs, but also higher lockIDs which are already in 'acquired' state



--
This message was sent by Atlassian Jira
(v8.3.4#803005)