[jira] [Created] (IMPALA-8853) Try to avoid leaking ACID locks/transactions

["Csaba Ringhofer (JIRA)" <ji...@apache.org>]

Csaba Ringhofer created IMPALA-8853:
---------------------------------------

             Summary: Try to avoid leaking ACID locks/transactions
                 Key: IMPALA-8853
                 URL: https://issues.apache.org/jira/browse/IMPALA-8853
             Project: IMPALA
          Issue Type: Improvement
          Components: Frontend
    Affects Versions: Impala 3.3.0
            Reporter: Csaba Ringhofer


Impala can leak locks/transactions in the following ways:
1. commit/abort fails (e.g. because HMS is down) - currently Impala does not have a re-try logic for these cases
2. Impala exits (e.g. crash or SIGTERM), and doesn't close it's open locks/transactions

Hive has a timeout logic that should drop locks/transactions if there was no heartbeat for them for some time. I think that Impala should not rely on this completely though, because:
- the timeout time can be too long for some workflows, especially if there are exclusive locks involved
- timeout logic doesn't seem to work in the Impala minicluster

Leak type 1. could be solved by periodically trying to abort transactions where commit/abort failed, similarly how heartbeating works now

Leak type 2. could be solved by aborting open locks/transactions that were opened by the given impalad/catalogd role during start up, so the what is leaked during exit would be cleared during restart. This would need impalad/catalogd to use unique "user" fields when acquiring locks/transactions instead of the current "impala".

Note that the transaction opened by INSERT seem problematic, as these are opened and possibly aborted by the coordinator, but commit happens on catalogd.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org