[jira] [Created] (IMPALA-7909) Consider instrumenting locks

["Tim Armstrong (JIRA)" <ji...@apache.org>]

Tim Armstrong created IMPALA-7909:
-------------------------------------

             Summary: Consider instrumenting locks
                 Key: IMPALA-7909
                 URL: https://issues.apache.org/jira/browse/IMPALA-7909
             Project: IMPALA
          Issue Type: Improvement
          Components: Backend
            Reporter: Tim Armstrong


This isn't a fully baked idea but could be very helpful with diagnosing issues around lock contention and lock order.

It would be nice to be able to identify the following things:
* Highly contended locks, based on how often threads get blocked and how long they get blocked for. E.g. [~thundergun] was looking at an issue that looked like IMPALA-4456 but we had no way to confirm.
* Validation of lock order by defining a static lock order and then enforcing it with DCHECKs
* Detection of deadlocks or livelocks involving multiple locks. This requires a graph of which locks are held or waited for by the different threads.
* Some kind of per-query metrics for the profile, e.g. the number of times threads got blocked on locks.

The difficulty is doing this with low overhead and integrating with the lock implementations that we currently use (boost::mutex, boost::shared_mutex and SpinLock). Maybe it's best to do it incrementally or selectively only for locks that we suspect will be contention points.

Abseil (Apache licensed) has some of this functionality: https://github.com/abseil/abseil-cpp/blob/master/absl/synchronization/mutex.cc



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)