You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Oliver Draese (JIRA)" <ji...@apache.org> on 2019/02/01 19:43:08 UTC
[jira] [Created] (HIVE-21204) Instrumentation for read/write locks
in LLAP
Oliver Draese created HIVE-21204:
------------------------------------
Summary: Instrumentation for read/write locks in LLAP
Key: HIVE-21204
URL: https://issues.apache.org/jira/browse/HIVE-21204
Project: Hive
Issue Type: Improvement
Components: llap
Reporter: Oliver Draese
Assignee: Oliver Draese
LLAP has several R/W locks for serialization of updates to query tracker, file data, ....
Instrumentation is added to monitor the
* total amount of R/W locks within a particular category
* average + max wait/suspension time to get the R/W lock
A category includes all lock instances for particular areas (i.e. category is FileData and all R/W locks that are used in FileData instances are accounted within the one category).
The monitoring/accounting is done via Hadoop Metrics 2, making them accessible via JMX. In addition, a new "locking" GET endpoint is added to the LLAP daemon's REST interface. It produces output like the following example:
{{{}}
{{ "statsCollection": "enabled",}}
{{ "lockStats": [}}
{{ {}}{{ "type": "R/W Lock Stats",}}
{{ "label": "FileData",}}
{{ "totalLockWaitTimeMillis": 0,}}
{{ "readLock": {}}
{{ "count": 0,}}
{{ "avgWaitTimeNanos": 0,}}
{{ "maxWaitTimeNanos": 0}}
{{ },}}
{{ "writeLock": {}}
{{ "count": 0,}}
{{ "avgWaitTimeNanos": 0,}}
{{ "maxWaitTimeNanos": 0}}
{{ }}}
{{ },}}
{{ { "}}{{type": "R/W Lock Stats",}}
{{ "label": "QueryTracker",}}
{{ "totalLockWaitTimeMillis": 0,}}
{{ "readLock": {}}
{{ "count": 0,}}
{{ "avgWaitTimeNanos": 0,}}
{{ "maxWaitTimeNanos": 0}}
{{ },}}
{{ "writeLock": {}}
{{ "count": 0,}}
{{ "avgWaitTimeNanos": 0,}}
{{ "maxWaitTimeNanos": 0}}
{{ }}}
{{ } }}{{]}}
{{ }}}
{{}}}
To avoid the overhead of lock instrumentation, lock metrics collection is disabled by default and can be enabled via the following configuration parameter:
{{hive.llap.lockmetrics.collect = true}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)