You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2022/04/18 07:06:22 UTC
[GitHub] [druid] zachjsh opened a new pull request, #12446: Worker level task metrics
zachjsh opened a new pull request, #12446:
URL: https://github.com/apache/druid/pull/12446
Added a new monitor, `WorkerTaskCountStatsMonitor`, that allows each middle manage worker to report metrics for successful / failed tasks, and task slot usage. This monitor is only supported on MiddleManager type NodeRole. I've added a check in the `MetricsModule` class that only loads the monitor if the NodeRole is MiddleManager. Without this check, the monitor was being automatically configured by PEON services spawned by middle managers, as we copy the java properties from the parent middle manager process. Ideally, I think can be made cleaner by the monitor interface exposing a function that provides the supported druid Node types that are supported for the respective monitor, however the NodeRole class was not visible to the MetricsModule, and I thought that adding the dependency now could be a bigger change.
Also fixes an inconsistency in the name of the existing task metric for tracking taskslot usage.
This PR has:
- [x] been self-reviewed.
- [ ] using the [concurrency checklist](https://github.com/apache/druid/blob/master/dev/code-review/concurrency.md) (Remove this item if the PR doesn't have any relation to concurrency.)
- [ ] added documentation for new or modified features or behaviors.
- [x] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
- [ ] added or updated version, license, or notice information in [licenses.yaml](https://github.com/apache/druid/blob/master/dev/license.md)
- [ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
- [x] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for [code coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md) is met.
- [ ] added integration tests.
- [x] been tested in a test Druid cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] zachjsh commented on pull request #12446: Worker level task metrics
Posted by GitBox <gi...@apache.org>.
zachjsh commented on PR #12446:
URL: https://github.com/apache/druid/pull/12446#issuecomment-1101166574
Will add documentation for the new metrics to markdown docs, just wanted to check first if this approach makes sense to others.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] jon-wei commented on a diff in pull request #12446: Worker level task metrics
Posted by GitBox <gi...@apache.org>.
jon-wei commented on code in PR #12446:
URL: https://github.com/apache/druid/pull/12446#discussion_r854313316
##########
extensions-contrib/statsd-emitter/src/main/resources/defaultMetricDimensions.json:
##########
@@ -63,9 +63,15 @@
"task/pending/count" : { "dimensions" : ["dataSource"], "type" : "gauge" },
"task/waiting/count" : { "dimensions" : ["dataSource"], "type" : "gauge" },
+ "worker/task/failed/count" : { "dimensions" : ["category", "vesion"], "type" : "count" },
Review Comment:
vesion -> version
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] zachjsh commented on a diff in pull request #12446: Worker level task metrics
Posted by GitBox <gi...@apache.org>.
zachjsh commented on code in PR #12446:
URL: https://github.com/apache/druid/pull/12446#discussion_r854319913
##########
extensions-contrib/statsd-emitter/src/main/resources/defaultMetricDimensions.json:
##########
@@ -63,9 +63,15 @@
"task/pending/count" : { "dimensions" : ["dataSource"], "type" : "gauge" },
"task/waiting/count" : { "dimensions" : ["dataSource"], "type" : "gauge" },
+ "worker/task/failed/count" : { "dimensions" : ["category", "vesion"], "type" : "count" },
Review Comment:
ahhh dang lol. Good catch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] zachjsh commented on a diff in pull request #12446: Worker level task metrics
Posted by GitBox <gi...@apache.org>.
zachjsh commented on code in PR #12446:
URL: https://github.com/apache/druid/pull/12446#discussion_r854321198
##########
extensions-contrib/statsd-emitter/src/main/resources/defaultMetricDimensions.json:
##########
@@ -63,9 +63,15 @@
"task/pending/count" : { "dimensions" : ["dataSource"], "type" : "gauge" },
"task/waiting/count" : { "dimensions" : ["dataSource"], "type" : "gauge" },
+ "worker/task/failed/count" : { "dimensions" : ["category", "vesion"], "type" : "count" },
Review Comment:
Fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] jon-wei merged pull request #12446: Worker level task metrics
Posted by GitBox <gi...@apache.org>.
jon-wei merged PR #12446:
URL: https://github.com/apache/druid/pull/12446
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org