You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by GitBox <gi...@apache.org> on 2020/06/24 16:40:21 UTC
[GitHub] [storm] agresch opened a new pull request #3294: STORM-3648 add meter to track worker heartbeat rate
agresch opened a new pull request #3294:
URL: https://github.com/apache/storm/pull/3294
## What is the purpose of the change
Add a meter to allow users to track/alert on slowdown of worker heartbeat rate.
## How was the change tested
built code.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [storm] Ethanlm commented on a change in pull request #3294: STORM-3648 add meter to track worker heartbeat rate
Posted by GitBox <gi...@apache.org>.
Ethanlm commented on a change in pull request #3294:
URL: https://github.com/apache/storm/pull/3294#discussion_r445752673
##########
File path: storm-client/src/jvm/org/apache/storm/daemon/worker/Worker.java
##########
@@ -196,6 +198,8 @@ private Object loadWorker(IStateStorage stateStorage, IStormClusterState stormCl
throws Exception {
workerState = new WorkerState(conf, context, topologyId, assignmentId, supervisorIfaceSupplier, port, workerId,
topologyConf, stateStorage, stormClusterState, autoCreds, metricRegistry);
+ this.heatbeatMeter = metricRegistry.meter("doHeartbeat-calls", workerState.getWorkerTopologyContext(),
+ "__system", -1, "default");
Review comment:
Also I feel like the metrics should have the key before the value, for example, we now have
`xx.<compId>.<taskId>.yy`.
I think it should be `xx.componentId.<compId>.taskId.<taskId>.yy` so the values are more readable. But this is a separate issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [storm] agresch commented on a change in pull request #3294: STORM-3648 add meter to track worker heartbeat rate
Posted by GitBox <gi...@apache.org>.
agresch commented on a change in pull request #3294:
URL: https://github.com/apache/storm/pull/3294#discussion_r445749542
##########
File path: storm-client/src/jvm/org/apache/storm/daemon/worker/Worker.java
##########
@@ -196,6 +198,8 @@ private Object loadWorker(IStateStorage stateStorage, IStormClusterState stormCl
throws Exception {
workerState = new WorkerState(conf, context, topologyId, assignmentId, supervisorIfaceSupplier, port, workerId,
topologyConf, stateStorage, stormClusterState, autoCreds, metricRegistry);
+ this.heatbeatMeter = metricRegistry.meter("doHeartbeat-calls", workerState.getWorkerTopologyContext(),
+ "__system", -1, "default");
Review comment:
Would you prefer I pass in "" for streamId? Or are you indicating we should have a metric name convention that allows skipping inclusion of stream?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [storm] Ethanlm commented on a change in pull request #3294: STORM-3648 add meter to track worker heartbeat rate
Posted by GitBox <gi...@apache.org>.
Ethanlm commented on a change in pull request #3294:
URL: https://github.com/apache/storm/pull/3294#discussion_r445741554
##########
File path: storm-client/src/jvm/org/apache/storm/daemon/worker/Worker.java
##########
@@ -196,6 +198,8 @@ private Object loadWorker(IStateStorage stateStorage, IStormClusterState stormCl
throws Exception {
workerState = new WorkerState(conf, context, topologyId, assignmentId, supervisorIfaceSupplier, port, workerId,
topologyConf, stateStorage, stormClusterState, autoCreds, metricRegistry);
+ this.heatbeatMeter = metricRegistry.meter("doHeartbeat-calls", workerState.getWorkerTopologyContext(),
+ "__system", -1, "default");
Review comment:
It is better to not hardcode it. We can use `SYSTEM_COMPONENT_ID`, `SYSTEM_TASK_ID`. And it looks like it doesn't make much sense to have `streamId` here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [storm] agresch commented on a change in pull request #3294: STORM-3648 add meter to track worker heartbeat rate
Posted by GitBox <gi...@apache.org>.
agresch commented on a change in pull request #3294:
URL: https://github.com/apache/storm/pull/3294#discussion_r446225056
##########
File path: storm-client/src/jvm/org/apache/storm/daemon/worker/Worker.java
##########
@@ -196,6 +198,8 @@ private Object loadWorker(IStateStorage stateStorage, IStormClusterState stormCl
throws Exception {
workerState = new WorkerState(conf, context, topologyId, assignmentId, supervisorIfaceSupplier, port, workerId,
topologyConf, stateStorage, stormClusterState, autoCreds, metricRegistry);
+ this.heatbeatMeter = metricRegistry.meter("doHeartbeat-calls", workerState.getWorkerTopologyContext(),
+ "__system", -1, "default");
Review comment:
@Ethanlm - updated
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [storm] Ethanlm merged pull request #3294: STORM-3648 add meter to track worker heartbeat rate
Posted by GitBox <gi...@apache.org>.
Ethanlm merged pull request #3294:
URL: https://github.com/apache/storm/pull/3294
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [storm] Ethanlm commented on a change in pull request #3294: STORM-3648 add meter to track worker heartbeat rate
Posted by GitBox <gi...@apache.org>.
Ethanlm commented on a change in pull request #3294:
URL: https://github.com/apache/storm/pull/3294#discussion_r445751740
##########
File path: storm-client/src/jvm/org/apache/storm/daemon/worker/Worker.java
##########
@@ -196,6 +198,8 @@ private Object loadWorker(IStateStorage stateStorage, IStormClusterState stormCl
throws Exception {
workerState = new WorkerState(conf, context, topologyId, assignmentId, supervisorIfaceSupplier, port, workerId,
topologyConf, stateStorage, stormClusterState, autoCreds, metricRegistry);
+ this.heatbeatMeter = metricRegistry.meter("doHeartbeat-calls", workerState.getWorkerTopologyContext(),
+ "__system", -1, "default");
Review comment:
I see there is already a metricName function https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/metrics2/StormMetricRegistry.java#L244 which doesn't take streamId
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org