You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by GitBox <gi...@apache.org> on 2021/10/19 19:36:00 UTC
[GitHub] [storm] agresch opened a new pull request #3417: STORM-3801 add option for gauges to report values per reporter if sup…
agresch opened a new pull request #3417:
URL: https://github.com/apache/storm/pull/3417
…ported
## What is the purpose of the change
newWorkerEvent only works properly for the first reporter using it. Added PerReporterGauge to allow reporters who want to support using it to add a reporter-specific version to get the value. Other gauges can use this going forward.
## How was the change tested
Ran storm-client unit tests. Ran topology and validated logging works for topology.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@storm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [storm] agresch merged pull request #3417: STORM-3801 add option for gauges to report values per reporter if sup…
Posted by GitBox <gi...@apache.org>.
agresch merged pull request #3417:
URL: https://github.com/apache/storm/pull/3417
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@storm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [storm] agresch commented on a change in pull request #3417: STORM-3801 add option for gauges to report values per reporter if sup…
Posted by GitBox <gi...@apache.org>.
agresch commented on a change in pull request #3417:
URL: https://github.com/apache/storm/pull/3417#discussion_r732236032
##########
File path: storm-client/src/jvm/org/apache/storm/metric/SystemBolt.java
##########
@@ -63,33 +65,46 @@ public Long getValue() {
}
});
- // newWorkerEvent: 1 when a worker is first started and 0 all other times.
- // This can be used to tell when a worker has crashed and is restarted.
- final IMetric newWorkerEvent = new IMetric() {
- boolean doEvent = true;
- @Override
- public Object getValueAndReset() {
- if (doEvent) {
- doEvent = false;
- return 1;
- } else {
- return 0;
- }
- }
- };
- context.registerGauge("newWorkerEvent", new Gauge<Integer>() {
- @Override
- public Integer getValue() {
- return (Integer) newWorkerEvent.getValueAndReset();
- }
- });
+ context.registerGauge("newWorkerEvent", new NewWorkerGauge());
int bucketSize = ObjectReader.getInt(topoConf.get(Config.TOPOLOGY_BUILTIN_METRICS_BUCKET_SIZE_SECS));
registerMetrics(context, (Map<String, String>) topoConf.get(Config.WORKER_METRICS), bucketSize, topoConf);
registerMetrics(context, (Map<String, String>) topoConf.get(Config.TOPOLOGY_WORKER_METRICS), bucketSize, topoConf);
}
+ // newWorkerEvent: 1 when a worker is first started and 0 all other times.
+ // This can be used to tell when a worker has crashed and is restarted.
+ private class NewWorkerImetric implements IMetric {
+ boolean doEvent = true;
+
+ @Override
+ public Object getValueAndReset() {
+ if (doEvent) {
+ doEvent = false;
+ return 1;
+ } else {
+ return 0;
+ }
+ }
+ }
+
+ // allow reporting new worker metric for multiple reporters if they support getValueForReporter().
+ private class NewWorkerGauge extends PerReporterGauge<Integer> {
+ private final NewWorkerImetric defaultValue = new NewWorkerImetric();
+ private final Map<Object, NewWorkerImetric> reporterValues = new HashMap<>();
+
+ @Override
+ public Integer getValue() {
+ return (Integer) defaultValue.getValueAndReset();
+ }
Review comment:
getValueForReporter() should be called when possible, but reporters need to know that this gauge is of this class to do this. So a codahale ConsoleReporter won't be calling this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@storm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [storm] agresch commented on a change in pull request #3417: STORM-3801 add option for gauges to report values per reporter if sup…
Posted by GitBox <gi...@apache.org>.
agresch commented on a change in pull request #3417:
URL: https://github.com/apache/storm/pull/3417#discussion_r733126693
##########
File path: storm-client/src/jvm/org/apache/storm/metric/SystemBolt.java
##########
@@ -63,33 +65,46 @@ public Long getValue() {
}
});
- // newWorkerEvent: 1 when a worker is first started and 0 all other times.
- // This can be used to tell when a worker has crashed and is restarted.
- final IMetric newWorkerEvent = new IMetric() {
- boolean doEvent = true;
- @Override
- public Object getValueAndReset() {
- if (doEvent) {
- doEvent = false;
- return 1;
- } else {
- return 0;
- }
- }
- };
- context.registerGauge("newWorkerEvent", new Gauge<Integer>() {
- @Override
- public Integer getValue() {
- return (Integer) newWorkerEvent.getValueAndReset();
- }
- });
+ context.registerGauge("newWorkerEvent", new NewWorkerGauge());
int bucketSize = ObjectReader.getInt(topoConf.get(Config.TOPOLOGY_BUILTIN_METRICS_BUCKET_SIZE_SECS));
registerMetrics(context, (Map<String, String>) topoConf.get(Config.WORKER_METRICS), bucketSize, topoConf);
registerMetrics(context, (Map<String, String>) topoConf.get(Config.TOPOLOGY_WORKER_METRICS), bucketSize, topoConf);
}
+ // newWorkerEvent: 1 when a worker is first started and 0 all other times.
+ // This can be used to tell when a worker has crashed and is restarted.
+ private class NewWorkerImetric implements IMetric {
Review comment:
I will put up a new PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@storm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [storm] Ethanlm commented on a change in pull request #3417: STORM-3801 add option for gauges to report values per reporter if sup…
Posted by GitBox <gi...@apache.org>.
Ethanlm commented on a change in pull request #3417:
URL: https://github.com/apache/storm/pull/3417#discussion_r733111555
##########
File path: storm-client/src/jvm/org/apache/storm/metric/SystemBolt.java
##########
@@ -63,33 +65,46 @@ public Long getValue() {
}
});
- // newWorkerEvent: 1 when a worker is first started and 0 all other times.
- // This can be used to tell when a worker has crashed and is restarted.
- final IMetric newWorkerEvent = new IMetric() {
- boolean doEvent = true;
- @Override
- public Object getValueAndReset() {
- if (doEvent) {
- doEvent = false;
- return 1;
- } else {
- return 0;
- }
- }
- };
- context.registerGauge("newWorkerEvent", new Gauge<Integer>() {
- @Override
- public Integer getValue() {
- return (Integer) newWorkerEvent.getValueAndReset();
- }
- });
+ context.registerGauge("newWorkerEvent", new NewWorkerGauge());
int bucketSize = ObjectReader.getInt(topoConf.get(Config.TOPOLOGY_BUILTIN_METRICS_BUCKET_SIZE_SECS));
registerMetrics(context, (Map<String, String>) topoConf.get(Config.WORKER_METRICS), bucketSize, topoConf);
registerMetrics(context, (Map<String, String>) topoConf.get(Config.TOPOLOGY_WORKER_METRICS), bucketSize, topoConf);
}
+ // newWorkerEvent: 1 when a worker is first started and 0 all other times.
+ // This can be used to tell when a worker has crashed and is restarted.
+ private class NewWorkerImetric implements IMetric {
+ boolean doEvent = true;
+
+ @Override
+ public Object getValueAndReset() {
+ if (doEvent) {
+ doEvent = false;
+ return 1;
+ } else {
+ return 0;
+ }
+ }
+ }
+
+ // allow reporting new worker metric for multiple reporters if they support getValueForReporter().
+ private class NewWorkerGauge extends PerReporterGauge<Integer> {
+ private final NewWorkerImetric defaultValue = new NewWorkerImetric();
+ private final Map<Object, NewWorkerImetric> reporterValues = new HashMap<>();
+
+ @Override
+ public Integer getValue() {
+ return (Integer) defaultValue.getValueAndReset();
Review comment:
This can be confusing when this is invoked (by multiple reporters who don't recognize PerReporterGauge). Thinking we should add some comment at least
##########
File path: storm-client/src/jvm/org/apache/storm/metric/SystemBolt.java
##########
@@ -63,33 +65,46 @@ public Long getValue() {
}
});
- // newWorkerEvent: 1 when a worker is first started and 0 all other times.
- // This can be used to tell when a worker has crashed and is restarted.
- final IMetric newWorkerEvent = new IMetric() {
- boolean doEvent = true;
- @Override
- public Object getValueAndReset() {
- if (doEvent) {
- doEvent = false;
- return 1;
- } else {
- return 0;
- }
- }
- };
- context.registerGauge("newWorkerEvent", new Gauge<Integer>() {
- @Override
- public Integer getValue() {
- return (Integer) newWorkerEvent.getValueAndReset();
- }
- });
+ context.registerGauge("newWorkerEvent", new NewWorkerGauge());
int bucketSize = ObjectReader.getInt(topoConf.get(Config.TOPOLOGY_BUILTIN_METRICS_BUCKET_SIZE_SECS));
registerMetrics(context, (Map<String, String>) topoConf.get(Config.WORKER_METRICS), bucketSize, topoConf);
registerMetrics(context, (Map<String, String>) topoConf.get(Config.TOPOLOGY_WORKER_METRICS), bucketSize, topoConf);
}
+ // newWorkerEvent: 1 when a worker is first started and 0 all other times.
+ // This can be used to tell when a worker has crashed and is restarted.
+ private class NewWorkerImetric implements IMetric {
Review comment:
This has nothing to do with IMetric anymore. Feel like we should simplify it since we are here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@storm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [storm] bipinprasad commented on a change in pull request #3417: STORM-3801 add option for gauges to report values per reporter if sup…
Posted by GitBox <gi...@apache.org>.
bipinprasad commented on a change in pull request #3417:
URL: https://github.com/apache/storm/pull/3417#discussion_r732214003
##########
File path: storm-client/src/jvm/org/apache/storm/metric/SystemBolt.java
##########
@@ -63,33 +65,46 @@ public Long getValue() {
}
});
- // newWorkerEvent: 1 when a worker is first started and 0 all other times.
- // This can be used to tell when a worker has crashed and is restarted.
- final IMetric newWorkerEvent = new IMetric() {
- boolean doEvent = true;
- @Override
- public Object getValueAndReset() {
- if (doEvent) {
- doEvent = false;
- return 1;
- } else {
- return 0;
- }
- }
- };
- context.registerGauge("newWorkerEvent", new Gauge<Integer>() {
- @Override
- public Integer getValue() {
- return (Integer) newWorkerEvent.getValueAndReset();
- }
- });
+ context.registerGauge("newWorkerEvent", new NewWorkerGauge());
int bucketSize = ObjectReader.getInt(topoConf.get(Config.TOPOLOGY_BUILTIN_METRICS_BUCKET_SIZE_SECS));
registerMetrics(context, (Map<String, String>) topoConf.get(Config.WORKER_METRICS), bucketSize, topoConf);
registerMetrics(context, (Map<String, String>) topoConf.get(Config.TOPOLOGY_WORKER_METRICS), bucketSize, topoConf);
}
+ // newWorkerEvent: 1 when a worker is first started and 0 all other times.
+ // This can be used to tell when a worker has crashed and is restarted.
+ private class NewWorkerImetric implements IMetric {
+ boolean doEvent = true;
+
+ @Override
+ public Object getValueAndReset() {
+ if (doEvent) {
+ doEvent = false;
+ return 1;
+ } else {
+ return 0;
+ }
+ }
+ }
+
+ // allow reporting new worker metric for multiple reporters if they support getValueForReporter().
+ private class NewWorkerGauge extends PerReporterGauge<Integer> {
+ private final NewWorkerImetric defaultValue = new NewWorkerImetric();
+ private final Map<Object, NewWorkerImetric> reporterValues = new HashMap<>();
+
+ @Override
+ public Integer getValue() {
+ return (Integer) defaultValue.getValueAndReset();
+ }
Review comment:
Should this be ever called?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@storm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [storm] Ethanlm commented on a change in pull request #3417: STORM-3801 add option for gauges to report values per reporter if sup…
Posted by GitBox <gi...@apache.org>.
Ethanlm commented on a change in pull request #3417:
URL: https://github.com/apache/storm/pull/3417#discussion_r733133132
##########
File path: storm-client/src/jvm/org/apache/storm/metric/SystemBolt.java
##########
@@ -63,33 +65,46 @@ public Long getValue() {
}
});
- // newWorkerEvent: 1 when a worker is first started and 0 all other times.
- // This can be used to tell when a worker has crashed and is restarted.
- final IMetric newWorkerEvent = new IMetric() {
- boolean doEvent = true;
- @Override
- public Object getValueAndReset() {
- if (doEvent) {
- doEvent = false;
- return 1;
- } else {
- return 0;
- }
- }
- };
- context.registerGauge("newWorkerEvent", new Gauge<Integer>() {
- @Override
- public Integer getValue() {
- return (Integer) newWorkerEvent.getValueAndReset();
- }
- });
+ context.registerGauge("newWorkerEvent", new NewWorkerGauge());
int bucketSize = ObjectReader.getInt(topoConf.get(Config.TOPOLOGY_BUILTIN_METRICS_BUCKET_SIZE_SECS));
registerMetrics(context, (Map<String, String>) topoConf.get(Config.WORKER_METRICS), bucketSize, topoConf);
registerMetrics(context, (Map<String, String>) topoConf.get(Config.TOPOLOGY_WORKER_METRICS), bucketSize, topoConf);
}
+ // newWorkerEvent: 1 when a worker is first started and 0 all other times.
+ // This can be used to tell when a worker has crashed and is restarted.
+ private class NewWorkerImetric implements IMetric {
Review comment:
Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@storm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org