You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gobblin.apache.org by GitBox <gi...@apache.org> on 2021/08/16 19:44:24 UTC
[GitHub] [gobblin] sv2000 commented on a change in pull request #3365: [GOBBLIN-1516] Only schedule TaskMetricsUpdateder when metric-reporting is enabled by config
sv2000 commented on a change in pull request #3365:
URL: https://github.com/apache/gobblin/pull/3365#discussion_r689810471
##########
File path: gobblin-cluster/src/main/java/org/apache/gobblin/cluster/GobblinHelixTaskStateTracker.java
##########
@@ -61,7 +61,9 @@ public GobblinHelixTaskStateTracker(Properties properties) {
@Override
public void registerNewTask(Task task) {
try {
- this.scheduledReporters.put(task.getTaskId(), scheduleTaskMetricsUpdater(new TaskMetricsUpdater(task), task));
+ if (GobblinMetrics.isEnabled(task.getTaskState().getWorkunit())) {
+ this.scheduledReporters.put(task.getTaskId(), scheduleTaskMetricsUpdater(new TaskMetricsUpdater(task), task));
Review comment:
I think a better approach might be to not create one metrics updater per task, and create a single updater instead, which iterates over all the tasks and updates records/bytes written.
##########
File path: gobblin-cluster/src/main/java/org/apache/gobblin/cluster/GobblinHelixTaskStateTracker.java
##########
@@ -61,7 +61,9 @@ public GobblinHelixTaskStateTracker(Properties properties) {
@Override
public void registerNewTask(Task task) {
try {
- this.scheduledReporters.put(task.getTaskId(), scheduleTaskMetricsUpdater(new TaskMetricsUpdater(task), task));
+ if (GobblinMetrics.isEnabled(task.getTaskState().getWorkunit())) {
+ this.scheduledReporters.put(task.getTaskId(), scheduleTaskMetricsUpdater(new TaskMetricsUpdater(task), task));
Review comment:
Using the global "metrics.enabled" key to turn off task metrics reporting seems too heavy. This global config will turn off other reporting as well, e.g. GTEs, and other metrics.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org