[GitHub] [flink] Myasuka commented on issue #10466: [FLINK-15110][metrics] Support variable exclusion

[GitBox <gi...@apache.org>]

Myasuka commented on issue #10466: [FLINK-15110][metrics] Support variable exclusion
URL: https://github.com/apache/flink/pull/10466#issuecomment-562820301
 
 
   @zentol I agreed that this PR is a solution to solve problems like FLINK-13418. However, I don't think this is the best solution.
   
   Before offering my thoughts in detail, I would prefer to review current Metrics in Flink: there is no explicit concept of `tag` for [Flink metrics](https://ci.apache.org/projects/flink/flink-docs-master/monitoring/metrics.html#metric-types), and we only have `scope` and `variables` now. 
   
   For operator level metrics `metrics.scope.operator`, we have default scope as `<host>.taskmanager.<tm_id>.<job_name>.<operator_name>.<subtask_index>`, and take Slf4jReporter for example, we would get metrics like `localhost.taskmanager.792eb90fd3c67416f7e4509bb4c4bb8b.General purpose test job.ArtificalKeyedStateMapper_Kryo_and_Custom_Stateful.0.numRecordsOut`. As you can see, no `task_attempt_id` would be printed. Only if user add `task_attempt_id` to the scope format, then the Slf4jReporter could print the attempt id out.
   
   Based on this, I think we could match the concepts of `system scope` with `system variables` to solve problem like FLINK-13418. Only system variables in defined `system scopes` would be emitted, and user variables are not limited with these scopes.
   
   Last but not least, if we take this PR, how could user achieve goals like adding `task_attempt_id` in task level metrics but avoid them in operator level metrics? 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services