You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "pnowojski (via GitHub)" <gi...@apache.org> on 2025/07/30 12:43:25 UTC

[PR] [FLINK-38158][metrics] FLIP-484: Add custom metric variables to operators [flink]

pnowojski opened a new pull request, #26849:
URL: https://github.com/apache/flink/pull/26849

   ## What is the purpose of the change
   
   https://cwiki.apache.org/confluence/display/FLINK/FLIP-484%3A+Add+custom+metric+variables+to+operators
   
   ## Brief change log
   
   Please check individual commit messages.
   
   ## Verifying this change
   
   This change adds it's own unit tests and has been verified and running internally.
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): (yes / **no**)
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (**yes** / no)
     - The serializers: (yes / **no** / don't know)
     - The runtime per-record code paths (performance sensitive): (yes / no ****/ don't know)
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / **no** / don't know)
     - The S3 file system connector: (yes / **no** / don't know)
   
   ## Documentation
   
     - Does this pull request introduce a new feature? (**yes** / no)
     - If yes, how is the feature documented? (not applicable / **docs** / **JavaDocs** / not documented)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-38158][metrics] FLIP-484: Add custom metric variables to operators [flink]

Posted by "Izeren (via GitHub)" <gi...@apache.org>.
Izeren commented on code in PR #26849:
URL: https://github.com/apache/flink/pull/26849#discussion_r2242837417


##########
flink-runtime/src/test/java/org/apache/flink/runtime/metrics/groups/InternalOperatorGroupTest.java:
##########
@@ -30,33 +30,30 @@
 import org.apache.flink.runtime.metrics.dump.QueryScopeInfo;
 import org.apache.flink.runtime.metrics.scope.ScopeFormat;
 import org.apache.flink.runtime.metrics.util.DummyCharacterFilter;
+import org.apache.flink.runtime.metrics.util.TestingMetricRegistry;
+
+import org.apache.flink.shaded.guava32.com.google.common.collect.ImmutableSet;

Review Comment:
   I see CI failing with:
   ```
   InternalOperatorGroupTest.java:[35,65] package [org.apache.flink.shaded.guava32.com](http://org.apache.flink.shaded.guava32.com/).google.common.collect does not exist
   ```
   
   We might need to check guava version for this PR



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-38158][metrics] FLIP-484: Add custom metric variables to operators [flink]

Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #26849:
URL: https://github.com/apache/flink/pull/26849#issuecomment-3136172876

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "bd2a5ebbd9fc3a7163e878c5750b365033e8dc61",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "bd2a5ebbd9fc3a7163e878c5750b365033e8dc61",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * bd2a5ebbd9fc3a7163e878c5750b365033e8dc61 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-38158][metrics] FLIP-484: Add custom metric variables to operators [flink]

Posted by "pnowojski (via GitHub)" <gi...@apache.org>.
pnowojski merged PR #26849:
URL: https://github.com/apache/flink/pull/26849


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-38158][metrics] FLIP-484: Add custom metric variables to operators [flink]

Posted by "pnowojski (via GitHub)" <gi...@apache.org>.
pnowojski commented on code in PR #26849:
URL: https://github.com/apache/flink/pull/26849#discussion_r2245080367


##########
docs/content.zh/docs/ops/metrics.md:
##########
@@ -465,6 +465,38 @@ counter = runtime_context
 {{< /tab >}}
 {{< /tabs >}}
 
+### Additional Variables for operators
+
+You can define custom variables that will be assigned to all metrics reported by a given operator
+using `Transformation.addMetricVariable`. For example:
+
+{{< tabs "32c0ba7f-3acd-831f-4a8b-a2de81b0126e" >}}

Review Comment:
   🤔 I dunno if there are some expectation about the format of this uid. I have no idea how are they being used also 😳 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-38158][metrics] FLIP-484: Add custom metric variables to operators [flink]

Posted by "pnowojski (via GitHub)" <gi...@apache.org>.
pnowojski commented on code in PR #26849:
URL: https://github.com/apache/flink/pull/26849#discussion_r2243187495


##########
flink-runtime/src/test/java/org/apache/flink/runtime/metrics/groups/InternalOperatorGroupTest.java:
##########
@@ -30,33 +30,30 @@
 import org.apache.flink.runtime.metrics.dump.QueryScopeInfo;
 import org.apache.flink.runtime.metrics.scope.ScopeFormat;
 import org.apache.flink.runtime.metrics.util.DummyCharacterFilter;
+import org.apache.flink.runtime.metrics.util.TestingMetricRegistry;
+
+import org.apache.flink.shaded.guava32.com.google.common.collect.ImmutableSet;

Review Comment:
   Fixed



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-38158][metrics] FLIP-484: Add custom metric variables to operators [flink]

Posted by "Izeren (via GitHub)" <gi...@apache.org>.
Izeren commented on code in PR #26849:
URL: https://github.com/apache/flink/pull/26849#discussion_r2244833855


##########
docs/content.zh/docs/ops/metrics.md:
##########
@@ -465,6 +465,38 @@ counter = runtime_context
 {{< /tab >}}
 {{< /tabs >}}
 
+### Additional Variables for operators
+
+You can define custom variables that will be assigned to all metrics reported by a given operator
+using `Transformation.addMetricVariable`. For example:
+
+{{< tabs "32c0ba7f-3acd-831f-4a8b-a2de81b0126e" >}}
+{{< tab "Java" >}}
+```java
+
+fooSource =
+  execEnv.fromSource(
+    kafkaSource,
+    getWatermarkStrategy(),
+    "KafkaSource-Foo")
+      .addMetricVariable("table_name", "Foo"");

Review Comment:
   There is an extra quote After "Foo", same below after "Bar""



##########
docs/content.zh/docs/ops/metrics.md:
##########
@@ -465,6 +465,38 @@ counter = runtime_context
 {{< /tab >}}
 {{< /tabs >}}
 
+### Additional Variables for operators
+
+You can define custom variables that will be assigned to all metrics reported by a given operator
+using `Transformation.addMetricVariable`. For example:
+
+{{< tabs "32c0ba7f-3acd-831f-4a8b-a2de81b0126e" >}}

Review Comment:
   !nit you could probably use more descriptive tag name, but I see that it is common to have uids in docs



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-38158][metrics] FLIP-484: Add custom metric variables to operators [flink]

Posted by "Izeren (via GitHub)" <gi...@apache.org>.
Izeren commented on code in PR #26849:
URL: https://github.com/apache/flink/pull/26849#discussion_r2245279628


##########
docs/content.zh/docs/ops/metrics.md:
##########
@@ -465,6 +465,38 @@ counter = runtime_context
 {{< /tab >}}
 {{< /tabs >}}
 
+### Additional Variables for operators
+
+You can define custom variables that will be assigned to all metrics reported by a given operator
+using `Transformation.addMetricVariable`. For example:
+
+{{< tabs "32c0ba7f-3acd-831f-4a8b-a2de81b0126e" >}}

Review Comment:
   If I understand correctly, the only requirement for these groupIds to be unique. Given that it is not `customer facing` uid it doesn't have to be "nice", so this is a nit



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org