You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@inlong.apache.org by GitBox <gi...@apache.org> on 2022/09/01 08:44:30 UTC
[GitHub] [inlong] gong opened a new pull request, #5763: [INLONG-5762][Sort] Fix pulsar source metric computing
gong opened a new pull request, #5763:
URL: https://github.com/apache/inlong/pull/5763
### Prepare a Pull Request
- [INLONG-5762][Sort] Fix pulsar source metric computing
- Fixes #5762
### Motivation
Fix pulsar source metric computing
### Modifications
* Add CallbackCollector for computing
* modify metric name
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] healchow commented on a diff in pull request #5763: [INLONG-5762][Sort] Fix pulsar source metric computing
Posted by GitBox <gi...@apache.org>.
healchow commented on code in PR #5763:
URL: https://github.com/apache/inlong/pull/5763#discussion_r960578792
##########
inlong-sort/sort-connectors/pulsar/src/main/java/org/apache/inlong/sort/pulsar/table/DynamicPulsarDeserializationSchema.java:
##########
@@ -132,6 +140,39 @@ public void open(DeserializationSchema.InitializationContext context) throws Exc
}
+ /**
+ * reflect get metricGroup
+ *
+ * @param context
Review Comment:
Suggested adding description for `@param` and `@return`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] gong commented on a diff in pull request #5763: [INLONG-5762][Sort] Fix pulsar source metric computing
Posted by GitBox <gi...@apache.org>.
gong commented on code in PR #5763:
URL: https://github.com/apache/inlong/pull/5763#discussion_r960535561
##########
inlong-sort/sort-connectors/pulsar/src/main/java/org/apache/inlong/sort/pulsar/table/DynamicPulsarDeserializationSchema.java:
##########
@@ -132,6 +140,39 @@ public void open(DeserializationSchema.InitializationContext context) throws Exc
}
+ /**
+ * reflect get metricGroup
+ *
+ * @param context
+ * @return
+ */
+ private MetricGroup getMetricGroup(DeserializationSchema.InitializationContext context)
Review Comment:
@thesumery Because pulsar self metric will be affected if I modify FlinkPulsarSource#open directly. And `org.apache.flink.streaming.connectors.pulsar.FlinkPulsarSource` will need be modified if I use adminUrl pulsar.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] dockerzhang merged pull request #5763: [INLONG-5762][Sort] Fix the computing for the Pulsar source metric
Posted by GitBox <gi...@apache.org>.
dockerzhang merged PR #5763:
URL: https://github.com/apache/inlong/pull/5763
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] EMsnap commented on a diff in pull request #5763: [INLONG-5762][Sort] Fix the computing for the Pulsar source metric
Posted by GitBox <gi...@apache.org>.
EMsnap commented on code in PR #5763:
URL: https://github.com/apache/inlong/pull/5763#discussion_r960718704
##########
inlong-sort/sort-connectors/pulsar/src/main/java/org/apache/inlong/sort/pulsar/table/DynamicPulsarDeserializationSchema.java:
##########
@@ -132,6 +140,40 @@ public void open(DeserializationSchema.InitializationContext context) throws Exc
}
+ /**
+ * reflect get metricGroup
+ *
+ * @param context Contextual information that can be used during initialization.
+ * @return metric group that can be used to register new metrics with Flink and to create a nested hierarchy based
+ * on the group names.
+ */
+ private MetricGroup getMetricGroup(DeserializationSchema.InitializationContext context)
+ throws NoSuchFieldException, IllegalAccessException {
+ MetricGroup metricGroup;
+ String className = "RuntimeContextDeserializationInitializationContextAdapter";
+ String fieldName = "runtimeContext";
+ Class runtimeContextDeserializationInitializationContextAdapter = null;
+ Class[] innerClazz = RuntimeContextInitializationContextAdapters.class.getDeclaredClasses();
+ for (Class clazz : innerClazz) {
+ int mod = clazz.getModifiers();
Review Comment:
mod -> modifier since mod is a term in math
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] EMsnap commented on a diff in pull request #5763: [INLONG-5762][Sort] Fix the computing for the Pulsar source metric
Posted by GitBox <gi...@apache.org>.
EMsnap commented on code in PR #5763:
URL: https://github.com/apache/inlong/pull/5763#discussion_r960718704
##########
inlong-sort/sort-connectors/pulsar/src/main/java/org/apache/inlong/sort/pulsar/table/DynamicPulsarDeserializationSchema.java:
##########
@@ -132,6 +140,40 @@ public void open(DeserializationSchema.InitializationContext context) throws Exc
}
+ /**
+ * reflect get metricGroup
+ *
+ * @param context Contextual information that can be used during initialization.
+ * @return metric group that can be used to register new metrics with Flink and to create a nested hierarchy based
+ * on the group names.
+ */
+ private MetricGroup getMetricGroup(DeserializationSchema.InitializationContext context)
+ throws NoSuchFieldException, IllegalAccessException {
+ MetricGroup metricGroup;
+ String className = "RuntimeContextDeserializationInitializationContextAdapter";
+ String fieldName = "runtimeContext";
+ Class runtimeContextDeserializationInitializationContextAdapter = null;
+ Class[] innerClazz = RuntimeContextInitializationContextAdapters.class.getDeclaredClasses();
+ for (Class clazz : innerClazz) {
+ int mod = clazz.getModifiers();
Review Comment:
mod -> modifier since mod is a term in math
##########
inlong-sort/sort-connectors/pulsar/src/main/java/org/apache/inlong/sort/pulsar/table/DynamicPulsarDeserializationSchema.java:
##########
@@ -132,6 +140,40 @@ public void open(DeserializationSchema.InitializationContext context) throws Exc
}
+ /**
+ * reflect get metricGroup
+ *
+ * @param context Contextual information that can be used during initialization.
+ * @return metric group that can be used to register new metrics with Flink and to create a nested hierarchy based
+ * on the group names.
+ */
+ private MetricGroup getMetricGroup(DeserializationSchema.InitializationContext context)
+ throws NoSuchFieldException, IllegalAccessException {
+ MetricGroup metricGroup;
+ String className = "RuntimeContextDeserializationInitializationContextAdapter";
Review Comment:
pls extract constant for class name
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] gong commented on a diff in pull request #5763: [INLONG-5762][Sort] Fix the computing for the Pulsar source metric
Posted by GitBox <gi...@apache.org>.
gong commented on code in PR #5763:
URL: https://github.com/apache/inlong/pull/5763#discussion_r960810805
##########
inlong-sort/sort-connectors/pulsar/src/main/java/org/apache/inlong/sort/pulsar/table/DynamicPulsarDeserializationSchema.java:
##########
@@ -132,6 +140,40 @@ public void open(DeserializationSchema.InitializationContext context) throws Exc
}
+ /**
+ * reflect get metricGroup
+ *
+ * @param context Contextual information that can be used during initialization.
+ * @return metric group that can be used to register new metrics with Flink and to create a nested hierarchy based
+ * on the group names.
+ */
+ private MetricGroup getMetricGroup(DeserializationSchema.InitializationContext context)
+ throws NoSuchFieldException, IllegalAccessException {
+ MetricGroup metricGroup;
+ String className = "RuntimeContextDeserializationInitializationContextAdapter";
Review Comment:
I don't think that it need constant for class name.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] EMsnap commented on a diff in pull request #5763: [INLONG-5762][Sort] Fix the computing for the Pulsar source metric
Posted by GitBox <gi...@apache.org>.
EMsnap commented on code in PR #5763:
URL: https://github.com/apache/inlong/pull/5763#discussion_r960780949
##########
inlong-sort/sort-connectors/pulsar/src/main/java/org/apache/inlong/sort/pulsar/table/DynamicPulsarDeserializationSchema.java:
##########
@@ -132,6 +140,39 @@ public void open(DeserializationSchema.InitializationContext context) throws Exc
}
+ /**
+ * reflect get metricGroup
+ *
+ * @param context
+ * @return
+ */
+ private MetricGroup getMetricGroup(DeserializationSchema.InitializationContext context)
Review Comment:
> Why not modify FlinkPulsarSource#open directly?
same with me, no need to reflect the metric group. It's ok if we modify the user metrics group since we build the source by ourselves
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] thesumery commented on a diff in pull request #5763: [INLONG-5762][Sort] Fix pulsar source metric computing
Posted by GitBox <gi...@apache.org>.
thesumery commented on code in PR #5763:
URL: https://github.com/apache/inlong/pull/5763#discussion_r960400066
##########
inlong-sort/sort-connectors/pulsar/src/main/java/org/apache/inlong/sort/pulsar/table/DynamicPulsarDeserializationSchema.java:
##########
@@ -132,6 +140,39 @@ public void open(DeserializationSchema.InitializationContext context) throws Exc
}
+ /**
+ * reflect get metricGroup
+ *
+ * @param context
+ * @return
+ */
+ private MetricGroup getMetricGroup(DeserializationSchema.InitializationContext context)
Review Comment:
Why not modify FlinkPulsarSource#open directly?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [inlong] gong commented on a diff in pull request #5763: [INLONG-5762][Sort] Fix pulsar source metric computing
Posted by GitBox <gi...@apache.org>.
gong commented on code in PR #5763:
URL: https://github.com/apache/inlong/pull/5763#discussion_r960615373
##########
inlong-sort/sort-connectors/pulsar/src/main/java/org/apache/inlong/sort/pulsar/table/DynamicPulsarDeserializationSchema.java:
##########
@@ -132,6 +140,39 @@ public void open(DeserializationSchema.InitializationContext context) throws Exc
}
+ /**
+ * reflect get metricGroup
+ *
+ * @param context
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org