You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@uniffle.apache.org by "kuszz (via GitHub)" <gi...@apache.org> on 2024/07/31 07:20:48 UTC
[PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodically report metrics heavy behavior [incubator-uniffle]
kuszz opened a new pull request, #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991
### What changes were proposed in this pull request?
Add another method to add gauge metric, we can use lambda to describe a gauge metric.
### Why are the changes needed?
Fix: #1973
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
UTs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#issuecomment-2285244985
hmmm, it seems all the CI are failing? @zuston would you mind to take a look at the rust CI failure?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "kuszz (via GitHub)" <gi...@apache.org>.
kuszz commented on code in PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#discussion_r1704829185
##########
common/src/main/java/org/apache/uniffle/common/metrics/MetricsManager.java:
##########
@@ -44,6 +51,8 @@ public MetricsManager(CollectorRegistry collectorRegistry, Map<String, String> d
} else {
this.collectorRegistry = collectorRegistry;
}
+ metricRegistry = new MetricRegistry();
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "kuszz (via GitHub)" <gi...@apache.org>.
kuszz commented on code in PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#discussion_r1704829782
##########
common/src/main/java/org/apache/uniffle/common/metrics/MetricsManager.java:
##########
@@ -19,20 +19,27 @@
import java.util.Arrays;
import java.util.Map;
+import java.util.concurrent.TimeUnit;
+import com.codahale.metrics.CachedGauge;
+import com.codahale.metrics.MetricRegistry;
import com.google.common.collect.Maps;
import io.prometheus.client.CollectorRegistry;
import io.prometheus.client.Counter;
import io.prometheus.client.Gauge;
import io.prometheus.client.Histogram;
import io.prometheus.client.Summary;
+import io.prometheus.client.dropwizard.DropwizardExports;
public class MetricsManager {
+ private final MetricRegistry metricRegistry;
private final CollectorRegistry collectorRegistry;
private final String[] defaultLabelNames;
private final String[] defaultLabelValues;
private static final double[] QUANTILES = {0.50, 0.75, 0.90, 0.95, 0.99};
private static final double QUANTILE_ERROR = 0.01;
+ private static final String LABEL_SEPARATOR = ":";
+ private static final String NAME_SEPARATOR = "_";
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "kuszz (via GitHub)" <gi...@apache.org>.
kuszz commented on code in PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#discussion_r1704842179
##########
server/src/main/java/org/apache/uniffle/server/ShuffleServerMetrics.java:
##########
@@ -442,13 +438,7 @@ private static void setUpMetrics(ShuffleServerConf serverConf) {
gaugeInFlushBufferSize = metricsManager.addLabeledGauge(IN_FLUSH_BUFFER_SIZE);
gaugeUsedBufferSize = metricsManager.addLabeledGauge(USED_BUFFER_SIZE);
gaugeReadBufferUsedSize = metricsManager.addLabeledGauge(READ_USED_BUFFER_SIZE);
- gaugeUsedDirectMemorySize = metricsManager.addLabeledGauge(USED_DIRECT_MEMORY_SIZE);
- gaugeUsedDirectMemorySizeByNetty =
- metricsManager.addLabeledGauge(USED_DIRECT_MEMORY_SIZE_BY_NETTY);
- gaugeUsedDirectMemorySizeByGrpcNetty =
- metricsManager.addLabeledGauge(USED_DIRECT_MEMORY_SIZE_BY_GRPC_NETTY);
gaugeWriteHandler = metricsManager.addLabeledGauge(TOTAL_WRITE_HANDLER);
- gaugeEventQueueSize = metricsManager.addLabeledGauge(EVENT_QUEUE_SIZE);
Review Comment:
yes, I haven't found how to add labels through the new method yet, is a label necessary here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#issuecomment-2286305604
I'm fine to ignore that first, cc @zuston
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on code in PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#discussion_r1705189992
##########
server/src/main/java/org/apache/uniffle/server/ShuffleServerMetrics.java:
##########
@@ -442,13 +438,7 @@ private static void setUpMetrics(ShuffleServerConf serverConf) {
gaugeInFlushBufferSize = metricsManager.addLabeledGauge(IN_FLUSH_BUFFER_SIZE);
gaugeUsedBufferSize = metricsManager.addLabeledGauge(USED_BUFFER_SIZE);
gaugeReadBufferUsedSize = metricsManager.addLabeledGauge(READ_USED_BUFFER_SIZE);
- gaugeUsedDirectMemorySize = metricsManager.addLabeledGauge(USED_DIRECT_MEMORY_SIZE);
- gaugeUsedDirectMemorySizeByNetty =
- metricsManager.addLabeledGauge(USED_DIRECT_MEMORY_SIZE_BY_NETTY);
- gaugeUsedDirectMemorySizeByGrpcNetty =
- metricsManager.addLabeledGauge(USED_DIRECT_MEMORY_SIZE_BY_GRPC_NETTY);
gaugeWriteHandler = metricsManager.addLabeledGauge(TOTAL_WRITE_HANDLER);
- gaugeEventQueueSize = metricsManager.addLabeledGauge(EVENT_QUEUE_SIZE);
Review Comment:
I think it's necessary to add label as well.
Did a look at the code, I think you should use `io.prometheus.client.dropwizard.DropwizardExports#DropwizardExports(com.codahale.metrics.MetricRegistry, io.prometheus.client.dropwizard.samplebuilder.SampleBuilder)` this method and passing a default labeled SampleBuilder.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on code in PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#discussion_r1710852896
##########
common/src/main/java/org/apache/uniffle/common/metrics/MetricsManager.java:
##########
@@ -112,4 +130,10 @@ public Summary.Child addLabeledSummary(String name) {
}
return builder.register(collectorRegistry).labels(defaultLabelValues);
}
+
+ public void unRegisterSupplierGauge() {
Review Comment:
hmm, maybe I wasn't clear. I think we should provide a method to unregister a specific gauge with gauge name?
Others, this method should be called as `unRegisterAllSupplierGauges` or `resetSupplierGauges`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "kuszz (via GitHub)" <gi...@apache.org>.
kuszz commented on code in PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#discussion_r1701167515
##########
pom.xml:
##########
@@ -83,7 +83,7 @@
<maven.compiler.source>${java.version}</maven.compiler.source>
<maven.compiler.target>${java.version}</maven.compiler.target>
<spotless-maven-plugin.version>2.30.0</spotless-maven-plugin.version>
- <metrics.version>3.1.0</metrics.version>
+ <metrics.version>3.1.2</metrics.version>
Review Comment:
[WARNING] Rule 0: org.apache.maven.plugins.enforcer.RequireUpperBoundDeps failed with message:
Failed while enforcing RequireUpperBoundDeps. The error(s) are [
Require upper bound dependencies error for io.dropwizard.metrics:metrics-core:3.1.0 paths to dependency are:
+-org.apache.uniffle:coordinator:0.10.0-SNAPSHOT
+-org.apache.uniffle:rss-common:0.10.0-SNAPSHOT
+-io.prometheus:simpleclient_dropwizard:0.8.0
+-io.dropwizard.metrics:metrics-core:3.1.0 (managed) <-- io.dropwizard.metrics:metrics-core:3.1.2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "kuszz (via GitHub)" <gi...@apache.org>.
kuszz commented on code in PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#discussion_r1704830668
##########
common/src/main/java/org/apache/uniffle/common/metrics/MetricsManager.java:
##########
@@ -112,4 +150,8 @@ public Summary.Child addLabeledSummary(String name) {
}
return builder.register(collectorRegistry).labels(defaultLabelValues);
}
+
+ public com.codahale.metrics.Gauge getGauge(String name) {
Review Comment:
yes, this func display how to obtain the value of a gauge registered in new way
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#issuecomment-2268522181
> > 1. how will this new metrics system integrate with existing systems? I think a lot of people are already using prometheus to collect metrics data. It would be great if we can make a drop-in replacement for that. cc @xianjingfeng maybe you have some input about this part
>
> This PR does not affect the original collection logic. https://www.robustperception.io/exposing-dropwizard-metrics-to-prometheus/
This is good to know. But it seems it doesn't support labels? Or it could be supported in another way?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#issuecomment-2283339222
LGTM on my side too. @kuszz would you mind to rebase the latest master and make the CI happy.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "kuszz (via GitHub)" <gi...@apache.org>.
kuszz closed pull request #1991: [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load
URL: https://github.com/apache/incubator-uniffle/pull/1991
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "kuszz (via GitHub)" <gi...@apache.org>.
kuszz commented on code in PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#discussion_r1710845445
##########
common/src/main/java/org/apache/uniffle/common/metrics/MetricsManager.java:
##########
@@ -33,6 +35,7 @@ public class MetricsManager {
private final String[] defaultLabelValues;
private static final double[] QUANTILES = {0.50, 0.75, 0.90, 0.95, 0.99};
private static final double QUANTILE_ERROR = 0.01;
+ private Map<String, SupplierGauge> gaugeMap;
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "kuszz (via GitHub)" <gi...@apache.org>.
kuszz commented on code in PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#discussion_r1711066340
##########
common/src/main/java/org/apache/uniffle/common/metrics/MetricsManager.java:
##########
@@ -112,4 +130,17 @@ public Summary.Child addLabeledSummary(String name) {
}
return builder.register(collectorRegistry).labels(defaultLabelValues);
}
+
+ public void unregisterAllSupplierGauge() {
+ for (SupplierGauge gauge : supplierGaugeMap.values()) {
+ collectorRegistry.unregister(gauge);
+ }
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on code in PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#discussion_r1711019949
##########
common/src/main/java/org/apache/uniffle/common/metrics/MetricsManager.java:
##########
@@ -112,4 +130,17 @@ public Summary.Child addLabeledSummary(String name) {
}
return builder.register(collectorRegistry).labels(defaultLabelValues);
}
+
+ public void unregisterAllSupplierGauge() {
+ for (SupplierGauge gauge : supplierGaugeMap.values()) {
+ collectorRegistry.unregister(gauge);
+ }
Review Comment:
hmm, I think we should clear the supplierGaugeMap as well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "kuszz (via GitHub)" <gi...@apache.org>.
kuszz closed pull request #1991: [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load
URL: https://github.com/apache/incubator-uniffle/pull/1991
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "kuszz (via GitHub)" <gi...@apache.org>.
kuszz closed pull request #1991: [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load
URL: https://github.com/apache/incubator-uniffle/pull/1991
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "kuszz (via GitHub)" <gi...@apache.org>.
kuszz commented on code in PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#discussion_r1704829547
##########
server/src/main/java/org/apache/uniffle/server/ShuffleServerMetrics.java:
##########
@@ -513,4 +503,8 @@ private static void setUpMetrics(ShuffleServerConf serverConf) {
.labelNames("app_id")
.register(metricsManager.getCollectorRegistry());
}
+
+ public static MetricsManager getMetricsManager() {
Review Comment:
thanks, I removed this func
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#issuecomment-2260162601
## Test Results
2 746 files - 1 2 746 suites - 1 5h 39m 34s :stopwatch: - 1m 23s
979 tests ±0 977 :white_check_mark: +1 1 :zzz: ±0 1 :x: +1
12 293 runs - 1 12 277 :white_check_mark: ±0 15 :zzz: ±0 1 :x: +1
For more details on these failures, see [this check](https://github.com/apache/incubator-uniffle/runs/28151723368).
Results for commit b96bcfc6. ± Comparison against base commit be6afb92.
[test-results]:data:application/gzip;base64,H4sIAEAOqmYC/1XMyQ7CIBSF4VdpWLtgpvgyhqEkxA6Gwsr47l6wVVye7978TxTiPO3oOlDF5WVAe4m5374kk+O2VsFMcSC45/qhlT7XbS/ONVI/uscHEPlCMHH+gymlLYFgkFTW2iSUanbMMwrWsh87quKErtp2H3XbssQMA1ktrQtOBhsCJcY7hY0X0jIhsRzHUTFutRUTer0B/peOLRIBAAA=
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "kuszz (via GitHub)" <gi...@apache.org>.
kuszz closed pull request #1991: [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load
URL: https://github.com/apache/incubator-uniffle/pull/1991
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong commented on code in PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#discussion_r1700629203
##########
client-spark/spark2-shaded/pom.xml:
##########
@@ -216,6 +216,10 @@
<pattern>picocli</pattern>
<shadedPattern>${rss.shade.packageName}.picocli</shadedPattern>
</relocation>
+ <relocation>
+ <pattern>com.codahale.metrics</pattern>
+ <shadedPattern>${rss.shade.packageName}.com.codahale.metrics</shadedPattern>
Review Comment:
check the dependencies of this is shaded too
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "kuszz (via GitHub)" <gi...@apache.org>.
kuszz commented on code in PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#discussion_r1701171507
##########
client-spark/spark2-shaded/pom.xml:
##########
@@ -216,6 +216,10 @@
<pattern>picocli</pattern>
<shadedPattern>${rss.shade.packageName}.picocli</shadedPattern>
</relocation>
+ <relocation>
+ <pattern>com.codahale.metrics</pattern>
+ <shadedPattern>${rss.shade.packageName}.com.codahale.metrics</shadedPattern>
Review Comment:
<img width="1462" alt="image" src="https://github.com/user-attachments/assets/dfd6190e-2a1f-4f4c-b81e-57c26df01551">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong commented on PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#issuecomment-2286178455
@advancedxy If it is difficult to fix the rust related CI checks, we can ignore it by this for now？ https://github.com/apache/incubator-uniffle/pull/2041
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "kuszz (via GitHub)" <gi...@apache.org>.
kuszz commented on PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#issuecomment-2288055846
> Thanks to @maobaolong, the rust ci checked is disabled temporary. @kuszz please rebase the master and retry CI, sorry for the inconvenience.
@advancedxy done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "xianjingfeng (via GitHub)" <gi...@apache.org>.
xianjingfeng commented on PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#issuecomment-2268729725
> This is good to know. But it seems it doesn't support labels? Or it could be supported in another way?
It seems that there is no way.
https://groups.google.com/g/prometheus-users/c/6SwcjeAYkiE/m/VvOSd7nSAgAJ
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on code in PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#discussion_r1710614947
##########
common/src/main/java/org/apache/uniffle/common/metrics/MetricsManager.java:
##########
@@ -74,11 +78,37 @@ public Gauge addGauge(String name, String help, String[] labels) {
return Gauge.build().name(name).labelNames(labels).help(help).register(collectorRegistry);
}
+ public SupplierGauge addGauge(
Review Comment:
I think we can remove this method for now, as it's barely used.
##########
common/src/main/java/org/apache/uniffle/common/metrics/MetricsManager.java:
##########
@@ -33,6 +35,7 @@ public class MetricsManager {
private final String[] defaultLabelValues;
private static final double[] QUANTILES = {0.50, 0.75, 0.90, 0.95, 0.99};
private static final double QUANTILE_ERROR = 0.01;
+ private Map<String, SupplierGauge> gaugeMap;
Review Comment:
let's call it `supplierGauggeMap`?
##########
server/src/main/java/org/apache/uniffle/server/DefaultFlushEventHandler.java:
##########
@@ -178,6 +176,8 @@ protected void initFlushEventExecutor() {
hadoopThreadPoolExecutor = createFlushEventExecutor(poolSize, "HadoopFlushEventThreadPool");
}
fallbackThreadPoolExecutor = createFlushEventExecutor(5, "FallBackFlushEventThreadPool");
+ ShuffleServerMetrics.gaugeEventQueueSize =
+ ShuffleServerMetrics.addLabeledGauge(EVENT_QUEUE_SIZE, () -> (double) flushQueue.size());
Review Comment:
It could be:
```
ShuffleServerMetrics.addLabeledGauge(EVENT_QUEUE_SIZE, () -> (double) flushQueue.size());
```
##########
common/src/main/java/org/apache/uniffle/common/metrics/MetricsManager.java:
##########
@@ -47,6 +50,7 @@ public MetricsManager(CollectorRegistry collectorRegistry, Map<String, String> d
this.defaultLabelNames = defaultLabels.keySet().toArray(new String[0]);
this.defaultLabelValues =
Arrays.stream(defaultLabelNames).map(defaultLabels::get).toArray(String[]::new);
+ this.gaugeMap = new ConcurrentHashMap<>();
Review Comment:
Use `this.gaugeMap = JavaUtils.newConcurrentMap();`
This is a performance issue in JDK8 for concurrent hash map, so we have to wrap it with a special one.
##########
common/src/main/java/org/apache/uniffle/common/metrics/MetricsManager.java:
##########
@@ -74,11 +78,37 @@ public Gauge addGauge(String name, String help, String[] labels) {
return Gauge.build().name(name).labelNames(labels).help(help).register(collectorRegistry);
}
+ public SupplierGauge addGauge(
+ String name,
+ String help,
+ Supplier<Double> supplier,
+ String[] labelNames,
+ String[] labelValues) {
+ return gaugeMap.computeIfAbsent(
+ name,
+ metricName ->
+ new SupplierGauge(name, help, supplier, labelNames, labelValues)
+ .register(collectorRegistry));
+ }
+
public Gauge.Child addLabeledGauge(String name) {
Gauge c = addGauge(name, this.defaultLabelNames);
return c.labels(this.defaultLabelValues);
}
+ public SupplierGauge addLabeledGauge(String name, Supplier<Double> supplier) {
Review Comment:
I'm prefer to not exposed SupplierGauge to outside. In other words, this method should return void, and changed to something like:
```java
public void registerLabeledGauge(String name, Supplier<Double> supplier) {...}
// or
public void addLabeledGauge(String name, Supplier<Double> supplier) {...}
```
##########
common/src/main/java/org/apache/uniffle/common/metrics/MetricsManager.java:
##########
@@ -112,4 +142,10 @@ public Summary.Child addLabeledSummary(String name) {
}
return builder.register(collectorRegistry).labels(defaultLabelValues);
}
+
+ public void unRegister() {
Review Comment:
hmm. This signature doesn't seem right.
It should be something like:
```java
public void unregisterSupplierGauge(String name)
```
?
##########
common/src/main/java/org/apache/uniffle/common/metrics/SupplierGauge.java:
##########
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.uniffle.common.metrics;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.function.Supplier;
+
+import io.prometheus.client.Collector;
+import io.prometheus.client.GaugeMetricFamily;
+
+public class SupplierGauge extends Collector implements Collector.Describable {
Review Comment:
Like other comment pointed out, it can be package private, not public.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#issuecomment-2288634059
Merged, thanks @kuszz.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy merged PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "kuszz (via GitHub)" <gi...@apache.org>.
kuszz commented on code in PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#discussion_r1701175318
##########
pom.xml:
##########
@@ -105,6 +105,7 @@
<skipITs>${skipTests}</skipITs>
<skipBuildImage>true</skipBuildImage>
<snakeyaml.version>2.2</snakeyaml.version>
+ <prometheus.version>0.8.0</prometheus.version>
Review Comment:
done
##########
common/src/main/java/org/apache/uniffle/common/metrics/MetricsManager.java:
##########
@@ -66,14 +75,37 @@ public Counter.Child addLabeledCounter(String name) {
return c.labels(this.defaultLabelValues);
}
+ @Deprecated
public Gauge addGauge(String name, String... labels) {
return addGauge(name, "Gauge " + name, labels);
}
+ @Deprecated
public Gauge addGauge(String name, String help, String[] labels) {
return Gauge.build().name(name).labelNames(labels).help(help).register(collectorRegistry);
}
+ public synchronized <T> void addGauge(String name, com.codahale.metrics.Gauge<T> metric) {
Review Comment:
done
##########
common/src/main/java/org/apache/uniffle/common/metrics/MetricsManager.java:
##########
@@ -66,14 +75,37 @@ public Counter.Child addLabeledCounter(String name) {
return c.labels(this.defaultLabelValues);
}
+ @Deprecated
public Gauge addGauge(String name, String... labels) {
return addGauge(name, "Gauge " + name, labels);
}
+ @Deprecated
public Gauge addGauge(String name, String help, String[] labels) {
return Gauge.build().name(name).labelNames(labels).help(help).register(collectorRegistry);
}
+ public synchronized <T> void addGauge(String name, com.codahale.metrics.Gauge<T> metric) {
+ if (!metricRegistry.getMetrics().containsKey(name)) {
+ metricRegistry.register(name, metric);
+ }
+ }
+
+ public synchronized <T> void addGauge(
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong commented on code in PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#discussion_r1701236903
##########
pom.xml:
##########
@@ -83,7 +83,7 @@
<maven.compiler.source>${java.version}</maven.compiler.source>
<maven.compiler.target>${java.version}</maven.compiler.target>
<spotless-maven-plugin.version>2.30.0</spotless-maven-plugin.version>
- <metrics.version>3.1.0</metrics.version>
+ <metrics.version>3.1.2</metrics.version>
Review Comment:
As you change to use `<prometheus.simpleclient.version>0.9.0</prometheus.simpleclient.version>` for the version of `io.prometheus:simpleclient_dropwizard` now, can you try to build again without `metrics.version` change and check whether the enforcing error exists? If not, you can paste the new log part here again.
##########
pom.xml:
##########
@@ -83,7 +83,7 @@
<maven.compiler.source>${java.version}</maven.compiler.source>
<maven.compiler.target>${java.version}</maven.compiler.target>
<spotless-maven-plugin.version>2.30.0</spotless-maven-plugin.version>
- <metrics.version>3.1.0</metrics.version>
+ <metrics.version>3.1.2</metrics.version>
Review Comment:
@kuszz As you change to use `<prometheus.simpleclient.version>0.9.0</prometheus.simpleclient.version>` for the version of `io.prometheus:simpleclient_dropwizard` now, can you try to build again without `metrics.version` change and check whether the enforcing error exists? If not, you can paste the new log part here again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "kuszz (via GitHub)" <gi...@apache.org>.
kuszz commented on code in PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#discussion_r1704831520
##########
common/src/main/java/org/apache/uniffle/common/metrics/MetricsManager.java:
##########
@@ -66,14 +75,43 @@ public Counter.Child addLabeledCounter(String name) {
return c.labels(this.defaultLabelValues);
}
+ @Deprecated
Review Comment:
I removed the annotation, the new way can not add labels, so it cannot completely replace the old method
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "kuszz (via GitHub)" <gi...@apache.org>.
kuszz commented on code in PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#discussion_r1708702349
##########
server/src/main/java/org/apache/uniffle/server/ShuffleServerMetrics.java:
##########
@@ -442,13 +438,7 @@ private static void setUpMetrics(ShuffleServerConf serverConf) {
gaugeInFlushBufferSize = metricsManager.addLabeledGauge(IN_FLUSH_BUFFER_SIZE);
gaugeUsedBufferSize = metricsManager.addLabeledGauge(USED_BUFFER_SIZE);
gaugeReadBufferUsedSize = metricsManager.addLabeledGauge(READ_USED_BUFFER_SIZE);
- gaugeUsedDirectMemorySize = metricsManager.addLabeledGauge(USED_DIRECT_MEMORY_SIZE);
- gaugeUsedDirectMemorySizeByNetty =
- metricsManager.addLabeledGauge(USED_DIRECT_MEMORY_SIZE_BY_NETTY);
- gaugeUsedDirectMemorySizeByGrpcNetty =
- metricsManager.addLabeledGauge(USED_DIRECT_MEMORY_SIZE_BY_GRPC_NETTY);
gaugeWriteHandler = metricsManager.addLabeledGauge(TOTAL_WRITE_HANDLER);
- gaugeEventQueueSize = metricsManager.addLabeledGauge(EVENT_QUEUE_SIZE);
Review Comment:
@advancedxy I have resubmitted the code and now it supports labels
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong commented on code in PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#discussion_r1701494473
##########
pom.xml:
##########
@@ -83,7 +83,7 @@
<maven.compiler.source>${java.version}</maven.compiler.source>
<maven.compiler.target>${java.version}</maven.compiler.target>
<spotless-maven-plugin.version>2.30.0</spotless-maven-plugin.version>
- <metrics.version>3.1.0</metrics.version>
+ <metrics.version>3.1.2</metrics.version>
Review Comment:
Ok, it's clear for now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "kuszz (via GitHub)" <gi...@apache.org>.
kuszz commented on code in PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#discussion_r1701471210
##########
pom.xml:
##########
@@ -83,7 +83,7 @@
<maven.compiler.source>${java.version}</maven.compiler.source>
<maven.compiler.target>${java.version}</maven.compiler.target>
<spotless-maven-plugin.version>2.30.0</spotless-maven-plugin.version>
- <metrics.version>3.1.0</metrics.version>
+ <metrics.version>3.1.2</metrics.version>
Review Comment:
> @kuszz As you change to use `<prometheus.simpleclient.version>0.9.0</prometheus.simpleclient.version>` for the version of `io.prometheus:simpleclient_dropwizard` now, can you try to build again without `metrics.version` change and check whether the enforcing error exists? If not, you can paste the new log part here again.
@maobaolong
[INFO] --- enforcer:1.4.1:enforce (enforce) @ coordinator ---
[WARNING] Rule 0: org.apache.maven.plugins.enforcer.RequireUpperBoundDeps failed with message:
Failed while enforcing RequireUpperBoundDeps. The error(s) are [
Require upper bound dependencies error for io.dropwizard.metrics:metrics-core:3.1.0 paths to dependency are:
+-org.apache.uniffle:coordinator:0.10.0-SNAPSHOT
+-org.apache.uniffle:rss-common:0.10.0-SNAPSHOT
+-io.prometheus:simpleclient_dropwizard:0.9.0
+-io.dropwizard.metrics:metrics-core:3.1.0 (managed) <-- io.dropwizard.metrics:metrics-core:3.1.2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "kuszz (via GitHub)" <gi...@apache.org>.
kuszz commented on code in PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#discussion_r1710845944
##########
common/src/main/java/org/apache/uniffle/common/metrics/MetricsManager.java:
##########
@@ -112,4 +142,10 @@ public Summary.Child addLabeledSummary(String name) {
}
return builder.register(collectorRegistry).labels(defaultLabelValues);
}
+
+ public void unRegister() {
Review Comment:
done
##########
common/src/main/java/org/apache/uniffle/common/metrics/SupplierGauge.java:
##########
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.uniffle.common.metrics;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.function.Supplier;
+
+import io.prometheus.client.Collector;
+import io.prometheus.client.GaugeMetricFamily;
+
+public class SupplierGauge extends Collector implements Collector.Describable {
Review Comment:
done
##########
server/src/main/java/org/apache/uniffle/server/DefaultFlushEventHandler.java:
##########
@@ -178,6 +176,8 @@ protected void initFlushEventExecutor() {
hadoopThreadPoolExecutor = createFlushEventExecutor(poolSize, "HadoopFlushEventThreadPool");
}
fallbackThreadPoolExecutor = createFlushEventExecutor(5, "FallBackFlushEventThreadPool");
+ ShuffleServerMetrics.gaugeEventQueueSize =
+ ShuffleServerMetrics.addLabeledGauge(EVENT_QUEUE_SIZE, () -> (double) flushQueue.size());
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "kuszz (via GitHub)" <gi...@apache.org>.
kuszz closed pull request #1991: [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load
URL: https://github.com/apache/incubator-uniffle/pull/1991
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#issuecomment-2277259218
## [Codecov](https://app.codecov.io/gh/apache/incubator-uniffle/pull/1991?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
Attention: Patch coverage is `38.88889%` with `22 lines` in your changes missing coverage. Please review.
> Project coverage is 52.53%. Comparing base [(`5ddcc28`)](https://app.codecov.io/gh/apache/incubator-uniffle/commit/5ddcc28fa0bc1254dfd5805297ea90670304704e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`e42cd9f`)](https://app.codecov.io/gh/apache/incubator-uniffle/commit/e42cd9fd404b56849c7533c98ecc689c9a1aebb9?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
> Report is 37 commits behind head on master.
| [Files](https://app.codecov.io/gh/apache/incubator-uniffle/pull/1991?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...g/apache/uniffle/common/metrics/SupplierGauge.java](https://app.codecov.io/gh/apache/incubator-uniffle/pull/1991?src=pr&el=tree&filepath=common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Funiffle%2Fcommon%2Fmetrics%2FSupplierGauge.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS91bmlmZmxlL2NvbW1vbi9tZXRyaWNzL1N1cHBsaWVyR2F1Z2UuamF2YQ==) | 0.00% | [15 Missing :warning: ](https://app.codecov.io/gh/apache/incubator-uniffle/pull/1991?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [.../apache/uniffle/common/metrics/MetricsManager.java](https://app.codecov.io/gh/apache/incubator-uniffle/pull/1991?src=pr&el=tree&filepath=common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Funiffle%2Fcommon%2Fmetrics%2FMetricsManager.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS91bmlmZmxlL2NvbW1vbi9tZXRyaWNzL01ldHJpY3NNYW5hZ2VyLmphdmE=) | 12.50% | [7 Missing :warning: ](https://app.codecov.io/gh/apache/incubator-uniffle/pull/1991?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #1991 +/- ##
============================================
- Coverage 52.77% 52.53% -0.24%
- Complexity 2498 2925 +427
============================================
Files 398 447 +49
Lines 18135 23546 +5411
Branches 1660 2196 +536
============================================
+ Hits 9570 12370 +2800
- Misses 7981 10383 +2402
- Partials 584 793 +209
```
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/incubator-uniffle/pull/1991?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong commented on code in PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#discussion_r1701050744
##########
common/src/main/java/org/apache/uniffle/common/metrics/MetricsManager.java:
##########
@@ -66,14 +75,37 @@ public Counter.Child addLabeledCounter(String name) {
return c.labels(this.defaultLabelValues);
}
+ @Deprecated
public Gauge addGauge(String name, String... labels) {
return addGauge(name, "Gauge " + name, labels);
}
+ @Deprecated
public Gauge addGauge(String name, String help, String[] labels) {
return Gauge.build().name(name).labelNames(labels).help(help).register(collectorRegistry);
}
+ public synchronized <T> void addGauge(String name, com.codahale.metrics.Gauge<T> metric) {
Review Comment:
registerGaugeIfAbsent
##########
common/src/main/java/org/apache/uniffle/common/metrics/MetricsManager.java:
##########
@@ -66,14 +75,37 @@ public Counter.Child addLabeledCounter(String name) {
return c.labels(this.defaultLabelValues);
}
+ @Deprecated
public Gauge addGauge(String name, String... labels) {
return addGauge(name, "Gauge " + name, labels);
}
+ @Deprecated
public Gauge addGauge(String name, String help, String[] labels) {
return Gauge.build().name(name).labelNames(labels).help(help).register(collectorRegistry);
}
+ public synchronized <T> void addGauge(String name, com.codahale.metrics.Gauge<T> metric) {
+ if (!metricRegistry.getMetrics().containsKey(name)) {
+ metricRegistry.register(name, metric);
+ }
+ }
+
+ public synchronized <T> void addGauge(
Review Comment:
registerCachedGaugeIfAbsent
##########
pom.xml:
##########
@@ -83,7 +83,7 @@
<maven.compiler.source>${java.version}</maven.compiler.source>
<maven.compiler.target>${java.version}</maven.compiler.target>
<spotless-maven-plugin.version>2.30.0</spotless-maven-plugin.version>
- <metrics.version>3.1.0</metrics.version>
+ <metrics.version>3.1.2</metrics.version>
Review Comment:
Can you explain why upgrade metrics.version, is this necessary?
##########
pom.xml:
##########
@@ -105,6 +105,7 @@
<skipITs>${skipTests}</skipITs>
<skipBuildImage>true</skipBuildImage>
<snakeyaml.version>2.2</snakeyaml.version>
+ <prometheus.version>0.8.0</prometheus.version>
Review Comment:
Why not use ${prometheus.simpleclient.version}?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "kuszz (via GitHub)" <gi...@apache.org>.
kuszz closed pull request #1991: [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load
URL: https://github.com/apache/incubator-uniffle/pull/1991
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on code in PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#discussion_r1701809847
##########
common/src/main/java/org/apache/uniffle/common/metrics/MetricsManager.java:
##########
@@ -112,4 +150,8 @@ public Summary.Child addLabeledSummary(String name) {
}
return builder.register(collectorRegistry).labels(defaultLabelValues);
}
+
+ public com.codahale.metrics.Gauge getGauge(String name) {
Review Comment:
Never used?
##########
common/src/main/java/org/apache/uniffle/common/metrics/MetricsManager.java:
##########
@@ -44,6 +51,8 @@ public MetricsManager(CollectorRegistry collectorRegistry, Map<String, String> d
} else {
this.collectorRegistry = collectorRegistry;
}
+ metricRegistry = new MetricRegistry();
Review Comment:
Nit: -> `this.metricRegistry = xxx`.
It's clear to explicitly use `this.xx` when updating the instance field.
##########
common/src/main/java/org/apache/uniffle/common/metrics/MetricsManager.java:
##########
@@ -66,14 +75,43 @@ public Counter.Child addLabeledCounter(String name) {
return c.labels(this.defaultLabelValues);
}
+ @Deprecated
Review Comment:
I would like to add a javadoc to indicate which version this method will be removed and what alternative should be used.
##########
server/src/main/java/org/apache/uniffle/server/ShuffleServerMetrics.java:
##########
@@ -442,13 +438,7 @@ private static void setUpMetrics(ShuffleServerConf serverConf) {
gaugeInFlushBufferSize = metricsManager.addLabeledGauge(IN_FLUSH_BUFFER_SIZE);
gaugeUsedBufferSize = metricsManager.addLabeledGauge(USED_BUFFER_SIZE);
gaugeReadBufferUsedSize = metricsManager.addLabeledGauge(READ_USED_BUFFER_SIZE);
- gaugeUsedDirectMemorySize = metricsManager.addLabeledGauge(USED_DIRECT_MEMORY_SIZE);
- gaugeUsedDirectMemorySizeByNetty =
- metricsManager.addLabeledGauge(USED_DIRECT_MEMORY_SIZE_BY_NETTY);
- gaugeUsedDirectMemorySizeByGrpcNetty =
- metricsManager.addLabeledGauge(USED_DIRECT_MEMORY_SIZE_BY_GRPC_NETTY);
gaugeWriteHandler = metricsManager.addLabeledGauge(TOTAL_WRITE_HANDLER);
- gaugeEventQueueSize = metricsManager.addLabeledGauge(EVENT_QUEUE_SIZE);
Review Comment:
hmm. I think the default lables are lost by replacing this with codahale gauge.
addLabeledGauge will add the default label and value to a gauge.
##########
common/src/main/java/org/apache/uniffle/common/metrics/MetricsManager.java:
##########
@@ -19,20 +19,27 @@
import java.util.Arrays;
import java.util.Map;
+import java.util.concurrent.TimeUnit;
+import com.codahale.metrics.CachedGauge;
+import com.codahale.metrics.MetricRegistry;
import com.google.common.collect.Maps;
import io.prometheus.client.CollectorRegistry;
import io.prometheus.client.Counter;
import io.prometheus.client.Gauge;
import io.prometheus.client.Histogram;
import io.prometheus.client.Summary;
+import io.prometheus.client.dropwizard.DropwizardExports;
public class MetricsManager {
+ private final MetricRegistry metricRegistry;
private final CollectorRegistry collectorRegistry;
private final String[] defaultLabelNames;
private final String[] defaultLabelValues;
private static final double[] QUANTILES = {0.50, 0.75, 0.90, 0.95, 0.99};
private static final double QUANTILE_ERROR = 0.01;
+ private static final String LABEL_SEPARATOR = ":";
+ private static final String NAME_SEPARATOR = "_";
Review Comment:
Seems these two are never used?
##########
server/src/main/java/org/apache/uniffle/server/ShuffleServerMetrics.java:
##########
@@ -513,4 +503,8 @@ private static void setUpMetrics(ShuffleServerConf serverConf) {
.labelNames("app_id")
.register(metricsManager.getCollectorRegistry());
}
+
+ public static MetricsManager getMetricsManager() {
Review Comment:
I don't think we should expose metrics manager directly.
Instead, you could add methods in ShuffleServerMetrics to register new metrics, such as:
```java
public static void registerEventQueueSize(com.codahale.metrics.Gauge<Integer> eventQueueSize) {
// do all kinds of validation if needed.
metricsManager.registerGaugeIfAbsent(EVENT_QUEUE_SIZE, eventQueueSize);
}
```
Or we can expose the metrics names and the `registerMetrics` method combined.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "xianjingfeng (via GitHub)" <gi...@apache.org>.
xianjingfeng commented on PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#issuecomment-2268256682
> 1. how will this new metrics system integrate with existing systems? I think a lot of people are already using prometheus to collect metrics data. It would be great if we can make a drop-in replacement for that. cc @xianjingfeng maybe you have some input about this part
This PR does not affect the original collection logic. https://www.robustperception.io/exposing-dropwizard-metrics-to-prometheus/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "kuszz (via GitHub)" <gi...@apache.org>.
kuszz commented on code in PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#discussion_r1710845628
##########
common/src/main/java/org/apache/uniffle/common/metrics/MetricsManager.java:
##########
@@ -47,6 +50,7 @@ public MetricsManager(CollectorRegistry collectorRegistry, Map<String, String> d
this.defaultLabelNames = defaultLabels.keySet().toArray(new String[0]);
this.defaultLabelValues =
Arrays.stream(defaultLabelNames).map(defaultLabels::get).toArray(String[]::new);
+ this.gaugeMap = new ConcurrentHashMap<>();
Review Comment:
thanks, I have modified it
##########
common/src/main/java/org/apache/uniffle/common/metrics/MetricsManager.java:
##########
@@ -74,11 +78,37 @@ public Gauge addGauge(String name, String help, String[] labels) {
return Gauge.build().name(name).labelNames(labels).help(help).register(collectorRegistry);
}
+ public SupplierGauge addGauge(
Review Comment:
done
##########
common/src/main/java/org/apache/uniffle/common/metrics/MetricsManager.java:
##########
@@ -74,11 +78,37 @@ public Gauge addGauge(String name, String help, String[] labels) {
return Gauge.build().name(name).labelNames(labels).help(help).register(collectorRegistry);
}
+ public SupplierGauge addGauge(
+ String name,
+ String help,
+ Supplier<Double> supplier,
+ String[] labelNames,
+ String[] labelValues) {
+ return gaugeMap.computeIfAbsent(
+ name,
+ metricName ->
+ new SupplierGauge(name, help, supplier, labelNames, labelValues)
+ .register(collectorRegistry));
+ }
+
public Gauge.Child addLabeledGauge(String name) {
Gauge c = addGauge(name, this.defaultLabelNames);
return c.labels(this.defaultLabelValues);
}
+ public SupplierGauge addLabeledGauge(String name, Supplier<Double> supplier) {
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "kuszz (via GitHub)" <gi...@apache.org>.
kuszz commented on code in PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#discussion_r1711019543
##########
common/src/main/java/org/apache/uniffle/common/metrics/MetricsManager.java:
##########
@@ -112,4 +130,10 @@ public Summary.Child addLabeledSummary(String name) {
}
return builder.register(collectorRegistry).labels(defaultLabelValues);
}
+
+ public void unRegisterSupplierGauge() {
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "kuszz (via GitHub)" <gi...@apache.org>.
kuszz closed pull request #1991: [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load
URL: https://github.com/apache/incubator-uniffle/pull/1991
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "kuszz (via GitHub)" <gi...@apache.org>.
kuszz commented on PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#issuecomment-2285223467
> LGTM on my side too. @kuszz would you mind to rebase the latest master and make the CI happy.
@advancedxy rust ci reports 404 error
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1973] feat(server,coordinator): Refactor metrics system to reduce periodic reporting load [incubator-uniffle]
Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on PR #1991:
URL: https://github.com/apache/incubator-uniffle/pull/1991#issuecomment-2287768721
Thanks to @maobaolong, the rust ci checked is disabled temporary. @kuszz please rebase the master and retry CI, sorry for the inconvenience.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org