You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/06/23 14:32:01 UTC
[GitHub] [pulsar] tjiuming opened a new pull request, #16197: [broker][monitoring] Support gzip compression when expose metrics data
tjiuming opened a new pull request, #16197:
URL: https://github.com/apache/pulsar/pull/16197
### Motivation
support gzip compression when expose metrics data
### Modifications
1. add a configuration: enableMetricsDataCompression
### Documentation
Check the box below or label this PR directly.
Need to update docs?
- [x] `doc-required`
(Your PR needs to update docs and you will update later)
- [ ] `doc-not-needed`
(Please explain why)
- [ ] `doc`
(Your PR contains doc changes)
- [ ] `doc-complete`
(Docs have been already added)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat commented on a diff in pull request #16197: [broker][monitoring] Support gzip compression when expose metrics data
Posted by GitBox <gi...@apache.org>.
merlimat commented on code in PR #16197:
URL: https://github.com/apache/pulsar/pull/16197#discussion_r905256960
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/stats/prometheus/PrometheusMetricsGenerator.java:
##########
@@ -318,4 +298,40 @@ private static void generateManagedLedgerBookieClientMetrics(PulsarService pulsa
// nop
}
}
+
+
+ private static void writeOutMetricData(OutputStream out, ByteBuf buffer, boolean exposeBufferMetrics,
+ boolean enableCompress) throws IOException {
+ try {
+ if (enableCompress) {
+ out = new GZIPOutputStream(out);
Review Comment:
I think `GZIPOutputStream` will create a lot of garbage on the JVM heap :|
Could we just use `org.apache.pulsar.common.compression.CompressionCodecZLib` on the buffer?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] tjiuming commented on a diff in pull request #16197: [broker][monitoring] Support gzip compression when expose metrics data
Posted by GitBox <gi...@apache.org>.
tjiuming commented on code in PR #16197:
URL: https://github.com/apache/pulsar/pull/16197#discussion_r912342379
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/stats/prometheus/PrometheusMetricsGenerator.java:
##########
@@ -318,4 +298,40 @@ private static void generateManagedLedgerBookieClientMetrics(PulsarService pulsa
// nop
}
}
+
+
+ private static void writeOutMetricData(OutputStream out, ByteBuf buffer, boolean exposeBufferMetrics,
+ boolean enableCompress) throws IOException {
+ try {
+ if (enableCompress) {
+ out = new GZIPOutputStream(out);
Review Comment:
@merlimat But `CompressionCodecZLib` not in GZIP format, and Prometheus read metrics data in GZIP format:
```java
if s.gzipr == nil {
s.buf = bufio.NewReader(resp.Body)
s.gzipr, err = gzip.NewReader(s.buf)
if err != nil {
return "", err
}
} else {
s.buf.Reset(resp.Body)
if err = s.gzipr.Reset(s.buf); err != nil {
return "", err
}
}
n, err := io.Copy(w, io.LimitReader(s.gzipr, s.bodySizeLimit))
s.gzipr.Close()
if err != nil {
return "", err
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] tjiuming closed pull request #16197: [broker][monitoring] Support gzip compression when expose metrics data
Posted by GitBox <gi...@apache.org>.
tjiuming closed pull request #16197: [broker][monitoring] Support gzip compression when expose metrics data
URL: https://github.com/apache/pulsar/pull/16197
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org