You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2022/09/16 18:30:06 UTC

[GitHub] [pinot] gortiz opened a new pull request, #9416: Compatible dropwizard metrics

gortiz opened a new pull request, #9416:
URL: https://github.com/apache/pinot/pull/9416

   We currently use Yammer as the metric library, but Yammer is not maintained and it doesn’t have new features other more modern metric libraries (like Drowizard or Micrometer) have. For example, Yammer histograms are misleading in some situations (I strongly recommend to read [this article](https://medium.com/expedia-group-tech/your-latency-metrics-could-be-misleading-you-how-hdrhistogram-can-help-9d545b598374) to learn more about the problem). This issue also affects Dropwizard (in fact the referenced article is focused on Dropwizard) but Dropwizard has evolved in a way that the problem can be fixed by changing the reservoir.
   
   We already have a Dropwizard implementation, but we cannot currently use it because it produces metrics with other JMX names, which could break alerts and dashboards our users have right now.
   
   At this moment this PR is a draft on which we can discuss, not a ready to merge PR. This PR changes some default values and therefore we would need to test it properly in environments we can break.
   
   This PR creates a new metric plugin called _compound_. This metric plugin contains a list of other metric plugins. Each time a metric is registered or unregistered in the _compound_ registry, it is registered or unregistered in all other metric plugins. The metric plugins that are notified by the _compound_ registry can be configured, but by default it notifies all other metric plugins in the classpath.
   
   By using this _compound_ metric plugin and including both Dropwizard and Yammer plugins in the classpath, Pinot will register each metric twice: One in Yammer and one in Dropwizard. As said above, each registry produces its own JMX names, so alerts and dashboards created based on the Yammer metrics will continue to work as expected, but given that Dropwizard JMX names will also be published, dashboard and alerts can be migrated to use the new ones and therefore can use the new features.
   
   This PR also adds the ability to change the _domain_ on which Dropwizard metrics are published by changing the property `pinot.metrics.dropwizard.domain`, whose default value is `metrics`, following the Dropwizard default domain. This domain is the prefix used by Dropwizard to create its MBean names, which are something like `"<domain>";type="<type>";name="<metric_name>"`.
   
   By default a Pinot metric called `myTimer` of type `timer` in the Pinot server is exposed:
   - by Yammer as `"org.apache.pinot.common.metrics";type="ServerMetrics";name="myTymer"`
   - by Dropwizard as `"metrics";type="timers";name="myTymer"`
   
   When the new `pinot.metrics.dropwizard.domain` property is changed to `org.apache.pinot.common.metrics`, Dropwizard will export the metric as `"org.apache.pinot.common.metrics";type="timer";name="myTymer"`. As far as I know it is not possible to tell Dropwizard to use other value as type. It always use the type of the metric (gauge, timer, histogram, counter, etc).
   
   Therefore, with this property we are able to create closer names than the one created with Yammer, but not exactly the same. To palliate that, this PR relax the default prometheus JMX mappers, removing the `BrokerMetrics`, `ControllerMetrics`, `ServerMetrics`, `MinionMetrics` and `ValidationMetrics` type literals and changing them for any word.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Compatible dropwizard metrics [pinot]

Posted by "gortiz (via GitHub)" <gi...@apache.org>.
gortiz merged PR #9416:
URL: https://github.com/apache/pinot/pull/9416


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] gortiz commented on pull request #9416: Compatible dropwizard metrics

Posted by "gortiz (via GitHub)" <gi...@apache.org>.
gortiz commented on PR #9416:
URL: https://github.com/apache/pinot/pull/9416#issuecomment-1573590631

   Update on the issue:
   
   Metrics created using Dropwizard and Yammer are very close, but not exactly the same. By running using the new CompoundPinotMetricsFactory as factory, we can produce both metrics.
   After that, I used [jxmterm](https://cwiki.apache.org/confluence/display/KAFKA/jmxterm+quickstart) to read the JMX metrics exported by Yammer and Dropwizard.
   
   The first difference is the id of the beans created. JMX metrics have several coordinates. Some of them are standard (like domain, name) and some other are not. Specifically, our usage of Yammer and Dropwizard uses these three coordinates:
   - domain:
     - Yammer: we set the value to `"org.apache.pinot.common.metrics"` (note> we include the double quotes).
     - Dropwizard: the domain can be changed with `pinot.metrics.dropwizard.domain`. By default it is `metrics`. There is a Javadoc saying that we use that because it was the default in Pinot < 11. I've changed the default to `org.apache.pinot.common.metrics` (without double quotes) to make it closer to Yammer and more difficult to collide with other metrics running in the same system.
   - name:
     - Yammer: we wrap the names between double quotes. For example, metric `pinot.server.meetupRsvp_REALTIME.numSegmentsPrunedInvalid` is reported as `"pinot.server.meetupRsvp_REALTIME.numSegmentsPrunedInvalid"` by Yammer
     - Dropwizard: we reported the names without double quotes. For example, metric `pinot.server.meetupRsvp_REALTIME.numSegmentsPrunedInvalid` is reported as `pinot.server.meetupRsvp_REALTIME.numSegmentsPrunedInvalid` by Dropwizard.
   - type:
      - Yammer: We set as type either `ControllerMetrics`, `BrokerMetrics`, `MinionMetrics`, `ServerMetrics` or `ValidationMetrics`.
      - Dropwizard: The type is set by Dropwizard, using values like `gauge`, `meters`. It may be possible to change that, but doesn't seem trivial.
   
   As the last part, we configure the prometheus exporter to map JMX metrics to prometheus. In files like `docker/images/pinot/etc/jmx_prometheus_javaagent/configs/broker.yml` (and server.yml, controller.yml, etc) we configure rules that do that mapping. For example:
   
   ```yaml
   - pattern: "\"org.apache.pinot.common.metrics\"<type=\"BrokerMetrics\", name=\"pinot.broker.([^\\.]*?)\\.totalServerResponseSize\"><>(\\w+)"
     name: "pinot_broker_totalServerResponseSize_$2"
     cache: true
     labels:
       table: "$1"
   ```
   
   which follows the path: `${domain}<type=${type}, name=${name}`
   
   In order to be able to read Prometheus metrics, we should change the pattern to something like:
   ```yaml
   - pattern: "\"?org.apache.pinot.common.metrics\"?<type=\\w, name=\"pinot.broker.([^\\.]*?)\\.totalServerResponseSize\"><>(\\w+)"
   ```
   
   Probably we could even skip the type and generate something like:
   ```yaml
   - pattern: "\"?org.apache.pinot.common.metrics\"?<name=\"pinot.broker.([^\\.]*?)\\.totalServerResponseSize\"><>(\\w+)"
   ```
   
   This changes should not be done when using `CompoundPinotMetricsFactory`, given that it would catch metrics twice (once the one generated by Yammer and once the one generated by Dropwizard)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] npawar commented on pull request #9416: Compatible dropwizard metrics

Posted by "npawar (via GitHub)" <gi...@apache.org>.
npawar commented on PR #9416:
URL: https://github.com/apache/pinot/pull/9416#issuecomment-1577498589

   If we use Compound metrics registry, would the prometheus storage double?
   And in the end where you give example of what we can set in pattern with Dropwizard, what should we set to catch separately, if we are using Compound?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] jackjlli commented on a diff in pull request #9416: Compatible dropwizard metrics

Posted by "jackjlli (via GitHub)" <gi...@apache.org>.
jackjlli commented on code in PR #9416:
URL: https://github.com/apache/pinot/pull/9416#discussion_r1220152805


##########
pinot-plugins/pinot-metrics/pinot-compound-metrics/src/main/java/org/apache/pinot/plugin/metrics/compound/CompoundPinotMetricsFactory.java:
##########
@@ -0,0 +1,170 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.plugin.metrics.compound;
+
+import com.google.common.base.Preconditions;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Locale;
+import java.util.ServiceLoader;
+import java.util.Set;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+import org.apache.pinot.spi.annotations.metrics.MetricsFactory;
+import org.apache.pinot.spi.annotations.metrics.PinotMetricsFactory;
+import org.apache.pinot.spi.env.PinotConfiguration;
+import org.apache.pinot.spi.metrics.PinotGauge;
+import org.apache.pinot.spi.metrics.PinotJmxReporter;
+import org.apache.pinot.spi.metrics.PinotMetricName;
+import org.apache.pinot.spi.metrics.PinotMetricUtils;
+import org.apache.pinot.spi.metrics.PinotMetricsRegistry;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+@MetricsFactory
+public class CompoundPinotMetricsFactory implements PinotMetricsFactory {

Review Comment:
   Could you add some javadoc (e.g. how to use it and what to note) for this class in case ppl would like to try it out?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Compatible dropwizard metrics [pinot]

Posted by "gortiz (via GitHub)" <gi...@apache.org>.
gortiz commented on PR #9416:
URL: https://github.com/apache/pinot/pull/9416#issuecomment-2293569057

   Resolved.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] gortiz commented on a diff in pull request #9416: Compatible dropwizard metrics

Posted by "gortiz (via GitHub)" <gi...@apache.org>.
gortiz commented on code in PR #9416:
URL: https://github.com/apache/pinot/pull/9416#discussion_r1221862604


##########
pinot-plugins/pinot-metrics/pinot-compound-metrics/src/main/java/org/apache/pinot/plugin/metrics/compound/CompoundPinotMetricsFactory.java:
##########
@@ -0,0 +1,170 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.plugin.metrics.compound;
+
+import com.google.common.base.Preconditions;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Locale;
+import java.util.ServiceLoader;
+import java.util.Set;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+import org.apache.pinot.spi.annotations.metrics.MetricsFactory;
+import org.apache.pinot.spi.annotations.metrics.PinotMetricsFactory;
+import org.apache.pinot.spi.env.PinotConfiguration;
+import org.apache.pinot.spi.metrics.PinotGauge;
+import org.apache.pinot.spi.metrics.PinotJmxReporter;
+import org.apache.pinot.spi.metrics.PinotMetricName;
+import org.apache.pinot.spi.metrics.PinotMetricUtils;
+import org.apache.pinot.spi.metrics.PinotMetricsRegistry;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+@MetricsFactory
+public class CompoundPinotMetricsFactory implements PinotMetricsFactory {

Review Comment:
   I've added some lines. Is it clearer now?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] gortiz commented on pull request #9416: Compatible dropwizard metrics

Posted by "gortiz (via GitHub)" <gi...@apache.org>.
gortiz commented on PR #9416:
URL: https://github.com/apache/pinot/pull/9416#issuecomment-1577689769

   > If we use Compound metrics registry, would the prometheus storage double?
   
   That depends on our Prometheus config. We could configure Prometheus to store one set of the metrics or both. In the latter case the storage would be doubled.
   
   > And in the end where you give example of what we can set in pattern with Dropwizard, what should we set to catch separately, if we are using Compound?
   
   I think we should not use Compound in actual deployments. It may be useful to use that in a integration test in order to verify that both metric registries are actually returning compatible information and that we can create the same Prometheus metrics from one or the other.
   
   In case we want to actually use Compound and we want to register both metrics in Prometheus, we would need to have two rules per metric. One can be the same we have right now (which is going to read from Yammer) and then we can add another like:
   ```
   - pattern: "org.apache.pinot.common.metrics<name=\"pinot.broker.([^\\.]*?)\\.totalServerResponseSize\"><>(\\w+)"
   ```
   
   The latter will only match with Dropwizard because the domain does not start with `\"`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Compatible dropwizard metrics [pinot]

Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on code in PR #9416:
URL: https://github.com/apache/pinot/pull/9416#discussion_r1720687117


##########
pom.xml:
##########
@@ -413,6 +413,16 @@
         <artifactId>pinot-minion</artifactId>
         <version>${project.version}</version>
       </dependency>
+      <dependency>
+        <groupId>org.apache.pinot</groupId>
+        <artifactId>pinot-dropwizard</artifactId>

Review Comment:
   This is already defined.



##########
pinot-tools/pom.xml:
##########
@@ -125,6 +125,19 @@
       <artifactId>pinot-yammer</artifactId>
       <scope>runtime</scope>
     </dependency>
+    <dependency>
+      <groupId>org.apache.pinot</groupId>
+      <artifactId>pinot-yammer</artifactId>

Review Comment:
   This is defined twice



##########
pom.xml:
##########
@@ -413,6 +413,16 @@
         <artifactId>pinot-minion</artifactId>
         <version>${project.version}</version>
       </dependency>
+      <dependency>
+        <groupId>org.apache.pinot</groupId>
+        <artifactId>pinot-dropwizard</artifactId>
+        <version>${project.version}</version>
+      </dependency>
+      <dependency>
+        <groupId>org.apache.pinot</groupId>
+        <artifactId>pinot-compound-metrics</artifactId>

Review Comment:
   Can you move this after the `pinot-dropwizard` which is already defined in this file?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Compatible dropwizard metrics [pinot]

Posted by "real-mj-song (via GitHub)" <gi...@apache.org>.
real-mj-song commented on PR #9416:
URL: https://github.com/apache/pinot/pull/9416#issuecomment-2253177639

   @gortiz Is this PR still being worked on/active? The compound metrics plugin concept will also help for future open telemetry plugin (when/if it's added).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] gortiz commented on a diff in pull request #9416: Compatible dropwizard metrics

Posted by "gortiz (via GitHub)" <gi...@apache.org>.
gortiz commented on code in PR #9416:
URL: https://github.com/apache/pinot/pull/9416#discussion_r1221862604


##########
pinot-plugins/pinot-metrics/pinot-compound-metrics/src/main/java/org/apache/pinot/plugin/metrics/compound/CompoundPinotMetricsFactory.java:
##########
@@ -0,0 +1,170 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.plugin.metrics.compound;
+
+import com.google.common.base.Preconditions;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Locale;
+import java.util.ServiceLoader;
+import java.util.Set;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+import org.apache.pinot.spi.annotations.metrics.MetricsFactory;
+import org.apache.pinot.spi.annotations.metrics.PinotMetricsFactory;
+import org.apache.pinot.spi.env.PinotConfiguration;
+import org.apache.pinot.spi.metrics.PinotGauge;
+import org.apache.pinot.spi.metrics.PinotJmxReporter;
+import org.apache.pinot.spi.metrics.PinotMetricName;
+import org.apache.pinot.spi.metrics.PinotMetricUtils;
+import org.apache.pinot.spi.metrics.PinotMetricsRegistry;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+@MetricsFactory
+public class CompoundPinotMetricsFactory implements PinotMetricsFactory {

Review Comment:
   I've added some lines. Is easier to read now?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on pull request #9416: Compatible dropwizard metrics

Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on PR #9416:
URL: https://github.com/apache/pinot/pull/9416#issuecomment-1577305180

   Very promising work! @jackjlli Can you help review this?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Compatible dropwizard metrics [pinot]

Posted by "gortiz (via GitHub)" <gi...@apache.org>.
gortiz commented on PR #9416:
URL: https://github.com/apache/pinot/pull/9416#issuecomment-2255524941

   > Is this PR still being worked on/active? The compound metrics plugin concept will also help for future open telemetry plugin (when/if it's added).
   
   No, it is not. But if it is useful for someone we can add it. Anyway, it would need a +1 from some committer.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter commented on pull request #9416: Compatible dropwizard metrics

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #9416:
URL: https://github.com/apache/pinot/pull/9416#issuecomment-1573652183

   ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/9416?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   > Merging [#9416](https://app.codecov.io/gh/apache/pinot/pull/9416?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (be36697) into [master](https://app.codecov.io/gh/apache/pinot/commit/714ea4ed645407d0e3b9509bc702fa5967b0046a?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (714ea4e) will **decrease** coverage by `56.59%`.
   > The diff coverage is `0.00%`.
   
   ```diff
   @@              Coverage Diff              @@
   ##             master    #9416       +/-   ##
   =============================================
   - Coverage     70.18%   13.60%   -56.59%     
   + Complexity     6574      455     -6119     
   =============================================
     Files          2170     2139       -31     
     Lines        116632   114485     -2147     
     Branches      17648    17371      -277     
   =============================================
   - Hits          81863    15570    -66293     
   - Misses        29053    97638    +68585     
   + Partials       5716     1277     -4439     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration1 | `?` | |
   | integration2 | `?` | |
   | unittests1 | `?` | |
   | unittests2 | `13.60% <0.00%> (-0.02%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://app.codecov.io/gh/apache/pinot/pull/9416?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../metrics/compound/AbstractCompoundPinotMetric.java](https://app.codecov.io/gh/apache/pinot/pull/9416?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtcGx1Z2lucy9waW5vdC1tZXRyaWNzL3Bpbm90LWNvbXBvdW5kLW1ldHJpY3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9tZXRyaWNzL2NvbXBvdW5kL0Fic3RyYWN0Q29tcG91bmRQaW5vdE1ldHJpYy5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [.../plugin/metrics/compound/CompoundPinotCounter.java](https://app.codecov.io/gh/apache/pinot/pull/9416?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtcGx1Z2lucy9waW5vdC1tZXRyaWNzL3Bpbm90LWNvbXBvdW5kLW1ldHJpY3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9tZXRyaWNzL2NvbXBvdW5kL0NvbXBvdW5kUGlub3RDb3VudGVyLmphdmE=) | `0.00% <0.00%> (ø)` | |
   | [...ot/plugin/metrics/compound/CompoundPinotGauge.java](https://app.codecov.io/gh/apache/pinot/pull/9416?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtcGx1Z2lucy9waW5vdC1tZXRyaWNzL3Bpbm90LWNvbXBvdW5kLW1ldHJpY3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9tZXRyaWNzL2NvbXBvdW5kL0NvbXBvdW5kUGlub3RHYXVnZS5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [...lugin/metrics/compound/CompoundPinotHistogram.java](https://app.codecov.io/gh/apache/pinot/pull/9416?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtcGx1Z2lucy9waW5vdC1tZXRyaWNzL3Bpbm90LWNvbXBvdW5kLW1ldHJpY3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9tZXRyaWNzL2NvbXBvdW5kL0NvbXBvdW5kUGlub3RIaXN0b2dyYW0uamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...gin/metrics/compound/CompoundPinotJmxReporter.java](https://app.codecov.io/gh/apache/pinot/pull/9416?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtcGx1Z2lucy9waW5vdC1tZXRyaWNzL3Bpbm90LWNvbXBvdW5kLW1ldHJpY3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9tZXRyaWNzL2NvbXBvdW5kL0NvbXBvdW5kUGlub3RKbXhSZXBvcnRlci5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [...ot/plugin/metrics/compound/CompoundPinotMeter.java](https://app.codecov.io/gh/apache/pinot/pull/9416?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtcGx1Z2lucy9waW5vdC1tZXRyaWNzL3Bpbm90LWNvbXBvdW5kLW1ldHJpY3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9tZXRyaWNzL2NvbXBvdW5kL0NvbXBvdW5kUGlub3RNZXRlci5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [...ugin/metrics/compound/CompoundPinotMetricName.java](https://app.codecov.io/gh/apache/pinot/pull/9416?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtcGx1Z2lucy9waW5vdC1tZXRyaWNzL3Bpbm90LWNvbXBvdW5kLW1ldHJpY3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9tZXRyaWNzL2NvbXBvdW5kL0NvbXBvdW5kUGlub3RNZXRyaWNOYW1lLmphdmE=) | `0.00% <0.00%> (ø)` | |
   | [.../metrics/compound/CompoundPinotMetricRegistry.java](https://app.codecov.io/gh/apache/pinot/pull/9416?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtcGx1Z2lucy9waW5vdC1tZXRyaWNzL3Bpbm90LWNvbXBvdW5kLW1ldHJpY3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9tZXRyaWNzL2NvbXBvdW5kL0NvbXBvdW5kUGlub3RNZXRyaWNSZWdpc3RyeS5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [.../metrics/compound/CompoundPinotMetricsFactory.java](https://app.codecov.io/gh/apache/pinot/pull/9416?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtcGx1Z2lucy9waW5vdC1tZXRyaWNzL3Bpbm90LWNvbXBvdW5kLW1ldHJpY3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9tZXRyaWNzL2NvbXBvdW5kL0NvbXBvdW5kUGlub3RNZXRyaWNzRmFjdG9yeS5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [...ot/plugin/metrics/compound/CompoundPinotTimer.java](https://app.codecov.io/gh/apache/pinot/pull/9416?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtcGx1Z2lucy9waW5vdC1tZXRyaWNzL3Bpbm90LWNvbXBvdW5kLW1ldHJpY3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9tZXRyaWNzL2NvbXBvdW5kL0NvbXBvdW5kUGlub3RUaW1lci5qYXZh) | `0.00% <0.00%> (ø)` | |
   | ... and [4 more](https://app.codecov.io/gh/apache/pinot/pull/9416?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | |
   
   ... and [1724 files with indirect coverage changes](https://app.codecov.io/gh/apache/pinot/pull/9416/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   
   :mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] jackjlli commented on a diff in pull request #9416: Compatible dropwizard metrics

Posted by "jackjlli (via GitHub)" <gi...@apache.org>.
jackjlli commented on code in PR #9416:
URL: https://github.com/apache/pinot/pull/9416#discussion_r1222217645


##########
pinot-plugins/pinot-metrics/pinot-compound-metrics/src/main/java/org/apache/pinot/plugin/metrics/compound/CompoundPinotMetricsFactory.java:
##########
@@ -0,0 +1,170 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.plugin.metrics.compound;
+
+import com.google.common.base.Preconditions;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Locale;
+import java.util.ServiceLoader;
+import java.util.Set;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+import org.apache.pinot.spi.annotations.metrics.MetricsFactory;
+import org.apache.pinot.spi.annotations.metrics.PinotMetricsFactory;
+import org.apache.pinot.spi.env.PinotConfiguration;
+import org.apache.pinot.spi.metrics.PinotGauge;
+import org.apache.pinot.spi.metrics.PinotJmxReporter;
+import org.apache.pinot.spi.metrics.PinotMetricName;
+import org.apache.pinot.spi.metrics.PinotMetricUtils;
+import org.apache.pinot.spi.metrics.PinotMetricsRegistry;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+@MetricsFactory
+public class CompoundPinotMetricsFactory implements PinotMetricsFactory {

Review Comment:
   Great, thanks for that! 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org