You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by sh...@apache.org on 2022/07/13 15:45:14 UTC

[pulsar] branch master updated: Bump prometheus client version from 0.5.0 to 0.15.0 (#13785)

This is an automated email from the ASF dual-hosted git repository.

shoothzj pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git


The following commit(s) were added to refs/heads/master by this push:
     new c0b4d573ef9 Bump prometheus client version from 0.5.0 to 0.15.0 (#13785)
c0b4d573ef9 is described below

commit c0b4d573ef914ab799d35207e4eba82501b0453b
Author: ZhangJian He <sh...@gmail.com>
AuthorDate: Wed Jul 13 23:45:08 2022 +0800

    Bump prometheus client version from 0.5.0 to 0.15.0 (#13785)
    
    ### Motivation
    - prometheus `simpleclient` has broken api, it makes `pulsar-client-origin` can not well used with other libraries
    - specify `OpenMetrics`
    
    This PR contains metric name broken changes.
    
    ### Modifications
    Bump prometheus client version from 0.5.0 to 0.15.0
    
    ### changes
    - bump the client version
    - rename the field prometheus client `UNTYPED` to `UNKNOWN`
    - `OpenMetrics`'s counter name needs a `_total` suffix
    
    ### metrics name broken changes
    - rename `pulsar_expired_token_count` to `pulsar_expired_token_total`
    - rename `pulsar_authentication_success_count` to `pulsar_authentication_success_total`
    - rename `pulsar_authentication_failures_count` to `pulsar_authentication_failures_total`
    - rename `pulsar_sink_sink_exceptions_total_1min` to `pulsar_sink_sink_exceptions_1min_total`
    - rename `pulsar_source_received_total_1min` to `pulsar_source_received_1min_total`
    - rename `pulsar_source_written_total_1min` to `pulsar_source_written_1min_total`
    - rename `pulsar_source_source_total_1min` to `pulsar_source_source_exceptions_1min_total `
    - rename `pulsar_source_system_exceptions_total_1min` to `pulsar_source_system_exceptions_1min_total `
    - rename `pulsar_function_received_total_1min` to `pulsar_function_received_1min_total `
    - rename `pulsar_function_user_exceptions_total_1min` to `pulsar_function_user_exceptions_1min_total`
    - rename `pulsar_function_system_exceptions_total_1min` to `pulsar_function_system_exceptions_1min_total`
    - rename `pulsar_function_processed_successfully_total_1min` to `pulsar_function_processed_successfully_1min_total`
    - rename `pulsar_function_received_total_1min` to `pulsar_function_received_1min_total`
    - rename `pulsar_function_user_exceptions_total_1min` to `pulsar_function_user_exceptions_1min_total`
    - rename `pulsar_function_system_exceptions_total_1min` to `pulsar_function_system_exceptions_1min_total`
    - rename `pulsar_function_processed_successfully_total_1min` to `pulsar_function_processed_successfully_1min_total`
    - rename `pulsar_sink_received_total_1min` to `pulsar_sink_received_1min_total`
    - rename `pulsar_sink_written_total_1min` to `pulsar_sink_written_1min_total`
    - rename `pulsar_sink_sink_exceptions_total_1min` to `pulsar_sink_sink_exceptions_1min_total`
    - rename `pulsar_sink_system_exceptions_total_1min` to `pulsar_sink_system_exceptions_1min_total`
    - rename `pulsar_sink_received_total_1min` to `pulsar_sink_received_1min_total`
    - rename `pulsar_sink_written_total_1min` to `pulsar_sink_written_1min_total`
    - rename `pulsar_sink_sink_exceptions_total_1min` to `pulsar_sink_sink_exceptions_1min_total`
    - rename `pulsar_sink_system_exceptions_total_1min` to `pulsar_sink_system_exceptions_1min_total`
    
    ### Documentation
    
    Check the box below or label this PR directly.
    
    Need to update docs?
    
    - [x] `doc-required`
    As mentioned above, the metrics name has changed
---
 distribution/server/src/assemble/LICENSE.bin.txt   | 23 ++++++++++------
 pom.xml                                            |  4 +--
 .../AuthenticationProviderToken.java               |  2 +-
 .../metrics/AuthenticationMetrics.java             |  4 +--
 .../PrometheusMetricsGeneratorUtils.java           | 16 ++++++++---
 .../pulsar/broker/stats/PrometheusMetricsTest.java | 16 +++++++++--
 .../worker/PulsarFunctionLocalRunTest.java         |  2 +-
 .../apache/pulsar/io/PulsarBatchSourceE2ETest.java |  8 +++---
 .../apache/pulsar/io/PulsarFunctionE2ETest.java    | 16 +++++------
 .../org/apache/pulsar/io/PulsarSinkE2ETest.java    | 16 +++++------
 .../org/apache/pulsar/io/PulsarSourceE2ETest.java  |  8 +++---
 .../test/resources/prometheus_metrics_sample.txt   | 16 +++++------
 .../instance/stats/FunctionStatsManager.java       | 12 ++++----
 .../functions/instance/stats/SinkStatsManager.java |  8 +++---
 .../instance/stats/SourceStatsManager.java         |  8 +++---
 .../instance/src/main/python/function_stats.py     |  8 +++---
 .../proxy/server/ProxyPrometheusMetricsTest.java   |  4 +--
 pulsar-sql/presto-distribution/LICENSE             | 12 +++++---
 site2/docs/reference-metrics.md                    | 32 +++++++++++-----------
 19 files changed, 121 insertions(+), 94 deletions(-)

diff --git a/distribution/server/src/assemble/LICENSE.bin.txt b/distribution/server/src/assemble/LICENSE.bin.txt
index 34a3ca422b0..8f0b138d4d8 100644
--- a/distribution/server/src/assemble/LICENSE.bin.txt
+++ b/distribution/server/src/assemble/LICENSE.bin.txt
@@ -377,14 +377,19 @@ The Apache Software License, Version 2.0
     - io.netty-netty-tcnative-boringssl-static-2.0.52.Final-windows-x86_64.jar
     - io.netty-netty-tcnative-classes-2.0.52.Final.jar
  * Prometheus client
-    - io.prometheus-simpleclient-0.5.0.jar
-    - io.prometheus-simpleclient_common-0.5.0.jar
-    - io.prometheus-simpleclient_hotspot-0.5.0.jar
-    - io.prometheus-simpleclient_servlet-0.5.0.jar
-    - io.prometheus-simpleclient_log4j2-0.5.0.jar
-    - io.prometheus-simpleclient_jetty-0.5.0.jar
-    - io.prometheus.jmx-collector-0.14.0.jar
-    - io.prometheus-simpleclient_caffeine-0.5.0.jar
+    - io.prometheus.jmx-collector-0.16.1.jar
+    - io.prometheus-simpleclient-0.15.0.jar
+    - io.prometheus-simpleclient_caffeine-0.15.0.jar
+    - io.prometheus-simpleclient_common-0.15.0.jar
+    - io.prometheus-simpleclient_hotspot-0.15.0.jar
+    - io.prometheus-simpleclient_httpserver-0.15.0.jar
+    - io.prometheus-simpleclient_jetty-0.15.0.jar
+    - io.prometheus-simpleclient_log4j2-0.15.0.jar
+    - io.prometheus-simpleclient_servlet-0.15.0.jar
+    - io.prometheus-simpleclient_servlet_common-0.15.0.jar
+    - io.prometheus-simpleclient_tracer_common-0.15.0.jar
+    - io.prometheus-simpleclient_tracer_otel-0.15.0.jar
+    - io.prometheus-simpleclient_tracer_otel_agent-0.15.0.jar
  * Jakarta Bean Validation API
     - jakarta.validation-jakarta.validation-api-2.0.2.jar
     - javax.validation-validation-api-1.1.0.Final.jar
@@ -508,7 +513,7 @@ The Apache Software License, Version 2.0
     - io.dropwizard.metrics-metrics-jvm-4.1.12.1.jar
     - io.dropwizard.metrics-metrics-jmx-4.1.12.1.jar
   * Prometheus
-    - io.prometheus-simpleclient_httpserver-0.5.0.jar
+    - io.prometheus-simpleclient_httpserver-0.15.0.jar
   * Java JSON WebTokens
     - io.jsonwebtoken-jjwt-api-0.11.1.jar
     - io.jsonwebtoken-jjwt-impl-0.11.1.jar
diff --git a/pom.xml b/pom.xml
index 50a7217da71..cb35cf9ce7c 100644
--- a/pom.xml
+++ b/pom.xml
@@ -126,7 +126,7 @@ flexible messaging model and an intuitive client API.</description>
     <conscrypt.version>2.5.2</conscrypt.version>
     <jersey.version>2.34</jersey.version>
     <athenz.version>1.10.50</athenz.version>
-    <prometheus.version>0.5.0</prometheus.version>
+    <prometheus.version>0.15.0</prometheus.version>
     <vertx.version>3.9.8</vertx.version>
     <rocksdb.version>6.29.4.1</rocksdb.version>
     <slf4j.version>1.7.32</slf4j.version>
@@ -179,7 +179,7 @@ flexible messaging model and an intuitive client API.</description>
     <hbase.version>2.4.9</hbase.version>
     <guava.version>31.0.1-jre</guava.version>
     <jcip.version>1.0</jcip.version>
-    <prometheus-jmx.version>0.14.0</prometheus-jmx.version>
+    <prometheus-jmx.version>0.16.1</prometheus-jmx.version>
     <confluent.version>7.0.1</confluent.version>
     <kafka.confluent.schemaregistryclient.version>5.3.0</kafka.confluent.schemaregistryclient.version>
     <kafka.confluent.avroserializer.version>5.3.0</kafka.confluent.avroserializer.version>
diff --git a/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationProviderToken.java b/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationProviderToken.java
index ccd4e8e1aab..5c6d17a1d77 100644
--- a/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationProviderToken.java
+++ b/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationProviderToken.java
@@ -74,7 +74,7 @@ public class AuthenticationProviderToken implements AuthenticationProvider {
     static final String TOKEN = "token";
 
     private static final Counter expiredTokenMetrics = Counter.build()
-            .name("pulsar_expired_token_count")
+            .name("pulsar_expired_token_total")
             .help("Pulsar expired token")
             .register();
 
diff --git a/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/metrics/AuthenticationMetrics.java b/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/metrics/AuthenticationMetrics.java
index 60565db521d..b4ae44e200d 100644
--- a/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/metrics/AuthenticationMetrics.java
+++ b/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/metrics/AuthenticationMetrics.java
@@ -22,12 +22,12 @@ import io.prometheus.client.Counter;
 
 public class AuthenticationMetrics {
     private static final Counter authSuccessMetrics = Counter.build()
-            .name("pulsar_authentication_success_count")
+            .name("pulsar_authentication_success_total")
             .help("Pulsar authentication success")
             .labelNames("provider_name", "auth_method")
             .register();
     private static final Counter authFailuresMetrics = Counter.build()
-            .name("pulsar_authentication_failures_count")
+            .name("pulsar_authentication_failures_total")
             .help("Pulsar authentication failures")
             .labelNames("provider_name", "auth_method", "reason")
             .register();
diff --git a/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/stats/prometheus/PrometheusMetricsGeneratorUtils.java b/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/stats/prometheus/PrometheusMetricsGeneratorUtils.java
index ead3c332b2b..d341e699bf7 100644
--- a/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/stats/prometheus/PrometheusMetricsGeneratorUtils.java
+++ b/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/stats/prometheus/PrometheusMetricsGeneratorUtils.java
@@ -59,7 +59,7 @@ public class PrometheusMetricsGeneratorUtils {
             Collector.MetricFamilySamples metricFamily = metricFamilySamples.nextElement();
 
             // Write type of metric
-            stream.write("# TYPE ").write(metricFamily.name).write(' ')
+            stream.write("# TYPE ").write(metricFamily.name).write(getTypeNameSuffix(metricFamily.type)).write(' ')
                     .write(getTypeStr(metricFamily.type)).write('\n');
 
             for (int i = 0; i < metricFamily.samples.size(); i++) {
@@ -88,19 +88,27 @@ public class PrometheusMetricsGeneratorUtils {
         }
     }
 
+    static String getTypeNameSuffix(Collector.Type type) {
+        if (type.equals(Collector.Type.INFO)) {
+            return "_info";
+        }
+        return "";
+    }
+
     static String getTypeStr(Collector.Type type) {
         switch (type) {
             case COUNTER:
                 return "counter";
             case GAUGE:
+            case INFO:
                 return "gauge";
-            case SUMMARY        :
+            case SUMMARY:
                 return "summary";
             case HISTOGRAM:
                 return "histogram";
-            case UNTYPED:
+            case UNKNOWN:
             default:
-                return "untyped";
+                return "unknown";
         }
     }
 
diff --git a/pulsar-broker/src/test/java/org/apache/pulsar/broker/stats/PrometheusMetricsTest.java b/pulsar-broker/src/test/java/org/apache/pulsar/broker/stats/PrometheusMetricsTest.java
index 6663410e118..8eccf1785e4 100644
--- a/pulsar-broker/src/test/java/org/apache/pulsar/broker/stats/PrometheusMetricsTest.java
+++ b/pulsar-broker/src/test/java/org/apache/pulsar/broker/stats/PrometheusMetricsTest.java
@@ -847,6 +847,16 @@ public class PrometheusMetricsTest extends BrokerTestBase {
                     if (!typeDefs.containsKey(summaryMetricName)) {
                         fail("Metric " + metricName + " does not have a corresponding summary type definition");
                     }
+                } else if (metricName.endsWith("_created")) {
+                    String summaryMetricName = metricName.substring(0, metricName.indexOf("_created"));
+                    if (!typeDefs.containsKey(summaryMetricName)) {
+                        fail("Metric " + metricName + " does not have a corresponding summary type definition");
+                    }
+                } else if (metricName.endsWith("_total")) {
+                    String summaryMetricName = metricName.substring(0, metricName.indexOf("_total"));
+                    if (!typeDefs.containsKey(summaryMetricName)) {
+                        fail("Metric " + metricName + " does not have a corresponding counter type definition");
+                    }
                 } else {
                     fail("Metric " + metricName + " does not have a type definition");
                 }
@@ -1087,7 +1097,7 @@ public class PrometheusMetricsTest extends BrokerTestBase {
         PrometheusMetricsGenerator.generate(pulsar, false, false, false, statsOut);
         String metricsStr = statsOut.toString();
         Multimap<String, Metric> metrics = parseMetrics(metricsStr);
-        List<Metric> cm = (List<Metric>) metrics.get("pulsar_authentication_success_count");
+        List<Metric> cm = (List<Metric>) metrics.get("pulsar_authentication_success_total");
         boolean haveSucceed = false;
         for (Metric metric : cm) {
             if (Objects.equals(metric.tags.get("auth_method"), "token")
@@ -1097,7 +1107,7 @@ public class PrometheusMetricsTest extends BrokerTestBase {
         }
         Assert.assertTrue(haveSucceed);
 
-        cm = (List<Metric>) metrics.get("pulsar_authentication_failures_count");
+        cm = (List<Metric>) metrics.get("pulsar_authentication_failures_total");
 
         boolean haveFailed = false;
         for (Metric metric : cm) {
@@ -1147,7 +1157,7 @@ public class PrometheusMetricsTest extends BrokerTestBase {
         PrometheusMetricsGenerator.generate(pulsar, false, false, false, statsOut);
         String metricsStr = statsOut.toString();
         Multimap<String, Metric> metrics = parseMetrics(metricsStr);
-        List<Metric> cm = (List<Metric>) metrics.get("pulsar_expired_token_count");
+        List<Metric> cm = (List<Metric>) metrics.get("pulsar_expired_token_total");
         assertEquals(cm.size(), 1);
 
         provider.close();
diff --git a/pulsar-broker/src/test/java/org/apache/pulsar/functions/worker/PulsarFunctionLocalRunTest.java b/pulsar-broker/src/test/java/org/apache/pulsar/functions/worker/PulsarFunctionLocalRunTest.java
index a8344be8dc8..e810a1d7d93 100644
--- a/pulsar-broker/src/test/java/org/apache/pulsar/functions/worker/PulsarFunctionLocalRunTest.java
+++ b/pulsar-broker/src/test/java/org/apache/pulsar/functions/worker/PulsarFunctionLocalRunTest.java
@@ -987,7 +987,7 @@ public class PulsarFunctionLocalRunTest {
                 assertFalse(metrics.isEmpty());
                 PulsarFunctionTestUtils.Metric m = metrics.get("pulsar_sink_sink_exceptions_total");
                 if (m == null) {
-                    m = metrics.get("pulsar_sink_sink_exceptions_total_1min");
+                    m = metrics.get("pulsar_sink_sink_exceptions_1min_total");
                 }
                 assertEquals(m.value, 0);
             }
diff --git a/pulsar-broker/src/test/java/org/apache/pulsar/io/PulsarBatchSourceE2ETest.java b/pulsar-broker/src/test/java/org/apache/pulsar/io/PulsarBatchSourceE2ETest.java
index d1cd1dc0614..e371190a0d0 100644
--- a/pulsar-broker/src/test/java/org/apache/pulsar/io/PulsarBatchSourceE2ETest.java
+++ b/pulsar-broker/src/test/java/org/apache/pulsar/io/PulsarBatchSourceE2ETest.java
@@ -113,7 +113,7 @@ public class PulsarBatchSourceE2ETest extends AbstractPulsarE2ETest {
         assertEquals(m.tags.get("namespace"), String.format("%s/%s", tenant, namespacePortion));
         assertEquals(m.tags.get("fqfn"), FunctionCommon.getFullyQualifiedName(tenant, namespacePortion, sourceName));
         assertTrue(m.value > 0.0);
-        m = metrics.get("pulsar_source_received_total_1min");
+        m = metrics.get("pulsar_source_received_1min_total");
         assertEquals(m.tags.get("cluster"), config.getClusterName());
         assertEquals(m.tags.get("instance_id"), "0");
         assertEquals(m.tags.get("name"), sourceName);
@@ -127,7 +127,7 @@ public class PulsarBatchSourceE2ETest extends AbstractPulsarE2ETest {
         assertEquals(m.tags.get("namespace"), String.format("%s/%s", tenant, namespacePortion));
         assertEquals(m.tags.get("fqfn"), FunctionCommon.getFullyQualifiedName(tenant, namespacePortion, sourceName));
         assertTrue(m.value > 0.0);
-        m = metrics.get("pulsar_source_written_total_1min");
+        m = metrics.get("pulsar_source_written_1min_total");
         assertEquals(m.tags.get("cluster"), config.getClusterName());
         assertEquals(m.tags.get("instance_id"), "0");
         assertEquals(m.tags.get("name"), sourceName);
@@ -141,7 +141,7 @@ public class PulsarBatchSourceE2ETest extends AbstractPulsarE2ETest {
         assertEquals(m.tags.get("namespace"), String.format("%s/%s", tenant, namespacePortion));
         assertEquals(m.tags.get("fqfn"), FunctionCommon.getFullyQualifiedName(tenant, namespacePortion, sourceName));
         assertEquals(m.value, 0.0);
-        m = metrics.get("pulsar_source_source_exceptions_total_1min");
+        m = metrics.get("pulsar_source_source_exceptions_1min_total");
         assertEquals(m.tags.get("cluster"), config.getClusterName());
         assertEquals(m.tags.get("instance_id"), "0");
         assertEquals(m.tags.get("name"), sourceName);
@@ -155,7 +155,7 @@ public class PulsarBatchSourceE2ETest extends AbstractPulsarE2ETest {
         assertEquals(m.tags.get("namespace"), String.format("%s/%s", tenant, namespacePortion));
         assertEquals(m.tags.get("fqfn"), FunctionCommon.getFullyQualifiedName(tenant, namespacePortion, sourceName));
         assertEquals(m.value, 0.0);
-        m = metrics.get("pulsar_source_system_exceptions_total_1min");
+        m = metrics.get("pulsar_source_system_exceptions_1min_total");
         assertEquals(m.tags.get("cluster"), config.getClusterName());
         assertEquals(m.tags.get("instance_id"), "0");
         assertEquals(m.tags.get("name"), sourceName);
diff --git a/pulsar-broker/src/test/java/org/apache/pulsar/io/PulsarFunctionE2ETest.java b/pulsar-broker/src/test/java/org/apache/pulsar/io/PulsarFunctionE2ETest.java
index 32ea853f269..1502862d676 100644
--- a/pulsar-broker/src/test/java/org/apache/pulsar/io/PulsarFunctionE2ETest.java
+++ b/pulsar-broker/src/test/java/org/apache/pulsar/io/PulsarFunctionE2ETest.java
@@ -384,7 +384,7 @@ public class PulsarFunctionE2ETest extends AbstractPulsarE2ETest {
         assertEquals(m.tags.get("namespace"), String.format("%s/%s", tenant, namespacePortion));
         assertEquals(m.tags.get("fqfn"), FunctionCommon.getFullyQualifiedName(tenant, namespacePortion, functionName));
         assertEquals(m.value, 0.0);
-        m = metrics.get("pulsar_function_received_total_1min");
+        m = metrics.get("pulsar_function_received_1min_total");
         assertEquals(m.tags.get("cluster"), config.getClusterName());
         assertEquals(m.tags.get("instance_id"), "0");
         assertEquals(m.tags.get("name"), functionName);
@@ -398,7 +398,7 @@ public class PulsarFunctionE2ETest extends AbstractPulsarE2ETest {
         assertEquals(m.tags.get("namespace"), String.format("%s/%s", tenant, namespacePortion));
         assertEquals(m.tags.get("fqfn"), FunctionCommon.getFullyQualifiedName(tenant, namespacePortion, functionName));
         assertEquals(m.value, 0.0);
-        m = metrics.get("pulsar_function_user_exceptions_total_1min");
+        m = metrics.get("pulsar_function_user_exceptions_1min_total");
         assertEquals(m.tags.get("cluster"), config.getClusterName());
         assertEquals(m.tags.get("instance_id"), "0");
         assertEquals(m.tags.get("name"), functionName);
@@ -426,7 +426,7 @@ public class PulsarFunctionE2ETest extends AbstractPulsarE2ETest {
         assertEquals(m.tags.get("namespace"), String.format("%s/%s", tenant, namespacePortion));
         assertEquals(m.tags.get("fqfn"), FunctionCommon.getFullyQualifiedName(tenant, namespacePortion, functionName));
         assertEquals(m.value, 0.0);
-        m = metrics.get("pulsar_function_system_exceptions_total_1min");
+        m = metrics.get("pulsar_function_system_exceptions_1min_total");
         assertEquals(m.tags.get("cluster"), config.getClusterName());
         assertEquals(m.tags.get("instance_id"), "0");
         assertEquals(m.tags.get("name"), functionName);
@@ -447,7 +447,7 @@ public class PulsarFunctionE2ETest extends AbstractPulsarE2ETest {
         assertEquals(m.tags.get("namespace"), String.format("%s/%s", tenant, namespacePortion));
         assertEquals(m.tags.get("fqfn"), FunctionCommon.getFullyQualifiedName(tenant, namespacePortion, functionName));
         assertEquals(m.value, 0.0);
-        m = metrics.get("pulsar_function_processed_successfully_total_1min");
+        m = metrics.get("pulsar_function_processed_successfully_1min_total");
         assertEquals(m.tags.get("cluster"), config.getClusterName());
         assertEquals(m.tags.get("instance_id"), "0");
         assertEquals(m.tags.get("name"), functionName);
@@ -544,7 +544,7 @@ public class PulsarFunctionE2ETest extends AbstractPulsarE2ETest {
         assertEquals(m.tags.get("namespace"), String.format("%s/%s", tenant, namespacePortion));
         assertEquals(m.tags.get("fqfn"), FunctionCommon.getFullyQualifiedName(tenant, namespacePortion, functionName));
         assertEquals(m.value, (double) totalMsgs);
-        m = metrics.get("pulsar_function_received_total_1min");
+        m = metrics.get("pulsar_function_received_1min_total");
         assertEquals(m.tags.get("cluster"), config.getClusterName());
         assertEquals(m.tags.get("instance_id"), "0");
         assertEquals(m.tags.get("name"), functionName);
@@ -558,7 +558,7 @@ public class PulsarFunctionE2ETest extends AbstractPulsarE2ETest {
         assertEquals(m.tags.get("namespace"), String.format("%s/%s", tenant, namespacePortion));
         assertEquals(m.tags.get("fqfn"), FunctionCommon.getFullyQualifiedName(tenant, namespacePortion, functionName));
         assertEquals(m.value, 0.0);
-        m = metrics.get("pulsar_function_user_exceptions_total_1min");
+        m = metrics.get("pulsar_function_user_exceptions_1min_total");
         assertEquals(m.tags.get("cluster"), config.getClusterName());
         assertEquals(m.tags.get("instance_id"), "0");
         assertEquals(m.tags.get("name"), functionName);
@@ -586,7 +586,7 @@ public class PulsarFunctionE2ETest extends AbstractPulsarE2ETest {
         assertEquals(m.tags.get("namespace"), String.format("%s/%s", tenant, namespacePortion));
         assertEquals(m.tags.get("fqfn"), FunctionCommon.getFullyQualifiedName(tenant, namespacePortion, functionName));
         assertEquals(m.value, 0.0);
-        m = metrics.get("pulsar_function_system_exceptions_total_1min");
+        m = metrics.get("pulsar_function_system_exceptions_1min_total");
         assertEquals(m.tags.get("cluster"), config.getClusterName());
         assertEquals(m.tags.get("instance_id"), "0");
         assertEquals(m.tags.get("name"), functionName);
@@ -607,7 +607,7 @@ public class PulsarFunctionE2ETest extends AbstractPulsarE2ETest {
         assertEquals(m.tags.get("namespace"), String.format("%s/%s", tenant, namespacePortion));
         assertEquals(m.tags.get("fqfn"), FunctionCommon.getFullyQualifiedName(tenant, namespacePortion, functionName));
         assertEquals(m.value, (double) totalMsgs);
-        m = metrics.get("pulsar_function_processed_successfully_total_1min");
+        m = metrics.get("pulsar_function_processed_successfully_1min_total");
         assertEquals(m.tags.get("cluster"), config.getClusterName());
         assertEquals(m.tags.get("instance_id"), "0");
         assertEquals(m.tags.get("name"), functionName);
diff --git a/pulsar-broker/src/test/java/org/apache/pulsar/io/PulsarSinkE2ETest.java b/pulsar-broker/src/test/java/org/apache/pulsar/io/PulsarSinkE2ETest.java
index 4b7578dc606..24bea1d5994 100644
--- a/pulsar-broker/src/test/java/org/apache/pulsar/io/PulsarSinkE2ETest.java
+++ b/pulsar-broker/src/test/java/org/apache/pulsar/io/PulsarSinkE2ETest.java
@@ -284,7 +284,7 @@ public class PulsarSinkE2ETest extends AbstractPulsarE2ETest {
         assertEquals(m.tags.get("namespace"), String.format("%s/%s", tenant, namespacePortion));
         assertEquals(m.tags.get("fqfn"), FunctionCommon.getFullyQualifiedName(tenant, namespacePortion, sinkName));
         assertEquals(m.value, 0.0);
-        m = metrics.get("pulsar_sink_received_total_1min");
+        m = metrics.get("pulsar_sink_received_1min_total");
         assertEquals(m.tags.get("cluster"), config.getClusterName());
         assertEquals(m.tags.get("instance_id"), "0");
         assertEquals(m.tags.get("name"), sinkName);
@@ -298,7 +298,7 @@ public class PulsarSinkE2ETest extends AbstractPulsarE2ETest {
         assertEquals(m.tags.get("namespace"), String.format("%s/%s", tenant, namespacePortion));
         assertEquals(m.tags.get("fqfn"), FunctionCommon.getFullyQualifiedName(tenant, namespacePortion, sinkName));
         assertEquals(m.value, 0.0);
-        m = metrics.get("pulsar_sink_written_total_1min");
+        m = metrics.get("pulsar_sink_written_1min_total");
         assertEquals(m.tags.get("cluster"), config.getClusterName());
         assertEquals(m.tags.get("instance_id"), "0");
         assertEquals(m.tags.get("name"), sinkName);
@@ -312,7 +312,7 @@ public class PulsarSinkE2ETest extends AbstractPulsarE2ETest {
         assertEquals(m.tags.get("namespace"), String.format("%s/%s", tenant, namespacePortion));
         assertEquals(m.tags.get("fqfn"), FunctionCommon.getFullyQualifiedName(tenant, namespacePortion, sinkName));
         assertEquals(m.value, 0.0);
-        m = metrics.get("pulsar_sink_sink_exceptions_total_1min");
+        m = metrics.get("pulsar_sink_sink_exceptions_1min_total");
         assertEquals(m.tags.get("cluster"), config.getClusterName());
         assertEquals(m.tags.get("instance_id"), "0");
         assertEquals(m.tags.get("name"), sinkName);
@@ -326,7 +326,7 @@ public class PulsarSinkE2ETest extends AbstractPulsarE2ETest {
         assertEquals(m.tags.get("namespace"), String.format("%s/%s", tenant, namespacePortion));
         assertEquals(m.tags.get("fqfn"), FunctionCommon.getFullyQualifiedName(tenant, namespacePortion, sinkName));
         assertEquals(m.value, 0.0);
-        m = metrics.get("pulsar_sink_system_exceptions_total_1min");
+        m = metrics.get("pulsar_sink_system_exceptions_1min_total");
         assertEquals(m.tags.get("cluster"), config.getClusterName());
         assertEquals(m.tags.get("instance_id"), "0");
         assertEquals(m.tags.get("name"), sinkName);
@@ -377,7 +377,7 @@ public class PulsarSinkE2ETest extends AbstractPulsarE2ETest {
         assertEquals(m.tags.get("namespace"), String.format("%s/%s", tenant, namespacePortion));
         assertEquals(m.tags.get("fqfn"), FunctionCommon.getFullyQualifiedName(tenant, namespacePortion, sinkName));
         assertEquals(m.value, (double) totalMsgs);
-        m = metrics.get("pulsar_sink_received_total_1min");
+        m = metrics.get("pulsar_sink_received_1min_total");
         assertEquals(m.tags.get("cluster"), config.getClusterName());
         assertEquals(m.tags.get("instance_id"), "0");
         assertEquals(m.tags.get("name"), sinkName);
@@ -391,7 +391,7 @@ public class PulsarSinkE2ETest extends AbstractPulsarE2ETest {
         assertEquals(m.tags.get("namespace"), String.format("%s/%s", tenant, namespacePortion));
         assertEquals(m.tags.get("fqfn"), FunctionCommon.getFullyQualifiedName(tenant, namespacePortion, sinkName));
         assertEquals(m.value, (double) totalMsgs);
-        m = metrics.get("pulsar_sink_written_total_1min");
+        m = metrics.get("pulsar_sink_written_1min_total");
         assertEquals(m.tags.get("cluster"), config.getClusterName());
         assertEquals(m.tags.get("instance_id"), "0");
         assertEquals(m.tags.get("name"), sinkName);
@@ -405,7 +405,7 @@ public class PulsarSinkE2ETest extends AbstractPulsarE2ETest {
         assertEquals(m.tags.get("namespace"), String.format("%s/%s", tenant, namespacePortion));
         assertEquals(m.tags.get("fqfn"), FunctionCommon.getFullyQualifiedName(tenant, namespacePortion, sinkName));
         assertEquals(m.value, 0.0);
-        m = metrics.get("pulsar_sink_sink_exceptions_total_1min");
+        m = metrics.get("pulsar_sink_sink_exceptions_1min_total");
         assertEquals(m.tags.get("cluster"), config.getClusterName());
         assertEquals(m.tags.get("instance_id"), "0");
         assertEquals(m.tags.get("name"), sinkName);
@@ -419,7 +419,7 @@ public class PulsarSinkE2ETest extends AbstractPulsarE2ETest {
         assertEquals(m.tags.get("namespace"), String.format("%s/%s", tenant, namespacePortion));
         assertEquals(m.tags.get("fqfn"), FunctionCommon.getFullyQualifiedName(tenant, namespacePortion, sinkName));
         assertEquals(m.value, 0.0);
-        m = metrics.get("pulsar_sink_system_exceptions_total_1min");
+        m = metrics.get("pulsar_sink_system_exceptions_1min_total");
         assertEquals(m.tags.get("cluster"), config.getClusterName());
         assertEquals(m.tags.get("instance_id"), "0");
         assertEquals(m.tags.get("name"), sinkName);
diff --git a/pulsar-broker/src/test/java/org/apache/pulsar/io/PulsarSourceE2ETest.java b/pulsar-broker/src/test/java/org/apache/pulsar/io/PulsarSourceE2ETest.java
index bca1bbc477e..2dea2fb020e 100644
--- a/pulsar-broker/src/test/java/org/apache/pulsar/io/PulsarSourceE2ETest.java
+++ b/pulsar-broker/src/test/java/org/apache/pulsar/io/PulsarSourceE2ETest.java
@@ -117,7 +117,7 @@ public class PulsarSourceE2ETest extends AbstractPulsarE2ETest {
         assertEquals(m.tags.get("namespace"), String.format("%s/%s", tenant, namespacePortion));
         assertEquals(m.tags.get("fqfn"), FunctionCommon.getFullyQualifiedName(tenant, namespacePortion, sourceName));
         assertTrue(m.value > 0.0);
-        m = metrics.get("pulsar_source_received_total_1min");
+        m = metrics.get("pulsar_source_received_1min_total");
         assertEquals(m.tags.get("cluster"), config.getClusterName());
         assertEquals(m.tags.get("instance_id"), "0");
         assertEquals(m.tags.get("name"), sourceName);
@@ -131,7 +131,7 @@ public class PulsarSourceE2ETest extends AbstractPulsarE2ETest {
         assertEquals(m.tags.get("namespace"), String.format("%s/%s", tenant, namespacePortion));
         assertEquals(m.tags.get("fqfn"), FunctionCommon.getFullyQualifiedName(tenant, namespacePortion, sourceName));
         assertTrue(m.value > 0.0);
-        m = metrics.get("pulsar_source_written_total_1min");
+        m = metrics.get("pulsar_source_written_1min_total");
         assertEquals(m.tags.get("cluster"), config.getClusterName());
         assertEquals(m.tags.get("instance_id"), "0");
         assertEquals(m.tags.get("name"), sourceName);
@@ -145,7 +145,7 @@ public class PulsarSourceE2ETest extends AbstractPulsarE2ETest {
         assertEquals(m.tags.get("namespace"), String.format("%s/%s", tenant, namespacePortion));
         assertEquals(m.tags.get("fqfn"), FunctionCommon.getFullyQualifiedName(tenant, namespacePortion, sourceName));
         assertEquals(m.value, 0.0);
-        m = metrics.get("pulsar_source_source_exceptions_total_1min");
+        m = metrics.get("pulsar_source_source_exceptions_1min_total");
         assertEquals(m.tags.get("cluster"), config.getClusterName());
         assertEquals(m.tags.get("instance_id"), "0");
         assertEquals(m.tags.get("name"), sourceName);
@@ -159,7 +159,7 @@ public class PulsarSourceE2ETest extends AbstractPulsarE2ETest {
         assertEquals(m.tags.get("namespace"), String.format("%s/%s", tenant, namespacePortion));
         assertEquals(m.tags.get("fqfn"), FunctionCommon.getFullyQualifiedName(tenant, namespacePortion, sourceName));
         assertEquals(m.value, 0.0);
-        m = metrics.get("pulsar_source_system_exceptions_total_1min");
+        m = metrics.get("pulsar_source_system_exceptions_1min_total");
         assertEquals(m.tags.get("cluster"), config.getClusterName());
         assertEquals(m.tags.get("instance_id"), "0");
         assertEquals(m.tags.get("name"), sourceName);
diff --git a/pulsar-broker/src/test/resources/prometheus_metrics_sample.txt b/pulsar-broker/src/test/resources/prometheus_metrics_sample.txt
index 20c453c1f4c..5df34377d10 100644
--- a/pulsar-broker/src/test/resources/prometheus_metrics_sample.txt
+++ b/pulsar-broker/src/test/resources/prometheus_metrics_sample.txt
@@ -83,8 +83,8 @@ jvm_buffer_pool_used_buffers{cluster="use",pool="direct"} 97.0
 jvm_buffer_pool_used_buffers{cluster="use",pool="mapped"} 0.0
 # TYPE pulsar_broker_lookup_pending_requests gauge
 pulsar_broker_lookup_pending_requests{cluster="use"} 0.0
-# TYPE pulsar_authentication_success_count counter
-pulsar_authentication_success_count{cluster="use",provider_name="AuthenticationProviderTls",auth_method="tls"} 850.0
+# TYPE pulsar_authentication_success_total counter
+pulsar_authentication_success_total{cluster="use",provider_name="AuthenticationProviderTls",auth_method="tls"} 850.0
 # TYPE pulsar_version_info gauge
 pulsar_version_info{cluster="use",version="2.8.0-SNAPSHOT",commit="e600b65a05e610bc7cbd874d4c446619d9d9606f"} 1.0
 # TYPE zk_read_latency summary
@@ -311,8 +311,8 @@ jvm_gc_collection_seconds_count{cluster="use",gc="G1 Old Generation"} 0.0
 jvm_gc_collection_seconds_sum{cluster="use",gc="G1 Old Generation"} 0.0
 # TYPE pulsar_broker_lookup_failures counter
 pulsar_broker_lookup_failures{cluster="use"} 0.0
-# TYPE pulsar_authentication_failures_count counter
-pulsar_authentication_failures_count{cluster="use",provider_name="AuthenticationProviderTls",auth_method="tls",reason="Client unable to authenticate with TLS certificate"} 1.0
+# TYPE pulsar_authentication_failures_total counter
+pulsar_authentication_failures_total{cluster="use",provider_name="AuthenticationProviderTls",auth_method="tls",reason="Client unable to authenticate with TLS certificate"} 1.0
 # TYPE pulsar_broker_lookup_answers counter
 pulsar_broker_lookup_answers{cluster="use"} 134.0
 # TYPE pulsar_topics_count gauge
@@ -665,8 +665,8 @@ pulsar_function_worker_total_function_count{cluster="use",} 1
 pulsar_function_worker_total_expected_instance_count{cluster="use",} 1
 pulsar_function_worker_is_leader{cluster="use",} 1
 pulsar_function_last_invocation{tenant="external-repl-prop",namespace="external-repl-prop/io",name="PulsarSink-test",instance_id="0",cluster="use",fqfn="external-repl-prop/io/PulsarSink-test",} 0.0
-pulsar_function_processed_successfully_total_1min{tenant="external-repl-prop",namespace="external-repl-prop/io",name="PulsarSink-test",instance_id="0",cluster="use",fqfn="external-repl-prop/io/PulsarSink-test",} 0.0
-pulsar_function_received_total_1min{tenant="external-repl-prop",namespace="external-repl-prop/io",name="PulsarSink-test",instance_id="0",cluster="use",fqfn="external-repl-prop/io/PulsarSink-test",} 0.0
+pulsar_function_processed_successfully_1min_total{tenant="external-repl-prop",namespace="external-repl-prop/io",name="PulsarSink-test",instance_id="0",cluster="use",fqfn="external-repl-prop/io/PulsarSink-test",} 0.0
+pulsar_function_received_1min_total{tenant="external-repl-prop",namespace="external-repl-prop/io",name="PulsarSink-test",instance_id="0",cluster="use",fqfn="external-repl-prop/io/PulsarSink-test",} 0.0
 pulsar_function_user_exceptions_total{tenant="external-repl-prop",namespace="external-repl-prop/io",name="PulsarSink-test",instance_id="0",cluster="use",fqfn="external-repl-prop/io/PulsarSink-test",} 0.0
 pulsar_function_process_latency_ms_1min{tenant="external-repl-prop",namespace="external-repl-prop/io",name="PulsarSink-test",instance_id="0",cluster="use",fqfn="external-repl-prop/io/PulsarSink-test",quantile="0.5",} NaN
 pulsar_function_process_latency_ms_1min{tenant="external-repl-prop",namespace="external-repl-prop/io",name="PulsarSink-test",instance_id="0",cluster="use",fqfn="external-repl-prop/io/PulsarSink-test",quantile="0.9",} NaN
@@ -683,8 +683,8 @@ pulsar_function_process_latency_ms{tenant="external-repl-prop",namespace="extern
 pulsar_function_process_latency_ms{tenant="external-repl-prop",namespace="external-repl-prop/io",name="PulsarSink-test",instance_id="0",cluster="use",fqfn="external-repl-prop/io/PulsarSink-test",quantile="0.999",} NaN
 pulsar_function_process_latency_ms_count{tenant="external-repl-prop",namespace="external-repl-prop/io",name="PulsarSink-test",instance_id="0",cluster="use",fqfn="external-repl-prop/io/PulsarSink-test",} 0.0
 pulsar_function_process_latency_ms_sum{tenant="external-repl-prop",namespace="external-repl-prop/io",name="PulsarSink-test",instance_id="0",cluster="use",fqfn="external-repl-prop/io/PulsarSink-test",} 0.0
-pulsar_function_system_exceptions_total_1min{tenant="external-repl-prop",namespace="external-repl-prop/io",name="PulsarSink-test",instance_id="0",cluster="use",fqfn="external-repl-prop/io/PulsarSink-test",} 0.0
-pulsar_function_user_exceptions_total_1min{tenant="external-repl-prop",namespace="external-repl-prop/io",name="PulsarSink-test",instance_id="0",cluster="use",fqfn="external-repl-prop/io/PulsarSink-test",} 0.0
+pulsar_function_system_exceptions_1min_total{tenant="external-repl-prop",namespace="external-repl-prop/io",name="PulsarSink-test",instance_id="0",cluster="use",fqfn="external-repl-prop/io/PulsarSink-test",} 0.0
+pulsar_function_user_exceptions_1min_total{tenant="external-repl-prop",namespace="external-repl-prop/io",name="PulsarSink-test",instance_id="0",cluster="use",fqfn="external-repl-prop/io/PulsarSink-test",} 0.0
 # TYPE pulsar_ml_cache_evictions gauge
 pulsar_ml_cache_evictions{cluster="use"} 0 1617950344971
 # TYPE pulsar_ml_cache_hits_rate gauge
diff --git a/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/stats/FunctionStatsManager.java b/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/stats/FunctionStatsManager.java
index f693b2778c2..c15262fb0f6 100644
--- a/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/stats/FunctionStatsManager.java
+++ b/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/stats/FunctionStatsManager.java
@@ -51,13 +51,13 @@ public class FunctionStatsManager extends ComponentStatsManager {
     public static final String LAST_INVOCATION = "last_invocation";
     public static final String RECEIVED_TOTAL = "received_total";
 
-    public static final String PROCESSED_SUCCESSFULLY_TOTAL_1min = "processed_successfully_total_1min";
-    public static final String SYSTEM_EXCEPTIONS_TOTAL_1min = "system_exceptions_total_1min";
-    public static final String USER_EXCEPTIONS_TOTAL_1min = "user_exceptions_total_1min";
-    public static final String SOURCE_EXCEPTIONS_TOTAL_1min = "source_exceptions_total_1min";
-    public static final String SINK_EXCEPTIONS_TOTAL_1min = "sink_exceptions_total_1min";
+    public static final String PROCESSED_SUCCESSFULLY_TOTAL_1min = "processed_successfully_1min";
+    public static final String SYSTEM_EXCEPTIONS_TOTAL_1min = "system_exceptions_1min";
+    public static final String USER_EXCEPTIONS_TOTAL_1min = "user_exceptions_1min";
+    public static final String SOURCE_EXCEPTIONS_TOTAL_1min = "source_exceptions_1min";
+    public static final String SINK_EXCEPTIONS_TOTAL_1min = "sink_exceptions_1min";
     public static final String PROCESS_LATENCY_MS_1min = "process_latency_ms_1min";
-    public static final String RECEIVED_TOTAL_1min = "received_total_1min";
+    public static final String RECEIVED_TOTAL_1min = "received_1min";
 
     /** Declare Prometheus stats. **/
 
diff --git a/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/stats/SinkStatsManager.java b/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/stats/SinkStatsManager.java
index 4f0bbb04707..10460dcc8b0 100644
--- a/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/stats/SinkStatsManager.java
+++ b/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/stats/SinkStatsManager.java
@@ -39,10 +39,10 @@ public class SinkStatsManager extends ComponentStatsManager {
     public static final String RECEIVED_TOTAL = "received_total";
     public static final String WRITTEN_TOTAL = "written_total";
 
-    public static final String SYSTEM_EXCEPTIONS_TOTAL_1min = "system_exceptions_total_1min";
-    public static final String SINK_EXCEPTIONS_TOTAL_1min = "sink_exceptions_total_1min";
-    public static final String RECEIVED_TOTAL_1min = "received_total_1min";
-    public static final String WRITTEN_TOTAL_1min = "written_total_1min";
+    public static final String SYSTEM_EXCEPTIONS_TOTAL_1min = "system_exceptions_1min";
+    public static final String SINK_EXCEPTIONS_TOTAL_1min = "sink_exceptions_1min";
+    public static final String RECEIVED_TOTAL_1min = "received_1min";
+    public static final String WRITTEN_TOTAL_1min = "written_1min";
 
     /** Declare Prometheus stats. **/
 
diff --git a/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/stats/SourceStatsManager.java b/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/stats/SourceStatsManager.java
index e050dcb13dc..ad470aaa2ed 100644
--- a/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/stats/SourceStatsManager.java
+++ b/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/stats/SourceStatsManager.java
@@ -39,10 +39,10 @@ public class SourceStatsManager extends ComponentStatsManager {
     public static final String RECEIVED_TOTAL = "received_total";
     public static final String WRITTEN_TOTAL = "written_total";
 
-    public static final String SYSTEM_EXCEPTIONS_TOTAL_1min = "system_exceptions_total_1min";
-    public static final String SOURCE_EXCEPTIONS_TOTAL_1min = "source_exceptions_total_1min";
-    public static final String RECEIVED_TOTAL_1min = "received_total_1min";
-    public static final String WRITTEN_TOTAL_1min = "written_total_1min";
+    public static final String SYSTEM_EXCEPTIONS_TOTAL_1min = "system_exceptions_1min";
+    public static final String SOURCE_EXCEPTIONS_TOTAL_1min = "source_exceptions_1min";
+    public static final String RECEIVED_TOTAL_1min = "received_1min";
+    public static final String WRITTEN_TOTAL_1min = "written_1min";
 
     /** Declare Prometheus stats. **/
 
diff --git a/pulsar-functions/instance/src/main/python/function_stats.py b/pulsar-functions/instance/src/main/python/function_stats.py
index 8b54f75044d..39246a83c47 100644
--- a/pulsar-functions/instance/src/main/python/function_stats.py
+++ b/pulsar-functions/instance/src/main/python/function_stats.py
@@ -41,11 +41,11 @@ class Stats(object):
   LAST_INVOCATION = 'last_invocation'
   TOTAL_RECEIVED = 'received_total'
 
-  TOTAL_SUCCESSFULLY_PROCESSED_1min = 'processed_successfully_total_1min'
-  TOTAL_SYSTEM_EXCEPTIONS_1min = 'system_exceptions_total_1min'
-  TOTAL_USER_EXCEPTIONS_1min = 'user_exceptions_total_1min'
+  TOTAL_SUCCESSFULLY_PROCESSED_1min = 'processed_successfully_1min_total'
+  TOTAL_SYSTEM_EXCEPTIONS_1min = 'system_exceptions_1min_total'
+  TOTAL_USER_EXCEPTIONS_1min = 'user_exceptions_1min_total'
   PROCESS_LATENCY_MS_1min = 'process_latency_ms_1min'
-  TOTAL_RECEIVED_1min = 'received_total_1min'
+  TOTAL_RECEIVED_1min = 'received_1min_total'
 
   # Declare Prometheus
   stat_total_processed_successfully = Counter(PULSAR_FUNCTION_METRICS_PREFIX + TOTAL_SUCCESSFULLY_PROCESSED,
diff --git a/pulsar-proxy/src/test/java/org/apache/pulsar/proxy/server/ProxyPrometheusMetricsTest.java b/pulsar-proxy/src/test/java/org/apache/pulsar/proxy/server/ProxyPrometheusMetricsTest.java
index 86e0b8727ab..6d7ba4c72df 100644
--- a/pulsar-proxy/src/test/java/org/apache/pulsar/proxy/server/ProxyPrometheusMetricsTest.java
+++ b/pulsar-proxy/src/test/java/org/apache/pulsar/proxy/server/ProxyPrometheusMetricsTest.java
@@ -107,7 +107,7 @@ public class ProxyPrometheusMetricsTest extends MockedPulsarServiceBaseTest {
         Multimap<String, Metric> metrics = parseMetrics(response);
 
         // Check that ProxyService metrics are present
-        List<Metric> cm = (List<Metric>) metrics.get("pulsar_proxy_binary_bytes");
+        List<Metric> cm = (List<Metric>) metrics.get("pulsar_proxy_binary_bytes_total");
         assertEquals(cm.size(), 1);
         assertEquals(cm.get(0).tags.get("cluster"), TEST_CLUSTER);
 
@@ -117,7 +117,7 @@ public class ProxyPrometheusMetricsTest extends MockedPulsarServiceBaseTest {
         assertEquals(cm2.get(0).tags.get("label1"), "xyz");
 
         // Check that PrometheusRawMetricsProvider metrics are present
-        List<Metric> cm3 = (List<Metric>) metrics.get("test_counter");
+        List<Metric> cm3 = (List<Metric>) metrics.get("test_counter_total");
         assertEquals(cm3.size(), 1);
         assertEquals(cm3.get(0).tags.get("cluster"), TEST_CLUSTER);
     }
diff --git a/pulsar-sql/presto-distribution/LICENSE b/pulsar-sql/presto-distribution/LICENSE
index cd19bb308c7..32880122173 100644
--- a/pulsar-sql/presto-distribution/LICENSE
+++ b/pulsar-sql/presto-distribution/LICENSE
@@ -419,10 +419,14 @@ The Apache Software License, Version 2.0
     - metrics-graphite-4.1.12.1.jar
     - metrics-jvm-4.1.12.1.jar
   * Prometheus
-    - simpleclient-0.5.0.jar
-    - simpleclient_common-0.5.0.jar
-    - simpleclient_hotspot-0.5.0.jar
-    - simpleclient_servlet-0.5.0.jar
+    - simpleclient-0.15.0.jar
+    - simpleclient_common-0.15.0.jar
+    - simpleclient_hotspot-0.15.0.jar
+    - simpleclient_servlet-0.15.0.jar
+    - simpleclient_servlet_common-0.15.0.jar
+    - simpleclient_tracer_common-0.15.0.jar
+    - simpleclient_tracer_otel-0.15.0.jar
+    - simpleclient_tracer_otel_agent-0.15.0.jar
   * JCTools
     - jctools-core-2.1.2.jar
   * Asynchronous Http Client
diff --git a/site2/docs/reference-metrics.md b/site2/docs/reference-metrics.md
index 3ee6aa0e52b..b7493367c7d 100644
--- a/site2/docs/reference-metrics.md
+++ b/site2/docs/reference-metrics.md
@@ -479,7 +479,7 @@ All the token metrics are labelled with the following labels:
 
 | Name | Type | Description |
 |---|---|---|
-| pulsar_expired_token_count | Counter | The number of expired tokens in Pulsar. |
+| pulsar_expired_token_total | Counter | The number of expired tokens in Pulsar. |
 | pulsar_expiring_token_minutes | Histogram | The remaining time of expiring tokens in minutes. |
 
 ### Authentication metrics
@@ -489,12 +489,12 @@ All the authentication metrics are labelled with the following labels:
 - *cluster*: `cluster=${pulsar_cluster}`. `${pulsar_cluster}` is the cluster name that you have configured in the `broker.conf` file.
 - *provider_name*: `provider_name=${provider_name}`. `${provider_name}` is the class name of the authentication provider.
 - *auth_method*: `auth_method=${auth_method}`. `${auth_method}` is the authentication method of the authentication provider.
-- *reason*: `reason=${reason}`. `${reason}` is the reason for failing authentication operation. (This label is only for `pulsar_authentication_failures_count`.)
+- *reason*: `reason=${reason}`. `${reason}` is the reason for failing authentication operation. (This label is only for `pulsar_authentication_failures_total`.)
 
 | Name | Type | Description |
 |---|---|---|
-| pulsar_authentication_success_count| Counter | The number of successful authentication operations. |
-| pulsar_authentication_failures_count | Counter | The number of failing authentication operations. |
+| pulsar_authentication_success_total| Counter | The number of successful authentication operations. |
+| pulsar_authentication_failures_total | Counter | The number of failing authentication operations. |
 
 ### Jetty metrics
 
@@ -591,16 +591,16 @@ All the Pulsar Functions metrics are labelled with the following labels:
 | Name | Type | Description |
 |---|---|---|
 | pulsar_function_processed_successfully_total | Counter | The total number of messages processed successfully. |
-| pulsar_function_processed_successfully_total_1min | Counter | The total number of messages processed successfully in the last 1 minute. |
+| pulsar_function_processed_successfully_1min_total | Counter | The total number of messages processed successfully in the last 1 minute. |
 | pulsar_function_system_exceptions_total | Counter | The total number of system exceptions. |
-| pulsar_function_system_exceptions_total_1min | Counter | The total number of system exceptions in the last 1 minute. |
+| pulsar_function_system_exceptions_1min_total | Counter | The total number of system exceptions in the last 1 minute. |
 | pulsar_function_user_exceptions_total | Counter | The total number of user exceptions. |
-| pulsar_function_user_exceptions_total_1min | Counter | The total number of user exceptions in the last 1 minute. |
+| pulsar_function_user_exceptions_1min_total | Counter | The total number of user exceptions in the last 1 minute. |
 | pulsar_function_process_latency_ms | Summary | The process latency in milliseconds. |
 | pulsar_function_process_latency_ms_1min | Summary | The process latency in milliseconds in the last 1 minute. |
 | pulsar_function_last_invocation | Gauge | The timestamp of the last invocation of the function. |
 | pulsar_function_received_total | Counter | The total number of messages received from source. |
-| pulsar_function_received_total_1min | Counter | The total number of messages received from source in the last 1 minute. |
+| pulsar_function_received_1min_total | Counter | The total number of messages received from source in the last 1 minute. |
 pulsar_function_user_metric_ | Summary|The user-defined metrics.
 
 ## Connectors
@@ -617,16 +617,16 @@ Connector metrics contain **source** metrics and **sink** metrics.
   | Name | Type | Description |
   |---|---|---|
   pulsar_source_written_total|Counter|The total number of records written to a Pulsar topic.
-  pulsar_source_written_total_1min|Counter|The total number of records written to a Pulsar topic in the last 1 minute.
+  pulsar_source_written_1min_total|Counter|The total number of records written to a Pulsar topic in the last 1 minute.
   pulsar_source_received_total|Counter|The total number of records received from source.
-  pulsar_source_received_total_1min|Counter|The total number of records received from source in the last 1 minute.
+  pulsar_source_received_1min_total|Counter|The total number of records received from source in the last 1 minute.
   pulsar_source_last_invocation|Gauge|The timestamp of the last invocation of the source.
   pulsar_source_source_exception|Gauge|The exception from a source.
   pulsar_source_source_exceptions_total|Counter|The total number of source exceptions.
-  pulsar_source_source_exceptions_total_1min |Counter|The total number of source exceptions in the last 1 minute.
+  pulsar_source_source_exceptions_1min_total |Counter|The total number of source exceptions in the last 1 minute.
   pulsar_source_system_exception|Gauge|The exception from system code.
   pulsar_source_system_exceptions_total|Counter|The total number of system exceptions.
-  pulsar_source_system_exceptions_total_1min|Counter|The total number of system exceptions in the last 1 minute.
+  pulsar_source_system_exceptions_1min_total|Counter|The total number of system exceptions in the last 1 minute.
   pulsar_source_user_metric_ | Summary|The user-defined metrics.
 
 - **Sink** metrics
@@ -634,16 +634,16 @@ Connector metrics contain **source** metrics and **sink** metrics.
   | Name | Type | Description |
   |---|---|---|
   pulsar_sink_written_total|Counter| The total number of records processed by a sink.
-  pulsar_sink_written_total_1min|Counter| The total number of records processed by a sink in the last 1 minute.
-  pulsar_sink_received_total_1min|Counter| The total number of messages that a sink has received from Pulsar topics in the last 1 minute.
+  pulsar_sink_written_1min_total|Counter| The total number of records processed by a sink in the last 1 minute.
+  pulsar_sink_received_1min_total|Counter| The total number of messages that a sink has received from Pulsar topics in the last 1 minute.
   pulsar_sink_received_total|Counter| The total number of records that a sink has received from Pulsar topics.
   pulsar_sink_last_invocation|Gauge|The timestamp of the last invocation of the sink.
   pulsar_sink_sink_exception|Gauge|The exception from a sink.
   pulsar_sink_sink_exceptions_total|Counter|The total number of sink exceptions.
-  pulsar_sink_sink_exceptions_total_1min |Counter|The total number of sink exceptions in the last 1 minute.
+  pulsar_sink_sink_exceptions_1min_total |Counter|The total number of sink exceptions in the last 1 minute.
   pulsar_sink_system_exception|Gauge|The exception from system code.
   pulsar_sink_system_exceptions_total|Counter|The total number of system exceptions.
-  pulsar_sink_system_exceptions_total_1min|Counter|The total number of system exceptions in the last 1 minute.
+  pulsar_sink_system_exceptions_1min_total|Counter|The total number of system exceptions in the last 1 minute.
   pulsar_sink_user_metric_ | Summary|The user-defined metrics.
 
 ## Proxy