You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "apoorvmittal10 (via GitHub)" <gi...@apache.org> on 2023/10/23 17:00:23 UTC
[PR] KAFKA-15669: Implement telemetry metric naming strategy (KIP-714) [kafka]
apoorvmittal10 opened a new pull request, #14619:
URL: https://github.com/apache/kafka/pull/14619
The PR defined the naming convention for telemetry metric names for KIP-714. Telemetry metric name should be dot separated and tags should be snake case.
PR adds the interface which will be used in MetricsReporter implementation to construct metric names.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15669: Implement telemetry metric naming strategy (KIP-714) [kafka]
Posted by "wcarlson5 (via GitHub)" <gi...@apache.org>.
wcarlson5 merged PR #14619:
URL: https://github.com/apache/kafka/pull/14619
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15669: Implement telemetry metric naming strategy (KIP-714) [kafka]
Posted by "wcarlson5 (via GitHub)" <gi...@apache.org>.
wcarlson5 commented on PR #14619:
URL: https://github.com/apache/kafka/pull/14619#issuecomment-1787988965
The tests failing are not related to this PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15669: Implement telemetry metric naming strategy (KIP-714) [kafka]
Posted by "apoorvmittal10 (via GitHub)" <gi...@apache.org>.
apoorvmittal10 commented on code in PR #14619:
URL: https://github.com/apache/kafka/pull/14619#discussion_r1375100330
##########
clients/src/main/java/org/apache/kafka/common/telemetry/internals/MetricNamingConvention.java:
##########
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.telemetry.internals;
+
+import org.apache.kafka.common.MetricName;
+
+import java.util.Collections;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+/**
+ * This class encapsulates naming and mapping conventions defined as part of
+ * <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-714%3A+Client+metrics+and+observability#KIP714:Clientmetricsandobservability-Metricsnamingandformat">Metrics naming and format</a>
+ */
+public class MetricNamingConvention {
+
+ private static final String NAME_JOINER = ".";
+ private static final String TAG_JOINER = "_";
+
+ // remove metrics as it is redundant for telemetry metrics naming convention
+ private final static Pattern GROUP_PATTERN = Pattern.compile("\\.(metrics)");
+
+ public static MetricNamingStrategy<MetricName> getClientTelemetryMetricNamingStrategy(String domain) {
Review Comment:
Renamed `domain` -> `prefix`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15669: Implement telemetry metric naming strategy (KIP-714) [kafka]
Posted by "AndrewJSchofield (via GitHub)" <gi...@apache.org>.
AndrewJSchofield commented on code in PR #14619:
URL: https://github.com/apache/kafka/pull/14619#discussion_r1369227989
##########
clients/src/test/java/org/apache/kafka/common/telemetry/internals/MetricNamingConventionTest.java:
##########
@@ -0,0 +1,196 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.telemetry.internals;
+
+import org.apache.kafka.common.MetricName;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Collections;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+
+public class MetricNamingConventionTest {
+
+ private MetricNamingStrategy<MetricName> metricNamingStrategy;
+
+ @BeforeEach
+ public void setUp() {
+ metricNamingStrategy = MetricNamingConvention
+ .getClientTelemetryMetricNamingStrategy("org.apache.kafka");
+ }
+
+ @Test
+ public void testMetricKey() {
+ MetricName metricName = new MetricName("name", "group", "description",
+ Collections.emptyMap());
+ MetricKey metricKey = metricNamingStrategy.metricKey(metricName);
+
+ assertEquals("org.apache.kafka.group.name", metricKey.getName());
+ assertEquals(Collections.emptyMap(), metricKey.tags());
+ }
+
+ @Test
+ public void testMetricKeyWithHyphenNameAndTags() {
+ Map<String, String> tags = new HashMap<>();
+ tags.put("tag1", "value1");
+ tags.put("tag2", "value2");
+
+ MetricName metricName = new MetricName("test-name", "group-name", "description", tags);
+ MetricKey metricKey = metricNamingStrategy.metricKey(metricName);
+
+ assertEquals("org.apache.kafka.group.name.test.name", metricKey.getName());
+ assertEquals(tags, metricKey.tags());
+ }
+
+ /**
+ * Test metric key with mixed name and mixed tags where mixed represents combination of upper case,
+ * lower case, numbers, hyphen, dot, underscore and special characters.
+ */
+ @Test
+ public void testMetricKeyWithMixedNameAndMixedTags() {
+ Map<String, String> tags = new HashMap<>();
+ tags.put("tag1-Ab-2_(", "value1");
+ tags.put("tag2-HELLO.@", "value2");
+
+ MetricName metricName = new MetricName("test-Name-1.$", "grouP-name-AB_&", "description", tags);
+ MetricKey metricKey = metricNamingStrategy.metricKey(metricName);
+
+ tags.clear();
+ tags.put("tag1_ab_2_(", "value1");
+ tags.put("tag2_hello.@", "value2");
+
+ assertEquals("org.apache.kafka.group.name.ab_&.test.name.1.$", metricKey.getName());
+ assertEquals(tags, metricKey.tags());
+ }
+
+ @Test
+ public void testMetricKeyWithNullMetricName() {
+ Exception e = assertThrows(NullPointerException.class, () -> metricNamingStrategy.metricKey(null));
+ assertEquals("metric name cannot be null", e.getMessage());
+ }
+
+ @Test
+ public void testMetricKeyWithEmptyName() {
+ MetricName metricName = new MetricName("", "group-1A", "description",
+ Collections.emptyMap());
+ MetricKey metricKey = metricNamingStrategy.metricKey(metricName);
+
+ // If there is no name, then the telemetry metric name will have dot in the end though
+ // metric names always have a name.
+ assertEquals("org.apache.kafka.group.1a.", metricKey.getName());
+ assertEquals(Collections.emptyMap(), metricKey.tags());
+ }
+
+ @Test
+ public void testMetricKeyWithEmptyGroup() {
+ MetricName metricName = new MetricName("name", "", "description",
+ Collections.emptyMap());
+ MetricKey metricKey = metricNamingStrategy.metricKey(metricName);
+
+ // If there is no group, then the telemetry metric name will have consecutive dots, though
+ // metric names always have group name.
+ assertEquals("org.apache.kafka..name", metricKey.getName());
+ assertEquals(Collections.emptyMap(), metricKey.tags());
+ }
+
+ @Test
+ public void testMetricKeyWithAdditionalMetricsSuffixInGroup() {
+ MetricName metricName = new MetricName("name", "group-metrics", "description",
+ Collections.emptyMap());
+ MetricKey metricKey = metricNamingStrategy.metricKey(metricName);
+
+ // '-metrics' gets removed from the group name.
+ assertEquals("org.apache.kafka.group.name", metricKey.getName());
+ assertEquals(Collections.emptyMap(), metricKey.tags());
+ }
+
+ @Test
+ public void testMetricKeyWithMultipleMetricsSuffixInGroup() {
+ MetricName metricName = new MetricName("name-metrics", "group-metrics-metrics", "description",
+ Collections.emptyMap());
+ MetricKey metricKey = metricNamingStrategy.metricKey(metricName);
+
+ // '-metrics' gets removed from the group name.
+ assertEquals("org.apache.kafka.group.name.metrics", metricKey.getName());
+ assertEquals(Collections.emptyMap(), metricKey.tags());
+ }
+
+ @Test
+ public void testMetricKeyWithNullTagKey() {
+ MetricName metricName = new MetricName("name", "group", "description",
+ Collections.singletonMap(null, "value1"));
+ Exception e = assertThrows(NullPointerException.class, () -> metricNamingStrategy.metricKey(metricName));
+ assertEquals("metric data cannot be null", e.getMessage());
+ }
+
+ @Test
+ public void testMetricKeyWithBlankTagKey() {
+ MetricName metricName = new MetricName("name", "group", "description",
+ Collections.singletonMap("", "value1"));
+ MetricKey metricKey = metricNamingStrategy.metricKey(metricName);
+
+ assertEquals("org.apache.kafka.group.name", metricKey.getName());
+ assertEquals(Collections.singletonMap("", "value1"), metricKey.tags());
+ }
+
+ @Test
+ public void testDerivedMetricKey() {
+ MetricName metricName = new MetricName("name", "group", "description",
+ Collections.emptyMap());
+ MetricKey metricKey = metricNamingStrategy.derivedMetricKey(
+ metricNamingStrategy.metricKey(metricName), "delta");
+
+ assertEquals("org.apache.kafka.group.name.delta", metricKey.getName());
+ assertEquals(Collections.emptyMap(), metricKey.tags());
+ }
+
+ @Test
+ public void testDerivedMetricKeyWithTags() {
+ MetricName metricName = new MetricName("name", "group", "description",
+ Collections.singletonMap("tag1", "value1"));
+ MetricKey metricKey = metricNamingStrategy.derivedMetricKey(
+ metricNamingStrategy.metricKey(metricName), "delta");
+
+ assertEquals("org.apache.kafka.group.name.delta", metricKey.getName());
+ assertEquals(Collections.singletonMap("tag1", "value1"), metricKey.tags());
+ }
+
+ @Test
+ public void testDerivedMetricKeyWithNullComponent() {
+ MetricName metricName = new MetricName("name", "group", "description",
+ Collections.emptyMap());
+ Exception e = assertThrows(NullPointerException.class, () -> metricNamingStrategy.derivedMetricKey(
+ metricNamingStrategy.metricKey(metricName), null));
+ assertEquals("derived component cannot be null", e.getMessage());
+ }
+
+ @Test
+ public void testDerivedMetricKeyWithBlankComponent() {
+ MetricName metricName = new MetricName("name", "group", "description",
+ Collections.emptyMap());
+ MetricKey metricKey = metricNamingStrategy.derivedMetricKey(
+ metricNamingStrategy.metricKey(metricName), "");
+
+ // Ends with dot, though derived component should
Review Comment:
Looks like half a sentence in this comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15669: Implement telemetry metric naming strategy (KIP-714) [kafka]
Posted by "apoorvmittal10 (via GitHub)" <gi...@apache.org>.
apoorvmittal10 commented on code in PR #14619:
URL: https://github.com/apache/kafka/pull/14619#discussion_r1375099944
##########
clients/src/main/java/org/apache/kafka/common/telemetry/internals/MetricNamingConvention.java:
##########
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.telemetry.internals;
+
+import org.apache.kafka.common.MetricName;
+
+import java.util.Collections;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+/**
+ * This class encapsulates naming and mapping conventions defined as part of
+ * <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-714%3A+Client+metrics+and+observability#KIP714:Clientmetricsandobservability-Metricsnamingandformat">Metrics naming and format</a>
+ */
+public class MetricNamingConvention {
+
+ private static final String NAME_JOINER = ".";
+ private static final String TAG_JOINER = "_";
+
+ // remove metrics as it is redundant for telemetry metrics naming convention
+ private final static Pattern GROUP_PATTERN = Pattern.compile("\\.(metrics)");
+
+ public static MetricNamingStrategy<MetricName> getClientTelemetryMetricNamingStrategy(String domain) {
+ return new MetricNamingStrategy<MetricName>() {
+ @Override
+ public MetricKey metricKey(MetricName metricName) {
+ Objects.requireNonNull(metricName, "metric name cannot be null");
+ String group = metricName.group() == null ? "" : metricName.group();
+ String rawName = metricName.name() == null ? "" : metricName.name();
+
+ return new MetricKey(fullMetricName(domain, group, rawName),
+ Collections.unmodifiableMap(cleanTags(metricName.tags())));
+ }
+
+ @Override
+ public MetricKey derivedMetricKey(MetricKey key, String derivedComponent) {
+ Objects.requireNonNull(derivedComponent, "derived component cannot be null");
+ return new MetricKey(key.getName() + NAME_JOINER + derivedComponent, key.tags());
+ }
+ };
+ }
+
+ /**
+ * Creates a metric name given the domain, group, and name. The new String follows the following
+ * conventions and rules:
+ *
+ * <ul>
+ * <li>domain is expected to be a host-name like value, e.g. {@code org.apache.kafka}</li>
+ * <li>group is cleaned of redundant words: "-metrics"</li>
+ * <li>the group and metric name is dot separated</li>
+ * <li>The name is created by joining the three components, e.g.:
+ * {@code org.apache.kafka.producer.connection.creation.rate}</li>
+ * </ul>
+ */
+ private static String fullMetricName(String domain, String group, String name) {
+ return domain
+ + NAME_JOINER
+ + cleanGroup(group)
+ + NAME_JOINER
+ + cleanMetric(name);
+ }
+
+ /**
+ * This method maps a raw name to follow conventions and cleans up the result to be more legible:
+ * <ul>
+ * <li> converts names to lower hyphen case conventions
Review Comment:
Corrected the comment, my bad.
##########
clients/src/test/java/org/apache/kafka/common/telemetry/internals/MetricNamingConventionTest.java:
##########
@@ -0,0 +1,196 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.telemetry.internals;
+
+import org.apache.kafka.common.MetricName;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Collections;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+
+public class MetricNamingConventionTest {
+
+ private MetricNamingStrategy<MetricName> metricNamingStrategy;
+
+ @BeforeEach
+ public void setUp() {
+ metricNamingStrategy = MetricNamingConvention
+ .getClientTelemetryMetricNamingStrategy("org.apache.kafka");
+ }
+
+ @Test
+ public void testMetricKey() {
+ MetricName metricName = new MetricName("name", "group", "description",
+ Collections.emptyMap());
+ MetricKey metricKey = metricNamingStrategy.metricKey(metricName);
+
+ assertEquals("org.apache.kafka.group.name", metricKey.getName());
+ assertEquals(Collections.emptyMap(), metricKey.tags());
+ }
+
+ @Test
+ public void testMetricKeyWithHyphenNameAndTags() {
+ Map<String, String> tags = new HashMap<>();
Review Comment:
Corrected test name.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15669: Implement telemetry metric naming strategy (KIP-714) [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on code in PR #14619:
URL: https://github.com/apache/kafka/pull/14619#discussion_r1374019186
##########
clients/src/main/java/org/apache/kafka/common/telemetry/internals/MetricNamingConvention.java:
##########
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.telemetry.internals;
+
+import org.apache.kafka.common.MetricName;
+
+import java.util.Collections;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+/**
+ * This class encapsulates naming and mapping conventions defined as part of
+ * <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-714%3A+Client+metrics+and+observability#KIP714:Clientmetricsandobservability-Metricsnamingandformat">Metrics naming and format</a>
+ */
+public class MetricNamingConvention {
+
+ private static final String NAME_JOINER = ".";
+ private static final String TAG_JOINER = "_";
+
+ // remove metrics as it is redundant for telemetry metrics naming convention
+ private final static Pattern GROUP_PATTERN = Pattern.compile("\\.(metrics)");
+
+ public static MetricNamingStrategy<MetricName> getClientTelemetryMetricNamingStrategy(String domain) {
+ return new MetricNamingStrategy<MetricName>() {
+ @Override
+ public MetricKey metricKey(MetricName metricName) {
+ Objects.requireNonNull(metricName, "metric name cannot be null");
+ String group = metricName.group() == null ? "" : metricName.group();
+ String rawName = metricName.name() == null ? "" : metricName.name();
+
+ return new MetricKey(fullMetricName(domain, group, rawName),
+ Collections.unmodifiableMap(cleanTags(metricName.tags())));
+ }
+
+ @Override
+ public MetricKey derivedMetricKey(MetricKey key, String derivedComponent) {
+ Objects.requireNonNull(derivedComponent, "derived component cannot be null");
+ return new MetricKey(key.getName() + NAME_JOINER + derivedComponent, key.tags());
+ }
+ };
+ }
+
+ /**
+ * Creates a metric name given the domain, group, and name. The new String follows the following
+ * conventions and rules:
+ *
+ * <ul>
+ * <li>domain is expected to be a host-name like value, e.g. {@code org.apache.kafka}</li>
+ * <li>group is cleaned of redundant words: "-metrics"</li>
+ * <li>the group and metric name is dot separated</li>
+ * <li>The name is created by joining the three components, e.g.:
+ * {@code org.apache.kafka.producer.connection.creation.rate}</li>
+ * </ul>
+ */
+ private static String fullMetricName(String domain, String group, String name) {
+ return domain
+ + NAME_JOINER
+ + cleanGroup(group)
+ + NAME_JOINER
+ + cleanMetric(name);
+ }
+
+ /**
+ * This method maps a raw name to follow conventions and cleans up the result to be more legible:
+ * <ul>
+ * <li> converts names to lower hyphen case conventions
+ * <li> strips redundant parts of the metric name, such as -metrics
+ * <li> normalizes artifacts of hyphen case to dot separated conversion
+ * </ul>
+ */
+ private static String cleanGroup(String group) {
+ group = clean(group, NAME_JOINER);
+ return GROUP_PATTERN.matcher(group).replaceAll("");
+ }
+
+ private static String cleanMetric(String metric) {
+ return clean(metric, NAME_JOINER);
+ }
+
+ /**
+ * Converts a tag name to match the telemetry naming conventions by converting snake_case.
+ * <p>
+ * Kafka metrics have tags name in lower case separated by hyphens. Eg: total-errors
+ *
+ * @param raw the input map
+ * @return the new map with keys replaced by snake_case representations.
+ */
+ private static Map<String, String> cleanTags(Map<String, String> raw) {
+ return raw.entrySet()
+ .stream()
+ .collect(Collectors.toMap(s -> clean(s.getKey(), TAG_JOINER), Entry::getValue));
+ }
+
+ private static String clean(String raw, String joiner) {
+ Objects.requireNonNull(raw, "metric data cannot be null");
+ // Convert to lower case.
Review Comment:
This comment is kinda point less -- it just says what the code say `.toLowerCase`
##########
clients/src/main/java/org/apache/kafka/common/telemetry/internals/MetricNamingConvention.java:
##########
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.telemetry.internals;
+
+import org.apache.kafka.common.MetricName;
+
+import java.util.Collections;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+/**
+ * This class encapsulates naming and mapping conventions defined as part of
+ * <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-714%3A+Client+metrics+and+observability#KIP714:Clientmetricsandobservability-Metricsnamingandformat">Metrics naming and format</a>
+ */
+public class MetricNamingConvention {
+
+ private static final String NAME_JOINER = ".";
+ private static final String TAG_JOINER = "_";
+
+ // remove metrics as it is redundant for telemetry metrics naming convention
+ private final static Pattern GROUP_PATTERN = Pattern.compile("\\.(metrics)");
+
+ public static MetricNamingStrategy<MetricName> getClientTelemetryMetricNamingStrategy(String domain) {
+ return new MetricNamingStrategy<MetricName>() {
+ @Override
+ public MetricKey metricKey(MetricName metricName) {
+ Objects.requireNonNull(metricName, "metric name cannot be null");
+ String group = metricName.group() == null ? "" : metricName.group();
+ String rawName = metricName.name() == null ? "" : metricName.name();
+
+ return new MetricKey(fullMetricName(domain, group, rawName),
+ Collections.unmodifiableMap(cleanTags(metricName.tags())));
+ }
+
+ @Override
+ public MetricKey derivedMetricKey(MetricKey key, String derivedComponent) {
+ Objects.requireNonNull(derivedComponent, "derived component cannot be null");
+ return new MetricKey(key.getName() + NAME_JOINER + derivedComponent, key.tags());
+ }
+ };
+ }
+
+ /**
+ * Creates a metric name given the domain, group, and name. The new String follows the following
+ * conventions and rules:
+ *
+ * <ul>
+ * <li>domain is expected to be a host-name like value, e.g. {@code org.apache.kafka}</li>
+ * <li>group is cleaned of redundant words: "-metrics"</li>
+ * <li>the group and metric name is dot separated</li>
+ * <li>The name is created by joining the three components, e.g.:
+ * {@code org.apache.kafka.producer.connection.creation.rate}</li>
+ * </ul>
+ */
+ private static String fullMetricName(String domain, String group, String name) {
+ return domain
+ + NAME_JOINER
+ + cleanGroup(group)
+ + NAME_JOINER
+ + cleanMetric(name);
+ }
+
+ /**
+ * This method maps a raw name to follow conventions and cleans up the result to be more legible:
+ * <ul>
+ * <li> converts names to lower hyphen case conventions
Review Comment:
I don't see any conversion into lower hyphen in this method?
##########
clients/src/test/java/org/apache/kafka/common/telemetry/internals/MetricNamingConventionTest.java:
##########
@@ -0,0 +1,196 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.telemetry.internals;
+
+import org.apache.kafka.common.MetricName;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Collections;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+
+public class MetricNamingConventionTest {
+
+ private MetricNamingStrategy<MetricName> metricNamingStrategy;
+
+ @BeforeEach
+ public void setUp() {
+ metricNamingStrategy = MetricNamingConvention
+ .getClientTelemetryMetricNamingStrategy("org.apache.kafka");
+ }
+
+ @Test
+ public void testMetricKey() {
+ MetricName metricName = new MetricName("name", "group", "description",
+ Collections.emptyMap());
+ MetricKey metricKey = metricNamingStrategy.metricKey(metricName);
+
+ assertEquals("org.apache.kafka.group.name", metricKey.getName());
+ assertEquals(Collections.emptyMap(), metricKey.tags());
+ }
+
+ @Test
+ public void testMetricKeyWithHyphenNameAndTags() {
+ Map<String, String> tags = new HashMap<>();
Review Comment:
Based on the test name, we should use `emptyMap` for tags in this test.
##########
clients/src/main/java/org/apache/kafka/common/telemetry/internals/MetricNamingConvention.java:
##########
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.telemetry.internals;
+
+import org.apache.kafka.common.MetricName;
+
+import java.util.Collections;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+/**
+ * This class encapsulates naming and mapping conventions defined as part of
+ * <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-714%3A+Client+metrics+and+observability#KIP714:Clientmetricsandobservability-Metricsnamingandformat">Metrics naming and format</a>
+ */
+public class MetricNamingConvention {
+
+ private static final String NAME_JOINER = ".";
+ private static final String TAG_JOINER = "_";
+
+ // remove metrics as it is redundant for telemetry metrics naming convention
+ private final static Pattern GROUP_PATTERN = Pattern.compile("\\.(metrics)");
+
+ public static MetricNamingStrategy<MetricName> getClientTelemetryMetricNamingStrategy(String domain) {
Review Comment:
If this is an internal class, do we expect to pass in anything else but `org.apache.kafka`?
##########
clients/src/main/java/org/apache/kafka/common/telemetry/internals/MetricNamingConvention.java:
##########
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.telemetry.internals;
+
+import org.apache.kafka.common.MetricName;
+
+import java.util.Collections;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+/**
+ * This class encapsulates naming and mapping conventions defined as part of
+ * <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-714%3A+Client+metrics+and+observability#KIP714:Clientmetricsandobservability-Metricsnamingandformat">Metrics naming and format</a>
+ */
+public class MetricNamingConvention {
+
+ private static final String NAME_JOINER = ".";
+ private static final String TAG_JOINER = "_";
+
+ // remove metrics as it is redundant for telemetry metrics naming convention
+ private final static Pattern GROUP_PATTERN = Pattern.compile("\\.(metrics)");
+
+ public static MetricNamingStrategy<MetricName> getClientTelemetryMetricNamingStrategy(String domain) {
+ return new MetricNamingStrategy<MetricName>() {
+ @Override
+ public MetricKey metricKey(MetricName metricName) {
+ Objects.requireNonNull(metricName, "metric name cannot be null");
+ String group = metricName.group() == null ? "" : metricName.group();
+ String rawName = metricName.name() == null ? "" : metricName.name();
+
+ return new MetricKey(fullMetricName(domain, group, rawName),
+ Collections.unmodifiableMap(cleanTags(metricName.tags())));
+ }
+
+ @Override
+ public MetricKey derivedMetricKey(MetricKey key, String derivedComponent) {
+ Objects.requireNonNull(derivedComponent, "derived component cannot be null");
+ return new MetricKey(key.getName() + NAME_JOINER + derivedComponent, key.tags());
+ }
+ };
+ }
+
+ /**
+ * Creates a metric name given the domain, group, and name. The new String follows the following
+ * conventions and rules:
+ *
+ * <ul>
+ * <li>domain is expected to be a host-name like value, e.g. {@code org.apache.kafka}</li>
+ * <li>group is cleaned of redundant words: "-metrics"</li>
+ * <li>the group and metric name is dot separated</li>
+ * <li>The name is created by joining the three components, e.g.:
+ * {@code org.apache.kafka.producer.connection.creation.rate}</li>
+ * </ul>
+ */
+ private static String fullMetricName(String domain, String group, String name) {
+ return domain
+ + NAME_JOINER
+ + cleanGroup(group)
+ + NAME_JOINER
+ + cleanMetric(name);
+ }
+
+ /**
+ * This method maps a raw name to follow conventions and cleans up the result to be more legible:
+ * <ul>
+ * <li> converts names to lower hyphen case conventions
+ * <li> strips redundant parts of the metric name, such as -metrics
+ * <li> normalizes artifacts of hyphen case to dot separated conversion
+ * </ul>
+ */
+ private static String cleanGroup(String group) {
+ group = clean(group, NAME_JOINER);
+ return GROUP_PATTERN.matcher(group).replaceAll("");
+ }
+
+ private static String cleanMetric(String metric) {
+ return clean(metric, NAME_JOINER);
+ }
+
+ /**
+ * Converts a tag name to match the telemetry naming conventions by converting snake_case.
+ * <p>
+ * Kafka metrics have tags name in lower case separated by hyphens. Eg: total-errors
+ *
+ * @param raw the input map
+ * @return the new map with keys replaced by snake_case representations.
+ */
+ private static Map<String, String> cleanTags(Map<String, String> raw) {
+ return raw.entrySet()
+ .stream()
+ .collect(Collectors.toMap(s -> clean(s.getKey(), TAG_JOINER), Entry::getValue));
+ }
+
+ private static String clean(String raw, String joiner) {
+ Objects.requireNonNull(raw, "metric data cannot be null");
+ // Convert to lower case.
+ String lowerCase = raw.toLowerCase(Locale.ROOT);
+ // Replace hyphens with telemetry name joiner i.e. dot.
Review Comment:
Same
##########
clients/src/main/java/org/apache/kafka/common/telemetry/internals/MetricNamingConvention.java:
##########
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.telemetry.internals;
+
+import org.apache.kafka.common.MetricName;
+
+import java.util.Collections;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+/**
+ * This class encapsulates naming and mapping conventions defined as part of
+ * <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-714%3A+Client+metrics+and+observability#KIP714:Clientmetricsandobservability-Metricsnamingandformat">Metrics naming and format</a>
+ */
+public class MetricNamingConvention {
+
+ private static final String NAME_JOINER = ".";
+ private static final String TAG_JOINER = "_";
+
+ // remove metrics as it is redundant for telemetry metrics naming convention
+ private final static Pattern GROUP_PATTERN = Pattern.compile("\\.(metrics)");
+
+ public static MetricNamingStrategy<MetricName> getClientTelemetryMetricNamingStrategy(String domain) {
+ return new MetricNamingStrategy<MetricName>() {
+ @Override
+ public MetricKey metricKey(MetricName metricName) {
+ Objects.requireNonNull(metricName, "metric name cannot be null");
+ String group = metricName.group() == null ? "" : metricName.group();
Review Comment:
It seems that `MetricName` guarantees that `group` is no null -- this check seems to be unnecessary. (same for `name` below)
##########
clients/src/main/java/org/apache/kafka/common/telemetry/internals/MetricNamingConvention.java:
##########
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.telemetry.internals;
+
+import org.apache.kafka.common.MetricName;
+
+import java.util.Collections;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+/**
+ * This class encapsulates naming and mapping conventions defined as part of
+ * <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-714%3A+Client+metrics+and+observability#KIP714:Clientmetricsandobservability-Metricsnamingandformat">Metrics naming and format</a>
+ */
+public class MetricNamingConvention {
+
+ private static final String NAME_JOINER = ".";
+ private static final String TAG_JOINER = "_";
+
+ // remove metrics as it is redundant for telemetry metrics naming convention
+ private final static Pattern GROUP_PATTERN = Pattern.compile("\\.(metrics)");
+
+ public static MetricNamingStrategy<MetricName> getClientTelemetryMetricNamingStrategy(String domain) {
+ return new MetricNamingStrategy<MetricName>() {
Review Comment:
Should we add a null-check for `domain`?
##########
clients/src/main/java/org/apache/kafka/common/telemetry/internals/MetricNamingConvention.java:
##########
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.telemetry.internals;
+
+import org.apache.kafka.common.MetricName;
+
+import java.util.Collections;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+/**
+ * This class encapsulates naming and mapping conventions defined as part of
+ * <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-714%3A+Client+metrics+and+observability#KIP714:Clientmetricsandobservability-Metricsnamingandformat">Metrics naming and format</a>
+ */
+public class MetricNamingConvention {
+
+ private static final String NAME_JOINER = ".";
+ private static final String TAG_JOINER = "_";
+
+ // remove metrics as it is redundant for telemetry metrics naming convention
+ private final static Pattern GROUP_PATTERN = Pattern.compile("\\.(metrics)");
+
+ public static MetricNamingStrategy<MetricName> getClientTelemetryMetricNamingStrategy(String domain) {
+ return new MetricNamingStrategy<MetricName>() {
+ @Override
+ public MetricKey metricKey(MetricName metricName) {
+ Objects.requireNonNull(metricName, "metric name cannot be null");
+ String group = metricName.group() == null ? "" : metricName.group();
+ String rawName = metricName.name() == null ? "" : metricName.name();
+
+ return new MetricKey(fullMetricName(domain, group, rawName),
+ Collections.unmodifiableMap(cleanTags(metricName.tags())));
+ }
+
+ @Override
+ public MetricKey derivedMetricKey(MetricKey key, String derivedComponent) {
+ Objects.requireNonNull(derivedComponent, "derived component cannot be null");
+ return new MetricKey(key.getName() + NAME_JOINER + derivedComponent, key.tags());
+ }
+ };
+ }
+
+ /**
+ * Creates a metric name given the domain, group, and name. The new String follows the following
+ * conventions and rules:
+ *
+ * <ul>
+ * <li>domain is expected to be a host-name like value, e.g. {@code org.apache.kafka}</li>
+ * <li>group is cleaned of redundant words: "-metrics"</li>
+ * <li>the group and metric name is dot separated</li>
+ * <li>The name is created by joining the three components, e.g.:
+ * {@code org.apache.kafka.producer.connection.creation.rate}</li>
+ * </ul>
+ */
+ private static String fullMetricName(String domain, String group, String name) {
+ return domain
+ + NAME_JOINER
+ + cleanGroup(group)
+ + NAME_JOINER
+ + cleanMetric(name);
+ }
+
+ /**
+ * This method maps a raw name to follow conventions and cleans up the result to be more legible:
+ * <ul>
+ * <li> converts names to lower hyphen case conventions
+ * <li> strips redundant parts of the metric name, such as -metrics
+ * <li> normalizes artifacts of hyphen case to dot separated conversion
+ * </ul>
+ */
+ private static String cleanGroup(String group) {
+ group = clean(group, NAME_JOINER);
+ return GROUP_PATTERN.matcher(group).replaceAll("");
+ }
+
+ private static String cleanMetric(String metric) {
+ return clean(metric, NAME_JOINER);
+ }
+
+ /**
+ * Converts a tag name to match the telemetry naming conventions by converting snake_case.
Review Comment:
`by converting [into] snake_case.`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15669: Implement telemetry metric naming strategy (KIP-714) [kafka]
Posted by "xvrl (via GitHub)" <gi...@apache.org>.
xvrl commented on code in PR #14619:
URL: https://github.com/apache/kafka/pull/14619#discussion_r1375018518
##########
clients/src/main/java/org/apache/kafka/common/telemetry/internals/MetricNamingConvention.java:
##########
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.telemetry.internals;
+
+import org.apache.kafka.common.MetricName;
+
+import java.util.Collections;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+/**
+ * This class encapsulates naming and mapping conventions defined as part of
+ * <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-714%3A+Client+metrics+and+observability#KIP714:Clientmetricsandobservability-Metricsnamingandformat">Metrics naming and format</a>
+ */
+public class MetricNamingConvention {
Review Comment:
maybe a more explicit class name would help here. Since the goal of this mapping is to map the metric names to something that fits OpenTelemetry conventions, maybe we can call this OpenTelemetryMetricNamingConvention ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15669: Implement telemetry metric naming strategy (KIP-714) [kafka]
Posted by "apoorvmittal10 (via GitHub)" <gi...@apache.org>.
apoorvmittal10 commented on code in PR #14619:
URL: https://github.com/apache/kafka/pull/14619#discussion_r1375099765
##########
clients/src/main/java/org/apache/kafka/common/telemetry/internals/MetricNamingConvention.java:
##########
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.telemetry.internals;
+
+import org.apache.kafka.common.MetricName;
+
+import java.util.Collections;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+/**
+ * This class encapsulates naming and mapping conventions defined as part of
+ * <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-714%3A+Client+metrics+and+observability#KIP714:Clientmetricsandobservability-Metricsnamingandformat">Metrics naming and format</a>
+ */
+public class MetricNamingConvention {
+
+ private static final String NAME_JOINER = ".";
+ private static final String TAG_JOINER = "_";
+
+ // remove metrics as it is redundant for telemetry metrics naming convention
+ private final static Pattern GROUP_PATTERN = Pattern.compile("\\.(metrics)");
+
+ public static MetricNamingStrategy<MetricName> getClientTelemetryMetricNamingStrategy(String domain) {
+ return new MetricNamingStrategy<MetricName>() {
+ @Override
+ public MetricKey metricKey(MetricName metricName) {
+ Objects.requireNonNull(metricName, "metric name cannot be null");
+ String group = metricName.group() == null ? "" : metricName.group();
+ String rawName = metricName.name() == null ? "" : metricName.name();
+
+ return new MetricKey(fullMetricName(domain, group, rawName),
+ Collections.unmodifiableMap(cleanTags(metricName.tags())));
+ }
+
+ @Override
+ public MetricKey derivedMetricKey(MetricKey key, String derivedComponent) {
+ Objects.requireNonNull(derivedComponent, "derived component cannot be null");
+ return new MetricKey(key.getName() + NAME_JOINER + derivedComponent, key.tags());
+ }
+ };
+ }
+
+ /**
+ * Creates a metric name given the domain, group, and name. The new String follows the following
+ * conventions and rules:
+ *
+ * <ul>
+ * <li>domain is expected to be a host-name like value, e.g. {@code org.apache.kafka}</li>
+ * <li>group is cleaned of redundant words: "-metrics"</li>
+ * <li>the group and metric name is dot separated</li>
+ * <li>The name is created by joining the three components, e.g.:
+ * {@code org.apache.kafka.producer.connection.creation.rate}</li>
+ * </ul>
+ */
+ private static String fullMetricName(String domain, String group, String name) {
+ return domain
+ + NAME_JOINER
+ + cleanGroup(group)
+ + NAME_JOINER
+ + cleanMetric(name);
+ }
+
+ /**
+ * This method maps a raw name to follow conventions and cleans up the result to be more legible:
+ * <ul>
+ * <li> converts names to lower hyphen case conventions
+ * <li> strips redundant parts of the metric name, such as -metrics
+ * <li> normalizes artifacts of hyphen case to dot separated conversion
+ * </ul>
+ */
+ private static String cleanGroup(String group) {
+ group = clean(group, NAME_JOINER);
+ return GROUP_PATTERN.matcher(group).replaceAll("");
+ }
+
+ private static String cleanMetric(String metric) {
+ return clean(metric, NAME_JOINER);
+ }
+
+ /**
+ * Converts a tag name to match the telemetry naming conventions by converting snake_case.
+ * <p>
+ * Kafka metrics have tags name in lower case separated by hyphens. Eg: total-errors
+ *
+ * @param raw the input map
+ * @return the new map with keys replaced by snake_case representations.
+ */
+ private static Map<String, String> cleanTags(Map<String, String> raw) {
+ return raw.entrySet()
+ .stream()
+ .collect(Collectors.toMap(s -> clean(s.getKey(), TAG_JOINER), Entry::getValue));
+ }
+
+ private static String clean(String raw, String joiner) {
+ Objects.requireNonNull(raw, "metric data cannot be null");
+ // Convert to lower case.
Review Comment:
Removed it, moved the comments in javadoc as suggested in some other feedback.
##########
clients/src/main/java/org/apache/kafka/common/telemetry/internals/MetricNamingConvention.java:
##########
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.telemetry.internals;
+
+import org.apache.kafka.common.MetricName;
+
+import java.util.Collections;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+/**
+ * This class encapsulates naming and mapping conventions defined as part of
+ * <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-714%3A+Client+metrics+and+observability#KIP714:Clientmetricsandobservability-Metricsnamingandformat">Metrics naming and format</a>
+ */
+public class MetricNamingConvention {
+
+ private static final String NAME_JOINER = ".";
+ private static final String TAG_JOINER = "_";
+
+ // remove metrics as it is redundant for telemetry metrics naming convention
+ private final static Pattern GROUP_PATTERN = Pattern.compile("\\.(metrics)");
+
+ public static MetricNamingStrategy<MetricName> getClientTelemetryMetricNamingStrategy(String domain) {
+ return new MetricNamingStrategy<MetricName>() {
+ @Override
+ public MetricKey metricKey(MetricName metricName) {
+ Objects.requireNonNull(metricName, "metric name cannot be null");
+ String group = metricName.group() == null ? "" : metricName.group();
+ String rawName = metricName.name() == null ? "" : metricName.name();
+
+ return new MetricKey(fullMetricName(domain, group, rawName),
+ Collections.unmodifiableMap(cleanTags(metricName.tags())));
+ }
+
+ @Override
+ public MetricKey derivedMetricKey(MetricKey key, String derivedComponent) {
+ Objects.requireNonNull(derivedComponent, "derived component cannot be null");
+ return new MetricKey(key.getName() + NAME_JOINER + derivedComponent, key.tags());
+ }
+ };
+ }
+
+ /**
+ * Creates a metric name given the domain, group, and name. The new String follows the following
+ * conventions and rules:
+ *
+ * <ul>
+ * <li>domain is expected to be a host-name like value, e.g. {@code org.apache.kafka}</li>
+ * <li>group is cleaned of redundant words: "-metrics"</li>
+ * <li>the group and metric name is dot separated</li>
+ * <li>The name is created by joining the three components, e.g.:
+ * {@code org.apache.kafka.producer.connection.creation.rate}</li>
+ * </ul>
+ */
+ private static String fullMetricName(String domain, String group, String name) {
+ return domain
+ + NAME_JOINER
+ + cleanGroup(group)
+ + NAME_JOINER
+ + cleanMetric(name);
+ }
+
+ /**
+ * This method maps a raw name to follow conventions and cleans up the result to be more legible:
+ * <ul>
+ * <li> converts names to lower hyphen case conventions
+ * <li> strips redundant parts of the metric name, such as -metrics
+ * <li> normalizes artifacts of hyphen case to dot separated conversion
+ * </ul>
+ */
+ private static String cleanGroup(String group) {
+ group = clean(group, NAME_JOINER);
+ return GROUP_PATTERN.matcher(group).replaceAll("");
+ }
+
+ private static String cleanMetric(String metric) {
+ return clean(metric, NAME_JOINER);
+ }
+
+ /**
+ * Converts a tag name to match the telemetry naming conventions by converting snake_case.
+ * <p>
+ * Kafka metrics have tags name in lower case separated by hyphens. Eg: total-errors
+ *
+ * @param raw the input map
+ * @return the new map with keys replaced by snake_case representations.
+ */
+ private static Map<String, String> cleanTags(Map<String, String> raw) {
+ return raw.entrySet()
+ .stream()
+ .collect(Collectors.toMap(s -> clean(s.getKey(), TAG_JOINER), Entry::getValue));
+ }
+
+ private static String clean(String raw, String joiner) {
+ Objects.requireNonNull(raw, "metric data cannot be null");
+ // Convert to lower case.
+ String lowerCase = raw.toLowerCase(Locale.ROOT);
+ // Replace hyphens with telemetry name joiner i.e. dot.
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15669: Implement telemetry metric naming strategy (KIP-714) [kafka]
Posted by "xvrl (via GitHub)" <gi...@apache.org>.
xvrl commented on code in PR #14619:
URL: https://github.com/apache/kafka/pull/14619#discussion_r1375023475
##########
clients/src/main/java/org/apache/kafka/common/telemetry/internals/MetricNamingConvention.java:
##########
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.telemetry.internals;
+
+import org.apache.kafka.common.MetricName;
+
+import java.util.Collections;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+/**
+ * This class encapsulates naming and mapping conventions defined as part of
+ * <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-714%3A+Client+metrics+and+observability#KIP714:Clientmetricsandobservability-Metricsnamingandformat">Metrics naming and format</a>
+ */
+public class MetricNamingConvention {
+
+ private static final String NAME_JOINER = ".";
+ private static final String TAG_JOINER = "_";
+
+ // remove metrics as it is redundant for telemetry metrics naming convention
+ private final static Pattern GROUP_PATTERN = Pattern.compile("\\.(metrics)");
+
+ public static MetricNamingStrategy<MetricName> getClientTelemetryMetricNamingStrategy(String domain) {
Review Comment:
Maybe we can call it `prefix` instead of domain?
I believe this class should be reusable to also expose broker, streams, etc. metrics in OpenTelemetry format one day, so I would suggest to keep it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15669: Implement telemetry metric naming strategy (KIP-714) [kafka]
Posted by "apoorvmittal10 (via GitHub)" <gi...@apache.org>.
apoorvmittal10 commented on code in PR #14619:
URL: https://github.com/apache/kafka/pull/14619#discussion_r1375100061
##########
clients/src/main/java/org/apache/kafka/common/telemetry/internals/MetricNamingConvention.java:
##########
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.telemetry.internals;
+
+import org.apache.kafka.common.MetricName;
+
+import java.util.Collections;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+/**
+ * This class encapsulates naming and mapping conventions defined as part of
+ * <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-714%3A+Client+metrics+and+observability#KIP714:Clientmetricsandobservability-Metricsnamingandformat">Metrics naming and format</a>
+ */
+public class MetricNamingConvention {
+
+ private static final String NAME_JOINER = ".";
+ private static final String TAG_JOINER = "_";
+
+ // remove metrics as it is redundant for telemetry metrics naming convention
+ private final static Pattern GROUP_PATTERN = Pattern.compile("\\.(metrics)");
+
+ public static MetricNamingStrategy<MetricName> getClientTelemetryMetricNamingStrategy(String domain) {
+ return new MetricNamingStrategy<MetricName>() {
+ @Override
+ public MetricKey metricKey(MetricName metricName) {
+ Objects.requireNonNull(metricName, "metric name cannot be null");
+ String group = metricName.group() == null ? "" : metricName.group();
Review Comment:
Removed, thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15669: Implement telemetry metric naming strategy (KIP-714) [kafka]
Posted by "xvrl (via GitHub)" <gi...@apache.org>.
xvrl commented on code in PR #14619:
URL: https://github.com/apache/kafka/pull/14619#discussion_r1375025585
##########
clients/src/main/java/org/apache/kafka/common/telemetry/internals/MetricNamingConvention.java:
##########
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.telemetry.internals;
+
+import org.apache.kafka.common.MetricName;
+
+import java.util.Collections;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+/**
+ * This class encapsulates naming and mapping conventions defined as part of
+ * <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-714%3A+Client+metrics+and+observability#KIP714:Clientmetricsandobservability-Metricsnamingandformat">Metrics naming and format</a>
+ */
+public class MetricNamingConvention {
+
+ private static final String NAME_JOINER = ".";
+ private static final String TAG_JOINER = "_";
+
+ // remove metrics as it is redundant for telemetry metrics naming convention
+ private final static Pattern GROUP_PATTERN = Pattern.compile("\\.(metrics)");
+
+ public static MetricNamingStrategy<MetricName> getClientTelemetryMetricNamingStrategy(String domain) {
+ return new MetricNamingStrategy<MetricName>() {
+ @Override
+ public MetricKey metricKey(MetricName metricName) {
+ Objects.requireNonNull(metricName, "metric name cannot be null");
+ String group = metricName.group() == null ? "" : metricName.group();
+ String rawName = metricName.name() == null ? "" : metricName.name();
+
+ return new MetricKey(fullMetricName(domain, group, rawName),
+ Collections.unmodifiableMap(cleanTags(metricName.tags())));
+ }
+
+ @Override
+ public MetricKey derivedMetricKey(MetricKey key, String derivedComponent) {
+ Objects.requireNonNull(derivedComponent, "derived component cannot be null");
+ return new MetricKey(key.getName() + NAME_JOINER + derivedComponent, key.tags());
+ }
+ };
+ }
+
+ /**
+ * Creates a metric name given the domain, group, and name. The new String follows the following
+ * conventions and rules:
+ *
+ * <ul>
+ * <li>domain is expected to be a host-name like value, e.g. {@code org.apache.kafka}</li>
+ * <li>group is cleaned of redundant words: "-metrics"</li>
+ * <li>the group and metric name is dot separated</li>
+ * <li>The name is created by joining the three components, e.g.:
+ * {@code org.apache.kafka.producer.connection.creation.rate}</li>
+ * </ul>
+ */
+ private static String fullMetricName(String domain, String group, String name) {
+ return domain
+ + NAME_JOINER
+ + cleanGroup(group)
+ + NAME_JOINER
+ + cleanMetric(name);
+ }
+
+ /**
+ * This method maps a raw name to follow conventions and cleans up the result to be more legible:
+ * <ul>
+ * <li> converts names to lower hyphen case conventions
+ * <li> strips redundant parts of the metric name, such as -metrics
+ * <li> normalizes artifacts of hyphen case to dot separated conversion
+ * </ul>
+ */
+ private static String cleanGroup(String group) {
+ group = clean(group, NAME_JOINER);
+ return GROUP_PATTERN.matcher(group).replaceAll("");
+ }
+
+ private static String cleanMetric(String metric) {
+ return clean(metric, NAME_JOINER);
+ }
+
+ /**
+ * Converts a tag name to match the telemetry naming conventions by converting snake_case.
+ * <p>
+ * Kafka metrics have tags name in lower case separated by hyphens. Eg: total-errors
+ *
+ * @param raw the input map
+ * @return the new map with keys replaced by snake_case representations.
+ */
+ private static Map<String, String> cleanTags(Map<String, String> raw) {
+ return raw.entrySet()
+ .stream()
+ .collect(Collectors.toMap(s -> clean(s.getKey(), TAG_JOINER), Entry::getValue));
+ }
+
+ private static String clean(String raw, String joiner) {
+ Objects.requireNonNull(raw, "metric data cannot be null");
+ // Convert to lower case.
+ String lowerCase = raw.toLowerCase(Locale.ROOT);
+ // Replace hyphens with telemetry name joiner i.e. dot.
Review Comment:
maybe move both of those comments into the java docs for the methods to make it clear what it does
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15669: Implement telemetry metric naming strategy (KIP-714) [kafka]
Posted by "apoorvmittal10 (via GitHub)" <gi...@apache.org>.
apoorvmittal10 commented on code in PR #14619:
URL: https://github.com/apache/kafka/pull/14619#discussion_r1375100926
##########
clients/src/test/java/org/apache/kafka/common/telemetry/internals/MetricNamingConventionTest.java:
##########
@@ -0,0 +1,196 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.telemetry.internals;
+
+import org.apache.kafka.common.MetricName;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Collections;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+
+public class MetricNamingConventionTest {
Review Comment:
Added, thanks for the suggestion 👍.
##########
clients/src/test/java/org/apache/kafka/common/telemetry/internals/MetricNamingConventionTest.java:
##########
@@ -0,0 +1,196 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.telemetry.internals;
+
+import org.apache.kafka.common.MetricName;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Collections;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+
+public class MetricNamingConventionTest {
Review Comment:
Added, thanks for the suggestion 👍.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15669: Implement telemetry metric naming strategy (KIP-714) [kafka]
Posted by "xvrl (via GitHub)" <gi...@apache.org>.
xvrl commented on code in PR #14619:
URL: https://github.com/apache/kafka/pull/14619#discussion_r1375026123
##########
clients/src/test/java/org/apache/kafka/common/telemetry/internals/MetricNamingConventionTest.java:
##########
@@ -0,0 +1,196 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.telemetry.internals;
+
+import org.apache.kafka.common.MetricName;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Collections;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+
+public class MetricNamingConventionTest {
Review Comment:
can we add explicit tests for the metrics we have defined in the KIP?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15669: Implement telemetry metric naming strategy (KIP-714) [kafka]
Posted by "apoorvmittal10 (via GitHub)" <gi...@apache.org>.
apoorvmittal10 commented on code in PR #14619:
URL: https://github.com/apache/kafka/pull/14619#discussion_r1375100148
##########
clients/src/main/java/org/apache/kafka/common/telemetry/internals/MetricNamingConvention.java:
##########
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.telemetry.internals;
+
+import org.apache.kafka.common.MetricName;
+
+import java.util.Collections;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+/**
+ * This class encapsulates naming and mapping conventions defined as part of
+ * <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-714%3A+Client+metrics+and+observability#KIP714:Clientmetricsandobservability-Metricsnamingandformat">Metrics naming and format</a>
+ */
+public class MetricNamingConvention {
+
+ private static final String NAME_JOINER = ".";
+ private static final String TAG_JOINER = "_";
+
+ // remove metrics as it is redundant for telemetry metrics naming convention
+ private final static Pattern GROUP_PATTERN = Pattern.compile("\\.(metrics)");
+
+ public static MetricNamingStrategy<MetricName> getClientTelemetryMetricNamingStrategy(String domain) {
+ return new MetricNamingStrategy<MetricName>() {
Review Comment:
Done, added test case as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15669: Implement telemetry metric naming strategy (KIP-714) [kafka]
Posted by "apoorvmittal10 (via GitHub)" <gi...@apache.org>.
apoorvmittal10 commented on code in PR #14619:
URL: https://github.com/apache/kafka/pull/14619#discussion_r1375100801
##########
clients/src/main/java/org/apache/kafka/common/telemetry/internals/MetricNamingConvention.java:
##########
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.telemetry.internals;
+
+import org.apache.kafka.common.MetricName;
+
+import java.util.Collections;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+/**
+ * This class encapsulates naming and mapping conventions defined as part of
+ * <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-714%3A+Client+metrics+and+observability#KIP714:Clientmetricsandobservability-Metricsnamingandformat">Metrics naming and format</a>
+ */
+public class MetricNamingConvention {
Review Comment:
I have changed name to `TelemtryMetricNamingConvention` as we generally referred by same in KIP. Let me know if it works or we still feel that the name should be more specific as you suggested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15669: Implement telemetry metric naming strategy (KIP-714) [kafka]
Posted by "apoorvmittal10 (via GitHub)" <gi...@apache.org>.
apoorvmittal10 commented on PR #14619:
URL: https://github.com/apache/kafka/pull/14619#issuecomment-1776849161
Depends on PR - https://github.com/apache/kafka/pull/14575 for build.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15669: Implement telemetry metric naming strategy (KIP-714) [kafka]
Posted by "apoorvmittal10 (via GitHub)" <gi...@apache.org>.
apoorvmittal10 commented on code in PR #14619:
URL: https://github.com/apache/kafka/pull/14619#discussion_r1369241432
##########
clients/src/test/java/org/apache/kafka/common/telemetry/internals/MetricNamingConventionTest.java:
##########
@@ -0,0 +1,196 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.telemetry.internals;
+
+import org.apache.kafka.common.MetricName;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Collections;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+
+public class MetricNamingConventionTest {
+
+ private MetricNamingStrategy<MetricName> metricNamingStrategy;
+
+ @BeforeEach
+ public void setUp() {
+ metricNamingStrategy = MetricNamingConvention
+ .getClientTelemetryMetricNamingStrategy("org.apache.kafka");
+ }
+
+ @Test
+ public void testMetricKey() {
+ MetricName metricName = new MetricName("name", "group", "description",
+ Collections.emptyMap());
+ MetricKey metricKey = metricNamingStrategy.metricKey(metricName);
+
+ assertEquals("org.apache.kafka.group.name", metricKey.getName());
+ assertEquals(Collections.emptyMap(), metricKey.tags());
+ }
+
+ @Test
+ public void testMetricKeyWithHyphenNameAndTags() {
+ Map<String, String> tags = new HashMap<>();
+ tags.put("tag1", "value1");
+ tags.put("tag2", "value2");
+
+ MetricName metricName = new MetricName("test-name", "group-name", "description", tags);
+ MetricKey metricKey = metricNamingStrategy.metricKey(metricName);
+
+ assertEquals("org.apache.kafka.group.name.test.name", metricKey.getName());
+ assertEquals(tags, metricKey.tags());
+ }
+
+ /**
+ * Test metric key with mixed name and mixed tags where mixed represents combination of upper case,
+ * lower case, numbers, hyphen, dot, underscore and special characters.
+ */
+ @Test
+ public void testMetricKeyWithMixedNameAndMixedTags() {
+ Map<String, String> tags = new HashMap<>();
+ tags.put("tag1-Ab-2_(", "value1");
+ tags.put("tag2-HELLO.@", "value2");
+
+ MetricName metricName = new MetricName("test-Name-1.$", "grouP-name-AB_&", "description", tags);
+ MetricKey metricKey = metricNamingStrategy.metricKey(metricName);
+
+ tags.clear();
+ tags.put("tag1_ab_2_(", "value1");
+ tags.put("tag2_hello.@", "value2");
+
+ assertEquals("org.apache.kafka.group.name.ab_&.test.name.1.$", metricKey.getName());
+ assertEquals(tags, metricKey.tags());
+ }
+
+ @Test
+ public void testMetricKeyWithNullMetricName() {
+ Exception e = assertThrows(NullPointerException.class, () -> metricNamingStrategy.metricKey(null));
+ assertEquals("metric name cannot be null", e.getMessage());
+ }
+
+ @Test
+ public void testMetricKeyWithEmptyName() {
+ MetricName metricName = new MetricName("", "group-1A", "description",
+ Collections.emptyMap());
+ MetricKey metricKey = metricNamingStrategy.metricKey(metricName);
+
+ // If there is no name, then the telemetry metric name will have dot in the end though
+ // metric names always have a name.
+ assertEquals("org.apache.kafka.group.1a.", metricKey.getName());
+ assertEquals(Collections.emptyMap(), metricKey.tags());
+ }
+
+ @Test
+ public void testMetricKeyWithEmptyGroup() {
+ MetricName metricName = new MetricName("name", "", "description",
+ Collections.emptyMap());
+ MetricKey metricKey = metricNamingStrategy.metricKey(metricName);
+
+ // If there is no group, then the telemetry metric name will have consecutive dots, though
+ // metric names always have group name.
+ assertEquals("org.apache.kafka..name", metricKey.getName());
+ assertEquals(Collections.emptyMap(), metricKey.tags());
+ }
+
+ @Test
+ public void testMetricKeyWithAdditionalMetricsSuffixInGroup() {
+ MetricName metricName = new MetricName("name", "group-metrics", "description",
+ Collections.emptyMap());
+ MetricKey metricKey = metricNamingStrategy.metricKey(metricName);
+
+ // '-metrics' gets removed from the group name.
+ assertEquals("org.apache.kafka.group.name", metricKey.getName());
+ assertEquals(Collections.emptyMap(), metricKey.tags());
+ }
+
+ @Test
+ public void testMetricKeyWithMultipleMetricsSuffixInGroup() {
+ MetricName metricName = new MetricName("name-metrics", "group-metrics-metrics", "description",
+ Collections.emptyMap());
+ MetricKey metricKey = metricNamingStrategy.metricKey(metricName);
+
+ // '-metrics' gets removed from the group name.
+ assertEquals("org.apache.kafka.group.name.metrics", metricKey.getName());
+ assertEquals(Collections.emptyMap(), metricKey.tags());
+ }
+
+ @Test
+ public void testMetricKeyWithNullTagKey() {
+ MetricName metricName = new MetricName("name", "group", "description",
+ Collections.singletonMap(null, "value1"));
+ Exception e = assertThrows(NullPointerException.class, () -> metricNamingStrategy.metricKey(metricName));
+ assertEquals("metric data cannot be null", e.getMessage());
+ }
+
+ @Test
+ public void testMetricKeyWithBlankTagKey() {
+ MetricName metricName = new MetricName("name", "group", "description",
+ Collections.singletonMap("", "value1"));
+ MetricKey metricKey = metricNamingStrategy.metricKey(metricName);
+
+ assertEquals("org.apache.kafka.group.name", metricKey.getName());
+ assertEquals(Collections.singletonMap("", "value1"), metricKey.tags());
+ }
+
+ @Test
+ public void testDerivedMetricKey() {
+ MetricName metricName = new MetricName("name", "group", "description",
+ Collections.emptyMap());
+ MetricKey metricKey = metricNamingStrategy.derivedMetricKey(
+ metricNamingStrategy.metricKey(metricName), "delta");
+
+ assertEquals("org.apache.kafka.group.name.delta", metricKey.getName());
+ assertEquals(Collections.emptyMap(), metricKey.tags());
+ }
+
+ @Test
+ public void testDerivedMetricKeyWithTags() {
+ MetricName metricName = new MetricName("name", "group", "description",
+ Collections.singletonMap("tag1", "value1"));
+ MetricKey metricKey = metricNamingStrategy.derivedMetricKey(
+ metricNamingStrategy.metricKey(metricName), "delta");
+
+ assertEquals("org.apache.kafka.group.name.delta", metricKey.getName());
+ assertEquals(Collections.singletonMap("tag1", "value1"), metricKey.tags());
+ }
+
+ @Test
+ public void testDerivedMetricKeyWithNullComponent() {
+ MetricName metricName = new MetricName("name", "group", "description",
+ Collections.emptyMap());
+ Exception e = assertThrows(NullPointerException.class, () -> metricNamingStrategy.derivedMetricKey(
+ metricNamingStrategy.metricKey(metricName), null));
+ assertEquals("derived component cannot be null", e.getMessage());
+ }
+
+ @Test
+ public void testDerivedMetricKeyWithBlankComponent() {
+ MetricName metricName = new MetricName("name", "group", "description",
+ Collections.emptyMap());
+ MetricKey metricKey = metricNamingStrategy.derivedMetricKey(
+ metricNamingStrategy.metricKey(metricName), "");
+
+ // Ends with dot, though derived component should
Review Comment:
Completed it, sorry.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15669: Implement telemetry metric naming strategy (KIP-714) [kafka]
Posted by "apoorvmittal10 (via GitHub)" <gi...@apache.org>.
apoorvmittal10 commented on code in PR #14619:
URL: https://github.com/apache/kafka/pull/14619#discussion_r1375099869
##########
clients/src/main/java/org/apache/kafka/common/telemetry/internals/MetricNamingConvention.java:
##########
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.telemetry.internals;
+
+import org.apache.kafka.common.MetricName;
+
+import java.util.Collections;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+/**
+ * This class encapsulates naming and mapping conventions defined as part of
+ * <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-714%3A+Client+metrics+and+observability#KIP714:Clientmetricsandobservability-Metricsnamingandformat">Metrics naming and format</a>
+ */
+public class MetricNamingConvention {
+
+ private static final String NAME_JOINER = ".";
+ private static final String TAG_JOINER = "_";
+
+ // remove metrics as it is redundant for telemetry metrics naming convention
+ private final static Pattern GROUP_PATTERN = Pattern.compile("\\.(metrics)");
+
+ public static MetricNamingStrategy<MetricName> getClientTelemetryMetricNamingStrategy(String domain) {
+ return new MetricNamingStrategy<MetricName>() {
+ @Override
+ public MetricKey metricKey(MetricName metricName) {
+ Objects.requireNonNull(metricName, "metric name cannot be null");
+ String group = metricName.group() == null ? "" : metricName.group();
+ String rawName = metricName.name() == null ? "" : metricName.name();
+
+ return new MetricKey(fullMetricName(domain, group, rawName),
+ Collections.unmodifiableMap(cleanTags(metricName.tags())));
+ }
+
+ @Override
+ public MetricKey derivedMetricKey(MetricKey key, String derivedComponent) {
+ Objects.requireNonNull(derivedComponent, "derived component cannot be null");
+ return new MetricKey(key.getName() + NAME_JOINER + derivedComponent, key.tags());
+ }
+ };
+ }
+
+ /**
+ * Creates a metric name given the domain, group, and name. The new String follows the following
+ * conventions and rules:
+ *
+ * <ul>
+ * <li>domain is expected to be a host-name like value, e.g. {@code org.apache.kafka}</li>
+ * <li>group is cleaned of redundant words: "-metrics"</li>
+ * <li>the group and metric name is dot separated</li>
+ * <li>The name is created by joining the three components, e.g.:
+ * {@code org.apache.kafka.producer.connection.creation.rate}</li>
+ * </ul>
+ */
+ private static String fullMetricName(String domain, String group, String name) {
+ return domain
+ + NAME_JOINER
+ + cleanGroup(group)
+ + NAME_JOINER
+ + cleanMetric(name);
+ }
+
+ /**
+ * This method maps a raw name to follow conventions and cleans up the result to be more legible:
+ * <ul>
+ * <li> converts names to lower hyphen case conventions
+ * <li> strips redundant parts of the metric name, such as -metrics
+ * <li> normalizes artifacts of hyphen case to dot separated conversion
+ * </ul>
+ */
+ private static String cleanGroup(String group) {
+ group = clean(group, NAME_JOINER);
+ return GROUP_PATTERN.matcher(group).replaceAll("");
+ }
+
+ private static String cleanMetric(String metric) {
+ return clean(metric, NAME_JOINER);
+ }
+
+ /**
+ * Converts a tag name to match the telemetry naming conventions by converting snake_case.
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org