You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "AndrewJSchofield (via GitHub)" <gi...@apache.org> on 2025/03/26 15:14:44 UTC
[PR] KAFKA-16894: Define features to enable share groups [kafka]
AndrewJSchofield opened a new pull request, #19293:
URL: https://github.com/apache/kafka/pull/19293
This PR proposes two switches to enable share groups for 4.1 (preview) and 4.2 (GA).
* `group.version=2` to indicate that the group coordinator is able to understand share group records.
* `share.version=1` to indicate that share groups are enabled. This is used as the switch for turning share groups on and off.
In 4.1, the defaults will be `group.version=2` and `share.version=0`. Then a user wanting to evaluate the preview of KIP-932 would use `bin/kafka-features.sh --bootstrap.server xxxx upgrade --feature share.version=1`.
In 4.2, the default will be `share.version=1`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16894: Define features to enable share groups [kafka]
Posted by "AndrewJSchofield (via GitHub)" <gi...@apache.org>.
AndrewJSchofield commented on code in PR #19293:
URL: https://github.com/apache/kafka/pull/19293#discussion_r2022234245
##########
server-common/src/main/java/org/apache/kafka/server/common/ShareVersion.java:
##########
@@ -0,0 +1,78 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.server.common;
+
+import java.util.Map;
+
+public enum ShareVersion implements FeatureVersion {
+
+ // Version 0 does not enable share groups.
+ SV_0(0, MetadataVersion.MINIMUM_VERSION, Map.of()),
+
+ // Version 1 enables share groups (KIP-932).
+ // This is a preview in 4.1, and production-ready in 4.2.
+ SV_1(1, MetadataVersion.IBP_4_2_IV0, Map.of(MetadataVersion.FEATURE_NAME, MetadataVersion.IBP_4_1_IV1.featureLevel()));
Review Comment:
This PR is still a draft. I am going to remove the GroupVersion change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16894: Define feature to enable share groups [kafka]
Posted by "jolshan (via GitHub)" <gi...@apache.org>.
jolshan commented on code in PR #19293:
URL: https://github.com/apache/kafka/pull/19293#discussion_r2066993014
##########
server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java:
##########
@@ -112,7 +112,22 @@ public enum MetadataVersion {
//
// Enables ELR by default for new clusters (KIP-966).
- IBP_4_1_IV0(26, "4.1", "IV0", false);
+ IBP_4_1_IV0(26, "4.1", "IV0", false),
+
+ // Enables share groups. Note, share groups are for preview only in 4.1. (KIP-932).
+ IBP_4_1_IV1(27, "4.1", "IV1", false),
Review Comment:
+1 to Jun's comment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16894: Define features to enable share groups [kafka]
Posted by "dajac (via GitHub)" <gi...@apache.org>.
dajac commented on PR #19293:
URL: https://github.com/apache/kafka/pull/19293#issuecomment-2755031196
@AndrewJSchofield I would like to better understand how `group.version=2` will be used. Could you elaborate a bit more on what we will gate with it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16894: Define feature to enable share groups [kafka]
Posted by "AndrewJSchofield (via GitHub)" <gi...@apache.org>.
AndrewJSchofield merged PR #19293:
URL: https://github.com/apache/kafka/pull/19293
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16894: Define feature to enable share groups [kafka]
Posted by "junrao (via GitHub)" <gi...@apache.org>.
junrao commented on code in PR #19293:
URL: https://github.com/apache/kafka/pull/19293#discussion_r2066941920
##########
server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java:
##########
@@ -112,7 +112,22 @@ public enum MetadataVersion {
//
// Enables ELR by default for new clusters (KIP-966).
- IBP_4_1_IV0(26, "4.1", "IV0", false);
+ IBP_4_1_IV0(26, "4.1", "IV0", false),
+
+ // Enables share groups. Note, share groups are for preview only in 4.1. (KIP-932).
+ IBP_4_1_IV1(27, "4.1", "IV1", false),
Review Comment:
@AndrewJSchofield : Thanks for the reply. Before feature was properly introduced, all inter broker RPCs and record formats are gated by MVs. When we started to use features, the intention was to break up the RPCs and records by feature. For example, all RPCs used by the group coordinator and all records used in the offset topic are gated by GV, not MV. Similarly, all RPCs used by Qfk should be gated by SV, not MV. Could you submit a followup PR on this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16894: Define feature to enable share groups [kafka]
Posted by "junrao (via GitHub)" <gi...@apache.org>.
junrao commented on code in PR #19293:
URL: https://github.com/apache/kafka/pull/19293#discussion_r2038441012
##########
server-common/src/main/java/org/apache/kafka/server/common/ShareVersion.java:
##########
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.server.common;
+
+import java.util.Map;
+
+public enum ShareVersion implements FeatureVersion {
+
+ // Version 0 does not enable share groups.
+ SV_0(0, MetadataVersion.MINIMUM_VERSION, Map.of()),
+
+ // Version 1 enables share groups (KIP-932).
+ // This is a preview in 4.1, and production-ready in 4.2.
+ SV_1(1, MetadataVersion.IBP_4_2_IV0, Map.of(MetadataVersion.FEATURE_NAME, MetadataVersion.IBP_4_1_IV1.featureLevel()));
+
+ public static final String FEATURE_NAME = "share.version";
+
+ public static final ShareVersion LATEST_PRODUCTION = SV_0;
+
+ private final short featureLevel;
+ private final MetadataVersion bootstrapMetadataVersion;
+ private final Map<String, Short> dependencies;
+
+ ShareVersion(
+ int featureLevel,
+ MetadataVersion bootstrapMetadataVersion,
+ Map<String, Short> dependencies
+ ) {
+ this.featureLevel = (short) featureLevel;
+ this.bootstrapMetadataVersion = bootstrapMetadataVersion;
+ this.dependencies = dependencies;
+ }
+
+ @Override
+ public short featureLevel() {
+ return featureLevel;
+ }
+
+ @Override
+ public String featureName() {
+ return FEATURE_NAME;
+ }
+
+ @Override
+ public MetadataVersion bootstrapMetadataVersion() {
+ return bootstrapMetadataVersion;
+ }
+
+ @Override
+ public Map<String, Short> dependencies() {
+ return dependencies;
+ }
+
+ public boolean supportsShareGroups() {
Review Comment:
This method is unused. Do you plan to use it in the future?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16894: Define features to enable share groups [kafka]
Posted by "AndrewJSchofield (via GitHub)" <gi...@apache.org>.
AndrewJSchofield commented on PR #19293:
URL: https://github.com/apache/kafka/pull/19293#issuecomment-2755063749
> @AndrewJSchofield I would like to better understand how `group.version=2` will be used. Could you elaborate a bit more on what we will gate with it?
It's for versioning of the records in the `__consumer_offsets` topic. If `group.version=2`, we know that the GC can understand the KIP-932 records on that topic. And then perhaps `group.version=3` adds KIP-1071 records too. Maybe it is unnecessary, and I can certainly be persuaded to remove it and stick with `ShareVersion` only.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16894: Define features to enable share groups [kafka]
Posted by "AndrewJSchofield (via GitHub)" <gi...@apache.org>.
AndrewJSchofield commented on code in PR #19293:
URL: https://github.com/apache/kafka/pull/19293#discussion_r2022235254
##########
server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java:
##########
@@ -132,7 +143,7 @@ public enum MetadataVersion {
* <strong>Think carefully before you update this value. ONCE A METADATA VERSION IS PRODUCTION,
* IT CANNOT BE CHANGED.</strong>
*/
- public static final MetadataVersion LATEST_PRODUCTION = IBP_4_0_IV3;
+ public static final MetadataVersion LATEST_PRODUCTION = IBP_4_1_IV1;
Review Comment:
At some point, we need to bump the LATEST_PRODUCTION for 4.1. If this PR is not going to be a suitable point for this, please let me know.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16894: Define feature to enable share groups [kafka]
Posted by "AndrewJSchofield (via GitHub)" <gi...@apache.org>.
AndrewJSchofield commented on code in PR #19293:
URL: https://github.com/apache/kafka/pull/19293#discussion_r2023287495
##########
server-common/src/main/java/org/apache/kafka/server/common/ShareVersion.java:
##########
@@ -0,0 +1,78 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.server.common;
+
+import java.util.Map;
+
+public enum ShareVersion implements FeatureVersion {
+
+ // Version 0 does not enable share groups.
+ SV_0(0, MetadataVersion.MINIMUM_VERSION, Map.of()),
+
+ // Version 1 enables share groups (KIP-932).
+ // This is a preview in 4.1, and production-ready in 4.2.
+ SV_1(1, MetadataVersion.IBP_4_2_IV0, Map.of(MetadataVersion.FEATURE_NAME, MetadataVersion.IBP_4_1_IV1.featureLevel()));
Review Comment:
I would say not now that AK 4.1 contains only the new group coordinator. GV_1 means consumer group protocol is enabled. Share group protocol doesn't rely on that. But @dajac will be the authority on this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16894: Define feature to enable share groups [kafka]
Posted by "junrao (via GitHub)" <gi...@apache.org>.
junrao commented on code in PR #19293:
URL: https://github.com/apache/kafka/pull/19293#discussion_r2038411412
##########
core/src/test/scala/unit/kafka/server/AbstractApiVersionsRequestTest.scala:
##########
@@ -85,6 +85,9 @@ abstract class AbstractApiVersionsRequestTest(cluster: ClusterInstance) {
assertEquals(0, apiVersionsResponse.data().supportedFeatures().find(EligibleLeaderReplicasVersion.FEATURE_NAME).minVersion())
assertEquals(EligibleLeaderReplicasVersion.ELRV_1.featureLevel(), apiVersionsResponse.data().supportedFeatures().find(EligibleLeaderReplicasVersion.FEATURE_NAME).maxVersion())
+
+ assertEquals(0, apiVersionsResponse.data().supportedFeatures().find(ShareVersion.FEATURE_NAME).minVersion())
+ assertEquals(ShareVersion.SV_1.featureLevel(), apiVersionsResponse.data().supportedFeatures().find(ShareVersion.FEATURE_NAME).maxVersion())
Review Comment:
Yes, I was just trying to understand where the test sets unstable.feature.versions.enable to true. It seems that this is done in `KafkaClusterTestKit`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16894: Define feature to enable share groups [kafka]
Posted by "AndrewJSchofield (via GitHub)" <gi...@apache.org>.
AndrewJSchofield commented on code in PR #19293:
URL: https://github.com/apache/kafka/pull/19293#discussion_r2039016521
##########
server-common/src/main/java/org/apache/kafka/server/common/ShareVersion.java:
##########
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.server.common;
+
+import java.util.Map;
+
+public enum ShareVersion implements FeatureVersion {
+
+ // Version 0 does not enable share groups.
+ SV_0(0, MetadataVersion.MINIMUM_VERSION, Map.of()),
+
+ // Version 1 enables share groups (KIP-932).
+ // This is a preview in 4.1, and production-ready in 4.2.
+ SV_1(1, MetadataVersion.IBP_4_2_IV0, Map.of(MetadataVersion.FEATURE_NAME, MetadataVersion.IBP_4_1_IV1.featureLevel()));
+
+ public static final String FEATURE_NAME = "share.version";
+
+ public static final ShareVersion LATEST_PRODUCTION = SV_0;
+
+ private final short featureLevel;
+ private final MetadataVersion bootstrapMetadataVersion;
+ private final Map<String, Short> dependencies;
+
+ ShareVersion(
+ int featureLevel,
+ MetadataVersion bootstrapMetadataVersion,
+ Map<String, Short> dependencies
+ ) {
+ this.featureLevel = (short) featureLevel;
+ this.bootstrapMetadataVersion = bootstrapMetadataVersion;
+ this.dependencies = dependencies;
+ }
+
+ @Override
+ public short featureLevel() {
+ return featureLevel;
+ }
+
+ @Override
+ public String featureName() {
+ return FEATURE_NAME;
+ }
+
+ @Override
+ public MetadataVersion bootstrapMetadataVersion() {
+ return bootstrapMetadataVersion;
+ }
+
+ @Override
+ public Map<String, Short> dependencies() {
+ return dependencies;
+ }
+
+ public boolean supportsShareGroups() {
Review Comment:
Yes. I am going to use this feature to gate support for the KIP-932 APIs in `KafkaApis.scala`. Equivalent to `isConsumerGroupProtocolEnabled` in that class. Keeping this PR nice and small as the first step.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16894: Define features to enable share groups [kafka]
Posted by "junrao (via GitHub)" <gi...@apache.org>.
junrao commented on code in PR #19293:
URL: https://github.com/apache/kafka/pull/19293#discussion_r2021846262
##########
server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java:
##########
@@ -132,7 +143,7 @@ public enum MetadataVersion {
* <strong>Think carefully before you update this value. ONCE A METADATA VERSION IS PRODUCTION,
* IT CANNOT BE CHANGED.</strong>
*/
- public static final MetadataVersion LATEST_PRODUCTION = IBP_4_0_IV3;
+ public static final MetadataVersion LATEST_PRODUCTION = IBP_4_1_IV1;
Review Comment:
Is this change intended?
##########
server-common/src/main/java/org/apache/kafka/server/common/ShareVersion.java:
##########
@@ -0,0 +1,78 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.server.common;
+
+import java.util.Map;
+
+public enum ShareVersion implements FeatureVersion {
+
+ // Version 0 does not enable share groups.
+ SV_0(0, MetadataVersion.MINIMUM_VERSION, Map.of()),
+
+ // Version 1 enables share groups (KIP-932).
+ // This is a preview in 4.1, and production-ready in 4.2.
+ SV_1(1, MetadataVersion.IBP_4_2_IV0, Map.of(MetadataVersion.FEATURE_NAME, MetadataVersion.IBP_4_1_IV1.featureLevel()));
Review Comment:
Should we add GroupVersion to its dependency too?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16894: Define feature to enable share groups [kafka]
Posted by "junrao (via GitHub)" <gi...@apache.org>.
junrao commented on code in PR #19293:
URL: https://github.com/apache/kafka/pull/19293#discussion_r2023177127
##########
server-common/src/main/java/org/apache/kafka/server/common/ShareVersion.java:
##########
@@ -0,0 +1,78 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.server.common;
+
+import java.util.Map;
+
+public enum ShareVersion implements FeatureVersion {
+
+ // Version 0 does not enable share groups.
+ SV_0(0, MetadataVersion.MINIMUM_VERSION, Map.of()),
+
+ // Version 1 enables share groups (KIP-932).
+ // This is a preview in 4.1, and production-ready in 4.2.
+ SV_1(1, MetadataVersion.IBP_4_2_IV0, Map.of(MetadataVersion.FEATURE_NAME, MetadataVersion.IBP_4_1_IV1.featureLevel()));
Review Comment:
But for SV to work, it needs at least some version of GV, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16894: Define feature to enable share groups [kafka]
Posted by "junrao (via GitHub)" <gi...@apache.org>.
junrao commented on code in PR #19293:
URL: https://github.com/apache/kafka/pull/19293#discussion_r2064970001
##########
server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java:
##########
@@ -112,7 +112,22 @@ public enum MetadataVersion {
//
// Enables ELR by default for new clusters (KIP-966).
- IBP_4_1_IV0(26, "4.1", "IV0", false);
+ IBP_4_1_IV0(26, "4.1", "IV0", false),
+
+ // Enables share groups. Note, share groups are for preview only in 4.1. (KIP-932).
+ IBP_4_1_IV1(27, "4.1", "IV1", false),
Review Comment:
@AndrewJSchofield : @jolshan and I were wondering why this particular MV was added. Does it depend on any records/RPCs controlled by MV?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16894: Define feature to enable share groups [kafka]
Posted by "AndrewJSchofield (via GitHub)" <gi...@apache.org>.
AndrewJSchofield commented on code in PR #19293:
URL: https://github.com/apache/kafka/pull/19293#discussion_r2065960167
##########
server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java:
##########
@@ -112,7 +112,22 @@ public enum MetadataVersion {
//
// Enables ELR by default for new clusters (KIP-966).
- IBP_4_1_IV0(26, "4.1", "IV0", false);
+ IBP_4_1_IV0(26, "4.1", "IV0", false),
+
+ // Enables share groups. Note, share groups are for preview only in 4.1. (KIP-932).
+ IBP_4_1_IV1(27, "4.1", "IV1", false),
Review Comment:
@junrao Let's call it ignorance. In principle, I think I could use `IBP_4_1_IV0`. I do note that there are several MV in the past which appear just to add a new RPC version such as `IBP_3_7_IV4`, which seems to fall into the same category. We have plenty of time to get this right before 4.1 chills.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16894: Define feature to enable share groups [kafka]
Posted by "AndrewJSchofield (via GitHub)" <gi...@apache.org>.
AndrewJSchofield commented on code in PR #19293:
URL: https://github.com/apache/kafka/pull/19293#discussion_r2066998073
##########
server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java:
##########
@@ -112,7 +112,22 @@ public enum MetadataVersion {
//
// Enables ELR by default for new clusters (KIP-966).
- IBP_4_1_IV0(26, "4.1", "IV0", false);
+ IBP_4_1_IV0(26, "4.1", "IV0", false),
+
+ // Enables share groups. Note, share groups are for preview only in 4.1. (KIP-932).
+ IBP_4_1_IV1(27, "4.1", "IV1", false),
Review Comment:
Yes. I will do this shortly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16894: Define feature to enable share groups [kafka]
Posted by "junrao (via GitHub)" <gi...@apache.org>.
junrao commented on code in PR #19293:
URL: https://github.com/apache/kafka/pull/19293#discussion_r2037984758
##########
core/src/test/scala/unit/kafka/server/AbstractApiVersionsRequestTest.scala:
##########
@@ -85,6 +85,9 @@ abstract class AbstractApiVersionsRequestTest(cluster: ClusterInstance) {
assertEquals(0, apiVersionsResponse.data().supportedFeatures().find(EligibleLeaderReplicasVersion.FEATURE_NAME).minVersion())
assertEquals(EligibleLeaderReplicasVersion.ELRV_1.featureLevel(), apiVersionsResponse.data().supportedFeatures().find(EligibleLeaderReplicasVersion.FEATURE_NAME).maxVersion())
+
+ assertEquals(0, apiVersionsResponse.data().supportedFeatures().find(ShareVersion.FEATURE_NAME).minVersion())
+ assertEquals(ShareVersion.SV_1.featureLevel(), apiVersionsResponse.data().supportedFeatures().find(ShareVersion.FEATURE_NAME).maxVersion())
Review Comment:
Hmm, this is only correct if unstable.feature.versions.enable is set to true. However, some of the callers like SaslApiVersionsRequestTest doesn't seem to set unstable.feature.versions.enable to true. So, how does this work?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16894: Define feature to enable share groups [kafka]
Posted by "AndrewJSchofield (via GitHub)" <gi...@apache.org>.
AndrewJSchofield commented on code in PR #19293:
URL: https://github.com/apache/kafka/pull/19293#discussion_r2038119258
##########
core/src/test/scala/unit/kafka/server/AbstractApiVersionsRequestTest.scala:
##########
@@ -85,6 +85,9 @@ abstract class AbstractApiVersionsRequestTest(cluster: ClusterInstance) {
assertEquals(0, apiVersionsResponse.data().supportedFeatures().find(EligibleLeaderReplicasVersion.FEATURE_NAME).minVersion())
assertEquals(EligibleLeaderReplicasVersion.ELRV_1.featureLevel(), apiVersionsResponse.data().supportedFeatures().find(EligibleLeaderReplicasVersion.FEATURE_NAME).maxVersion())
+
+ assertEquals(0, apiVersionsResponse.data().supportedFeatures().find(ShareVersion.FEATURE_NAME).minVersion())
+ assertEquals(ShareVersion.SV_1.featureLevel(), apiVersionsResponse.data().supportedFeatures().find(ShareVersion.FEATURE_NAME).maxVersion())
Review Comment:
The aim here is that `unstable.feature.versions.enable=true` should not be required to enable share groups in AK 4.1. They should be disabled on new brokers by default, but possible to enable using `kafka-features.sh upgrade`. I'm trying to do what was done for the ELR preview (https://github.com/apache/kafka/pull/17867 and then https://github.com/apache/kafka/pull/18954).
In AK 4.2, the metadata version will be `IBP_4_2_IV0`, so a new broker should get `share.version=1` by default.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16894: Define feature to enable share groups [kafka]
Posted by "AndrewJSchofield (via GitHub)" <gi...@apache.org>.
AndrewJSchofield commented on PR #19293:
URL: https://github.com/apache/kafka/pull/19293#issuecomment-2796604695
There is a final piece to this change. When `MetadataVersion.LATEST_PRODUCTION` is moved to `IBP_4_1_IV1` or later (as it will before AK 4.1 releases), we must change `ShareVersion.LATEST_PRODUCTION` to `SV_1`. We cannot do this while the latest production metadata version has not been changed or it is not possible to format a new broker without enabling unstable feature versions. This is due to the prerequisites of the `SV_1` feature version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org