You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "dajac (via GitHub)" <gi...@apache.org> on 2025/01/06 13:21:27 UTC

[PR] KAFKA-18303; Update ShareCoordinator to use new record format [kafka]

dajac opened a new pull request, #18396:
URL: https://github.com/apache/kafka/pull/18396

   Following https://github.com/apache/kafka/pull/18261, this patch updates the Share Coordinator to use the new record format.
   
   ### Committer Checklist (excluded from commit message)
   - [ ] Verify design and implementation 
   - [ ] Verify test coverage and CI build status
   - [ ] Verify documentation (including upgrade notes)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-18303; Update ShareCoordinator to use new record format [kafka]

Posted by "dajac (via GitHub)" <gi...@apache.org>.
dajac merged PR #18396:
URL: https://github.com/apache/kafka/pull/18396


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-18303; Update ShareCoordinator to use new record format [kafka]

Posted by "dajac (via GitHub)" <gi...@apache.org>.
dajac commented on code in PR #18396:
URL: https://github.com/apache/kafka/pull/18396#discussion_r1904148774


##########
share-coordinator/src/main/java/org/apache/kafka/coordinator/share/ShareCoordinatorRecordSerde.java:
##########
@@ -29,9 +29,9 @@ public class ShareCoordinatorRecordSerde extends CoordinatorRecordSerde {
     @Override
     protected ApiMessage apiMessageKeyFor(short recordVersion) {
         switch (recordVersion) {
-            case ShareCoordinator.SHARE_SNAPSHOT_RECORD_KEY_VERSION:
+            case 0:

Review Comment:
   This is a temporary step backward. I will reword it in https://issues.apache.org/jira/browse/KAFKA-18308.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-18303; Update ShareCoordinator to use new record format [kafka]

Posted by "dajac (via GitHub)" <gi...@apache.org>.
dajac commented on code in PR #18396:
URL: https://github.com/apache/kafka/pull/18396#discussion_r1905041754


##########
share-coordinator/src/main/java/org/apache/kafka/coordinator/share/ShareCoordinatorShard.java:
##########
@@ -206,15 +208,19 @@ public void replay(long offset, long producerId, short producerEpoch, Coordinato
         ApiMessageAndVersion key = record.key();
         ApiMessageAndVersion value = record.value();
 
-        switch (key.version()) {
-            case ShareCoordinator.SHARE_SNAPSHOT_RECORD_KEY_VERSION: // ShareSnapshot
-                handleShareSnapshot((ShareSnapshotKey) key.message(), (ShareSnapshotValue) messageOrNull(value), offset);
-                break;
-            case ShareCoordinator.SHARE_UPDATE_RECORD_KEY_VERSION: // ShareUpdate
-                handleShareUpdate((ShareUpdateKey) key.message(), (ShareUpdateValue) messageOrNull(value));
-                break;
-            default:
-                // Noop
+        try {
+            switch (CoordinatorRecordType.fromId(key.version())) {
+                case SHARE_SNAPSHOT:
+                    handleShareSnapshot((ShareSnapshotKey) key.message(), (ShareSnapshotValue) messageOrNull(value), offset);
+                    break;
+                case SHARE_UPDATE:
+                    handleShareUpdate((ShareUpdateKey) key.message(), (ShareUpdateValue) messageOrNull(value));
+                    break;
+                default:
+                    // Noop

Review Comment:
   There is a separate [PR](https://github.com/apache/kafka/pull/18371) for doing this.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-18303; Update ShareCoordinator to use new record format [kafka]

Posted by "dajac (via GitHub)" <gi...@apache.org>.
dajac commented on code in PR #18396:
URL: https://github.com/apache/kafka/pull/18396#discussion_r1904146020


##########
share-coordinator/src/main/java/org/apache/kafka/coordinator/share/ShareCoordinator.java:
##########
@@ -32,11 +32,6 @@
 import java.util.function.IntSupplier;
 
 public interface ShareCoordinator {
-    short SHARE_SNAPSHOT_RECORD_KEY_VERSION = 0;
-    short SHARE_SNAPSHOT_RECORD_VALUE_VERSION = 0;
-    short SHARE_UPDATE_RECORD_KEY_VERSION = 1;
-    short SHARE_UPDATE_RECORD_VALUE_VERSION = 1;

Review Comment:
   This is a bug. The version of the update value record must be 0.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-18303; Update ShareCoordinator to use new record format [kafka]

Posted by "dajac (via GitHub)" <gi...@apache.org>.
dajac commented on code in PR #18396:
URL: https://github.com/apache/kafka/pull/18396#discussion_r1904147998


##########
share-coordinator/src/main/java/org/apache/kafka/coordinator/share/ShareCoordinatorRecordHelpers.java:
##########
@@ -46,7 +47,7 @@ public static CoordinatorRecord newShareSnapshotRecord(String groupId, Uuid topi
                         .setDeliveryCount(batch.deliveryCount())
                         .setDeliveryState(batch.deliveryState()))
                     .collect(Collectors.toList())),
-                ShareCoordinator.SHARE_SNAPSHOT_RECORD_VALUE_VERSION)
+                (short) 0)

Review Comment:
   It is better to hardcode the version here in order to avoid using the wrong ones as it was the case.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-18303; Update ShareCoordinator to use new record format [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #18396:
URL: https://github.com/apache/kafka/pull/18396#discussion_r1905069205


##########
share-coordinator/src/main/java/org/apache/kafka/coordinator/share/ShareCoordinator.java:
##########
@@ -32,11 +32,6 @@
 import java.util.function.IntSupplier;
 
 public interface ShareCoordinator {
-    short SHARE_SNAPSHOT_RECORD_KEY_VERSION = 0;
-    short SHARE_SNAPSHOT_RECORD_VALUE_VERSION = 0;
-    short SHARE_UPDATE_RECORD_KEY_VERSION = 1;
-    short SHARE_UPDATE_RECORD_VALUE_VERSION = 1;

Review Comment:
   Pardon me, why not fix it for 4.0? I assume it would be an issue to 4.1 in reading the records having incorrect version.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-18303; Update ShareCoordinator to use new record format [kafka]

Posted by "dajac (via GitHub)" <gi...@apache.org>.
dajac commented on code in PR #18396:
URL: https://github.com/apache/kafka/pull/18396#discussion_r1905042395


##########
share-coordinator/src/main/java/org/apache/kafka/coordinator/share/ShareCoordinator.java:
##########
@@ -32,11 +32,6 @@
 import java.util.function.IntSupplier;
 
 public interface ShareCoordinator {
-    short SHARE_SNAPSHOT_RECORD_KEY_VERSION = 0;
-    short SHARE_SNAPSHOT_RECORD_VALUE_VERSION = 0;
-    short SHARE_UPDATE_RECORD_KEY_VERSION = 1;
-    short SHARE_UPDATE_RECORD_VALUE_VERSION = 1;

Review Comment:
   @AndrewJSchofield Is this something that we want to fix in 4.0?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-18303; Update ShareCoordinator to use new record format [kafka]

Posted by "dajac (via GitHub)" <gi...@apache.org>.
dajac commented on code in PR #18396:
URL: https://github.com/apache/kafka/pull/18396#discussion_r1904146020


##########
share-coordinator/src/main/java/org/apache/kafka/coordinator/share/ShareCoordinator.java:
##########
@@ -32,11 +32,6 @@
 import java.util.function.IntSupplier;
 
 public interface ShareCoordinator {
-    short SHARE_SNAPSHOT_RECORD_KEY_VERSION = 0;
-    short SHARE_SNAPSHOT_RECORD_VALUE_VERSION = 0;
-    short SHARE_UPDATE_RECORD_KEY_VERSION = 1;
-    short SHARE_UPDATE_RECORD_VALUE_VERSION = 1;

Review Comment:
   This is a bug. The version of the update record must be 0.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-18303; Update ShareCoordinator to use new record format [kafka]

Posted by "dajac (via GitHub)" <gi...@apache.org>.
dajac commented on code in PR #18396:
URL: https://github.com/apache/kafka/pull/18396#discussion_r1904148774


##########
share-coordinator/src/main/java/org/apache/kafka/coordinator/share/ShareCoordinatorRecordSerde.java:
##########
@@ -29,9 +29,9 @@ public class ShareCoordinatorRecordSerde extends CoordinatorRecordSerde {
     @Override
     protected ApiMessage apiMessageKeyFor(short recordVersion) {
         switch (recordVersion) {
-            case ShareCoordinator.SHARE_SNAPSHOT_RECORD_KEY_VERSION:
+            case 0:

Review Comment:
   This is a temporary step backward. I will rework it in https://issues.apache.org/jira/browse/KAFKA-18308.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-18303; Update ShareCoordinator to use new record format [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #18396:
URL: https://github.com/apache/kafka/pull/18396#discussion_r1904634091


##########
share-coordinator/src/main/java/org/apache/kafka/coordinator/share/ShareCoordinatorShard.java:
##########
@@ -206,15 +208,19 @@ public void replay(long offset, long producerId, short producerEpoch, Coordinato
         ApiMessageAndVersion key = record.key();
         ApiMessageAndVersion value = record.value();
 
-        switch (key.version()) {
-            case ShareCoordinator.SHARE_SNAPSHOT_RECORD_KEY_VERSION: // ShareSnapshot
-                handleShareSnapshot((ShareSnapshotKey) key.message(), (ShareSnapshotValue) messageOrNull(value), offset);
-                break;
-            case ShareCoordinator.SHARE_UPDATE_RECORD_KEY_VERSION: // ShareUpdate
-                handleShareUpdate((ShareUpdateKey) key.message(), (ShareUpdateValue) messageOrNull(value));
-                break;
-            default:
-                // Noop
+        try {
+            switch (CoordinatorRecordType.fromId(key.version())) {
+                case SHARE_SNAPSHOT:
+                    handleShareSnapshot((ShareSnapshotKey) key.message(), (ShareSnapshotValue) messageOrNull(value), offset);
+                    break;
+                case SHARE_UPDATE:
+                    handleShareUpdate((ShareUpdateKey) key.message(), (ShareUpdateValue) messageOrNull(value));
+                    break;
+                default:
+                    // Noop

Review Comment:
   maybe we can use java 17 enhanced switch to eliminate unnecessary default branch.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-18303; Update ShareCoordinator to use new record format [kafka]

Posted by "AndrewJSchofield (via GitHub)" <gi...@apache.org>.
AndrewJSchofield commented on code in PR #18396:
URL: https://github.com/apache/kafka/pull/18396#discussion_r1905846691


##########
share-coordinator/src/main/java/org/apache/kafka/coordinator/share/ShareCoordinator.java:
##########
@@ -32,11 +32,6 @@
 import java.util.function.IntSupplier;
 
 public interface ShareCoordinator {
-    short SHARE_SNAPSHOT_RECORD_KEY_VERSION = 0;
-    short SHARE_SNAPSHOT_RECORD_VALUE_VERSION = 0;
-    short SHARE_UPDATE_RECORD_KEY_VERSION = 1;
-    short SHARE_UPDATE_RECORD_VALUE_VERSION = 1;

Review Comment:
   We don't support migration of the share-group state topic data from 4.0 to 4.1, so not strictly necessary. But really, it is not ideal. Probably should fix it. I'll open an issue.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-18303; Update ShareCoordinator to use new record format [kafka]

Posted by "dajac (via GitHub)" <gi...@apache.org>.
dajac commented on code in PR #18396:
URL: https://github.com/apache/kafka/pull/18396#discussion_r1904147168


##########
share-coordinator/src/main/java/org/apache/kafka/coordinator/share/ShareCoordinatorShard.java:
##########
@@ -206,15 +208,19 @@ public void replay(long offset, long producerId, short producerEpoch, Coordinato
         ApiMessageAndVersion key = record.key();
         ApiMessageAndVersion value = record.value();
 
-        switch (key.version()) {
-            case ShareCoordinator.SHARE_SNAPSHOT_RECORD_KEY_VERSION: // ShareSnapshot
-                handleShareSnapshot((ShareSnapshotKey) key.message(), (ShareSnapshotValue) messageOrNull(value), offset);
-                break;
-            case ShareCoordinator.SHARE_UPDATE_RECORD_KEY_VERSION: // ShareUpdate
-                handleShareUpdate((ShareUpdateKey) key.message(), (ShareUpdateValue) messageOrNull(value));
-                break;
-            default:
-                // Noop
+        try {
+            switch (CoordinatorRecordType.fromId(key.version())) {
+                case SHARE_SNAPSHOT:
+                    handleShareSnapshot((ShareSnapshotKey) key.message(), (ShareSnapshotValue) messageOrNull(value), offset);
+                    break;
+                case SHARE_UPDATE:
+                    handleShareUpdate((ShareUpdateKey) key.message(), (ShareUpdateValue) messageOrNull(value));
+                    break;
+                default:
+                    // Noop
+            }
+        } catch (UnsupportedVersionException ex) {
+            // Ignore

Review Comment:
   I ignore unknown records to follow the current implementation.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org