You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "chirag-wadhwa5 (via GitHub)" <gi...@apache.org> on 2025/02/07 12:19:40 UTC

[PR] KAFKA-18756: Enabled share group configs for queues related system tests [kafka]

chirag-wadhwa5 opened a new pull request, #18829:
URL: https://github.com/apache/kafka/pull/18829

   This PR adds a way to set share group related configs for all the tests in share_consumer_test.py


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-18756: Enabled share group configs for queues related system tests [kafka]

Posted by "AndrewJSchofield (via GitHub)" <gi...@apache.org>.
AndrewJSchofield commented on code in PR #18829:
URL: https://github.com/apache/kafka/pull/18829#discussion_r1949060513


##########
tests/kafkatest/services/kafka/kafka.py:
##########
@@ -292,10 +292,18 @@ def __init__(self, context, num_nodes, zk, security_protocol=SecurityConfig.PLAI
                 use_new_coordinator = context.injected_args.get(arg_name)
             if use_new_coordinator is None:
                 use_new_coordinator = context.globals.get(arg_name)
+
+        # Set use_share_groups based on injected arguments.

Review Comment:
   Why is the handling of `use_share_groups` different than `use_transactions_v2`? It seems to me that this is different from both `use_new_coordinator` and `use_transactions_v2`, and I cannot see why it shouldn't just follow the simpler `use_transactions_v2` pattern.



##########
tests/kafkatest/services/kafka/kafka.py:
##########
@@ -778,6 +786,13 @@ def prop_file(self, node):
         for prop in self.per_node_server_prop_overrides.get(self.idx(node), []):
             override_configs[prop[0]] = prop[1]
 
+        if self.use_share_groups:
+            override_configs[config_property.SHARE_GROUP_ENABLE] = 'true'
+            override_configs[config_property.UNSTABLE_API_VERSIONS_ENABLE] = 'true'
+            override_configs[config_property.GROUP_COORDINATOR_REBALANCE_PROTOCOLS] = 'classic,consumer,share'
+            override_configs[config_property.SHARE_COORDINATOR_STATE_TOPIC_REPLICATION_FACTOR] = str(self.num_nodes)

Review Comment:
   I'd set the maximum to 3 here. Potentially someone could run a test with lots of nodes, and there's no need for a massive replication factor to match.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-18756: Enabled share group configs for queues related system tests [kafka]

Posted by "AndrewJSchofield (via GitHub)" <gi...@apache.org>.
AndrewJSchofield commented on code in PR #18829:
URL: https://github.com/apache/kafka/pull/18829#discussion_r1950538401


##########
tests/kafkatest/services/kafka/kafka.py:
##########
@@ -778,6 +790,13 @@ def prop_file(self, node):
         for prop in self.per_node_server_prop_overrides.get(self.idx(node), []):
             override_configs[prop[0]] = prop[1]
 
+        if self.use_share_groups is not None and self.use_share_groups is True:
+            override_configs[config_property.SHARE_GROUP_ENABLE] = str(self.use_share_groups)
+            override_configs[config_property.UNSTABLE_API_VERSIONS_ENABLE] = str(self.use_share_groups)
+            override_configs[config_property.GROUP_COORDINATOR_REBALANCE_PROTOCOLS] = 'classic,consumer,share'
+            override_configs[config_property.SHARE_COORDINATOR_STATE_TOPIC_REPLICATION_FACTOR] = str(min(self.num_nodes, 3))

Review Comment:
   I wonder why it is necessary to set the replication factor and min-isr for the share-state topic in this manner. I would have thought that the logic for setting values for the consumer-offsets and transaction-state topics would be the same as needed for the share-state topic. Please investigate more.
   
   Personally, I think the `use_share_groups` setting should just turn on the configs needed to turn on share groups.



##########
tests/kafkatest/services/kafka/kafka.py:
##########
@@ -206,7 +206,8 @@ def __init__(self, context, num_nodes, zk, security_protocol=SecurityConfig.PLAI
                  use_new_coordinator=None,
                  consumer_group_migration_policy=None,
                  dynamicRaftQuorum=False,
-                 use_transactions_v2=False
+                 use_transactions_v2=False,
+                 use_share_groups=None,

Review Comment:
   nit: Extraneous comma



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-18756: Enabled share group configs for queues related system tests [kafka]

Posted by "AndrewJSchofield (via GitHub)" <gi...@apache.org>.
AndrewJSchofield merged PR #18829:
URL: https://github.com/apache/kafka/pull/18829


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org