You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "aliehsaeedii (via GitHub)" <gi...@apache.org> on 2024/12/13 12:58:04 UTC

[PR] Implement kafka-streams-groups.sh --list [kafka]

aliehsaeedii opened a new pull request, #18167:
URL: https://github.com/apache/kafka/pull/18167

   Implement the core of `kafka-streams-groups.sh`
   Implement `--list` and its options: (only `--state`)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Implement kafka-streams-groups.sh --list [kafka]

Posted by "lucasbru (via GitHub)" <gi...@apache.org>.
lucasbru commented on code in PR #18167:
URL: https://github.com/apache/kafka/pull/18167#discussion_r1883968621


##########
tools/src/main/java/org/apache/kafka/tools/streams/StreamsGroupCommandOptions.java:
##########
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.tools.streams;
+
+import org.apache.kafka.server.util.CommandDefaultOptions;
+import org.apache.kafka.server.util.CommandLineUtils;
+
+import joptsimple.OptionSpec;
+
+
+public class StreamsGroupCommandOptions extends CommandDefaultOptions {
+    public static final String BOOTSTRAP_SERVER_DOC = "REQUIRED: The server(s) to connect to.";
+    public static final String LIST_DOC = "List all share groups.";

Review Comment:
   "List all streams groups"



##########
tools/src/main/java/org/apache/kafka/tools/streams/StreamsGroupCommandOptions.java:
##########
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.tools.streams;
+
+import org.apache.kafka.server.util.CommandDefaultOptions;
+import org.apache.kafka.server.util.CommandLineUtils;
+
+import joptsimple.OptionSpec;
+
+
+public class StreamsGroupCommandOptions extends CommandDefaultOptions {
+    public static final String BOOTSTRAP_SERVER_DOC = "REQUIRED: The server(s) to connect to.";
+    public static final String LIST_DOC = "List all share groups.";
+    public static final String NL = System.lineSeparator();
+    public static final String TIMEOUT_MS_DOC = "The timeout that can be set for some use cases. For example, it can be used when describing the group " +
+        "to specify the maximum amount of time in milliseconds to wait before the group stabilizes.";
+    public static final String COMMAND_CONFIG_DOC = "Property file containing configs to be passed to Admin Client.";
+    public static final String DRY_RUN_DOC = "Only show results without executing changes on share groups. Supported operations: reset-offsets.";
+    public static final String EXECUTE_DOC = "Execute operation. Supported operations: reset-offsets.";

Review Comment:
   remove



##########
tools/src/main/java/org/apache/kafka/tools/streams/StreamsGroupCommandOptions.java:
##########
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.tools.streams;
+
+import org.apache.kafka.server.util.CommandDefaultOptions;
+import org.apache.kafka.server.util.CommandLineUtils;
+
+import joptsimple.OptionSpec;
+
+

Review Comment:
   double new line



##########
tools/src/main/java/org/apache/kafka/tools/streams/StreamsGroupCommandOptions.java:
##########
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.tools.streams;
+
+import org.apache.kafka.server.util.CommandDefaultOptions;
+import org.apache.kafka.server.util.CommandLineUtils;
+
+import joptsimple.OptionSpec;
+
+
+public class StreamsGroupCommandOptions extends CommandDefaultOptions {
+    public static final String BOOTSTRAP_SERVER_DOC = "REQUIRED: The server(s) to connect to.";
+    public static final String LIST_DOC = "List all share groups.";
+    public static final String NL = System.lineSeparator();
+    public static final String TIMEOUT_MS_DOC = "The timeout that can be set for some use cases. For example, it can be used when describing the group " +
+        "to specify the maximum amount of time in milliseconds to wait before the group stabilizes.";
+    public static final String COMMAND_CONFIG_DOC = "Property file containing configs to be passed to Admin Client.";
+    public static final String DRY_RUN_DOC = "Only show results without executing changes on share groups. Supported operations: reset-offsets.";

Review Comment:
   We do not need --dry-run yet, as it's only supported by reset-offsets. Also, this shouldn't mention share groups.



##########
tools/src/main/java/org/apache/kafka/tools/streams/StreamsGroupCommandOptions.java:
##########
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.tools.streams;
+
+import org.apache.kafka.server.util.CommandDefaultOptions;
+import org.apache.kafka.server.util.CommandLineUtils;
+
+import joptsimple.OptionSpec;
+
+
+public class StreamsGroupCommandOptions extends CommandDefaultOptions {
+    public static final String BOOTSTRAP_SERVER_DOC = "REQUIRED: The server(s) to connect to.";
+    public static final String LIST_DOC = "List all share groups.";
+    public static final String NL = System.lineSeparator();
+    public static final String TIMEOUT_MS_DOC = "The timeout that can be set for some use cases. For example, it can be used when describing the group " +
+        "to specify the maximum amount of time in milliseconds to wait before the group stabilizes.";
+    public static final String COMMAND_CONFIG_DOC = "Property file containing configs to be passed to Admin Client.";
+    public static final String DRY_RUN_DOC = "Only show results without executing changes on share groups. Supported operations: reset-offsets.";
+    public static final String EXECUTE_DOC = "Execute operation. Supported operations: reset-offsets.";
+    public static final String STATE_DOC = "When specified with '--describe', includes the state of the group." + NL +
+        "When specified with '--list', it displays the state of all groups. It can also be used to list groups with specific states. " +
+        "Valid values are Empty, Stable and Dead.";

Review Comment:
   There are more possible states than just `Empty, Stable and Dead`.



##########
tools/src/main/java/org/apache/kafka/tools/streams/StreamsGroupCommandOptions.java:
##########
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.tools.streams;
+
+import org.apache.kafka.server.util.CommandDefaultOptions;
+import org.apache.kafka.server.util.CommandLineUtils;
+
+import joptsimple.OptionSpec;
+
+
+public class StreamsGroupCommandOptions extends CommandDefaultOptions {
+    public static final String BOOTSTRAP_SERVER_DOC = "REQUIRED: The server(s) to connect to.";
+    public static final String LIST_DOC = "List all share groups.";
+    public static final String NL = System.lineSeparator();
+    public static final String TIMEOUT_MS_DOC = "The timeout that can be set for some use cases. For example, it can be used when describing the group " +
+        "to specify the maximum amount of time in milliseconds to wait before the group stabilizes.";
+    public static final String COMMAND_CONFIG_DOC = "Property file containing configs to be passed to Admin Client.";
+    public static final String DRY_RUN_DOC = "Only show results without executing changes on share groups. Supported operations: reset-offsets.";
+    public static final String EXECUTE_DOC = "Execute operation. Supported operations: reset-offsets.";
+    public static final String STATE_DOC = "When specified with '--describe', includes the state of the group." + NL +
+        "When specified with '--list', it displays the state of all groups. It can also be used to list groups with specific states. " +
+        "Valid values are Empty, Stable and Dead.";
+
+    public final OptionSpec<String> bootstrapServerOpt;
+    public final OptionSpec<Void> listOpt;
+    public final OptionSpec<Long> timeoutMsOpt;
+    public final OptionSpec<String> commandConfigOpt;
+    public final OptionSpec<Void> dryRunOpt;
+    public final OptionSpec<Void> executeOpt;
+    public final OptionSpec<String> stateOpt;
+
+
+    public StreamsGroupCommandOptions(String[] args) {
+        super(args);
+
+        bootstrapServerOpt = parser.accepts("bootstrap-server", BOOTSTRAP_SERVER_DOC)
+            .withRequiredArg()
+            .describedAs("server to connect to")
+            .ofType(String.class);
+        listOpt = parser.accepts("list", LIST_DOC);
+        timeoutMsOpt = parser.accepts("timeout", TIMEOUT_MS_DOC)
+            .withRequiredArg()
+            .describedAs("timeout (ms)")
+            .ofType(Long.class)
+            .defaultsTo(5000L);
+        commandConfigOpt = parser.accepts("command-config", COMMAND_CONFIG_DOC)
+            .withRequiredArg()
+            .describedAs("command config property file")
+            .ofType(String.class);
+        dryRunOpt = parser.accepts("dry-run", DRY_RUN_DOC);
+        executeOpt = parser.accepts("execute", EXECUTE_DOC);
+        stateOpt = parser.accepts("state", STATE_DOC)
+            .availableIf(listOpt)
+            .withOptionalArg()
+            .ofType(String.class);
+
+        options = parser.parse(args);
+    }
+
+    public void checkArgs() {
+        CommandLineUtils.maybePrintHelpOrVersion(this, "This tool helps to list, describe, reset and delete share groups.");

Review Comment:
   Mentions share groups.
   reset and delete and describe are not probided yet



##########
tools/src/main/java/org/apache/kafka/tools/streams/StreamsGroupCommandOptions.java:
##########
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.tools.streams;
+
+import org.apache.kafka.server.util.CommandDefaultOptions;
+import org.apache.kafka.server.util.CommandLineUtils;
+
+import joptsimple.OptionSpec;
+
+
+public class StreamsGroupCommandOptions extends CommandDefaultOptions {
+    public static final String BOOTSTRAP_SERVER_DOC = "REQUIRED: The server(s) to connect to.";
+    public static final String LIST_DOC = "List all share groups.";
+    public static final String NL = System.lineSeparator();
+    public static final String TIMEOUT_MS_DOC = "The timeout that can be set for some use cases. For example, it can be used when describing the group " +
+        "to specify the maximum amount of time in milliseconds to wait before the group stabilizes.";
+    public static final String COMMAND_CONFIG_DOC = "Property file containing configs to be passed to Admin Client.";
+    public static final String DRY_RUN_DOC = "Only show results without executing changes on share groups. Supported operations: reset-offsets.";
+    public static final String EXECUTE_DOC = "Execute operation. Supported operations: reset-offsets.";
+    public static final String STATE_DOC = "When specified with '--describe', includes the state of the group." + NL +

Review Comment:
   Seems like this shouldn't mention `--describe` yet.



##########
tools/src/test/java/org/apache/kafka/tools/streams/StreamsGroupCommandTest.java:
##########
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.tools.streams;
+
+import org.apache.kafka.clients.admin.Admin;
+import org.apache.kafka.clients.admin.GroupListing;
+import org.apache.kafka.clients.admin.KafkaAdminClient;
+import org.apache.kafka.clients.admin.ListGroupsOptions;
+import org.apache.kafka.clients.admin.ListGroupsResult;
+import org.apache.kafka.clients.admin.MockAdminClient;
+import org.apache.kafka.common.GroupState;
+import org.apache.kafka.common.GroupType;
+import org.apache.kafka.common.KafkaFuture;
+import org.apache.kafka.test.TestUtils;
+
+import org.junit.jupiter.api.Test;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.Set;
+
+import joptsimple.OptionException;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+public class StreamsGroupCommandTest {

Review Comment:
   We'll also want to have an integration test for the tool as well, similar to `ListConsumerGroupTest`, which actually runs a cluster. Maybe we could rename this to `StreamsGroupCommandUnitTest` and add a `StreamsGroupCommandTest` that uses a similar pattern as `ListConsumerGroupTest`?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Implement kafka-streams-groups.sh --list [kafka]

Posted by "aliehsaeedii (via GitHub)" <gi...@apache.org>.
aliehsaeedii commented on PR #18167:
URL: https://github.com/apache/kafka/pull/18167#issuecomment-2541845011

   > Thanks for the PR! I left some comments
   
   Thanks, @lucasbru. I addressed the nits. I think we can merge this PR and meanwhile I work on the integration test.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Implement kafka-streams-groups.sh --list [kafka]

Posted by "aliehsaeedii (via GitHub)" <gi...@apache.org>.
aliehsaeedii commented on code in PR #18167:
URL: https://github.com/apache/kafka/pull/18167#discussion_r1883902706


##########
tools/src/main/java/org/apache/kafka/tools/streams/StreamsGroupCommand.java:
##########
@@ -0,0 +1,170 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.tools.streams;
+
+import org.apache.kafka.clients.CommonClientConfigs;
+import org.apache.kafka.clients.admin.Admin;
+import org.apache.kafka.clients.admin.GroupListing;
+import org.apache.kafka.clients.admin.ListGroupsOptions;
+import org.apache.kafka.clients.admin.ListGroupsResult;
+import org.apache.kafka.common.GroupState;
+import org.apache.kafka.common.GroupType;
+import org.apache.kafka.common.utils.Utils;
+import org.apache.kafka.server.util.CommandLineUtils;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Properties;
+import java.util.Set;
+import java.util.concurrent.ExecutionException;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+import joptsimple.OptionException;
+
+public class StreamsGroupCommand {
+
+    public static void main(String[] args) {
+        StreamsGroupCommandOptions opts = new StreamsGroupCommandOptions(args);
+        try {
+            opts.checkArgs();
+
+            CommandLineUtils.maybePrintHelpOrVersion(opts, "This tool helps to list all streams groups.");
+
+            // should have exactly one action
+            long actions = Stream.of(opts.listOpt).filter(opts.options::has).count();
+            if (actions != 1)
+                CommandLineUtils.printUsageAndExit(opts.parser, "Command must include exactly one action: --list.");

Review Comment:
   The same here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Implement kafka-streams-groups.sh --list [kafka]

Posted by "lucasbru (via GitHub)" <gi...@apache.org>.
lucasbru commented on code in PR #18167:
URL: https://github.com/apache/kafka/pull/18167#discussion_r1886300914


##########
tools/src/test/java/org/apache/kafka/tools/streams/StreamsGroupCommandUnitTest.java:
##########
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.tools.streams;
+
+import org.apache.kafka.clients.admin.Admin;
+import org.apache.kafka.clients.admin.GroupListing;
+import org.apache.kafka.clients.admin.KafkaAdminClient;
+import org.apache.kafka.clients.admin.ListGroupsOptions;
+import org.apache.kafka.clients.admin.ListGroupsResult;
+import org.apache.kafka.clients.admin.MockAdminClient;
+import org.apache.kafka.common.GroupState;
+import org.apache.kafka.common.GroupType;
+import org.apache.kafka.common.KafkaFuture;
+import org.apache.kafka.test.TestUtils;
+
+import org.junit.jupiter.api.Test;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.Set;
+
+import joptsimple.OptionException;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+public class StreamsGroupCommandUnitTest {
+
+    @Test
+    public void testListStreamsGroups() throws Exception {
+        String firstGroup = "first-group";
+        String secondGroup = "second-group";
+        String bootstrapServer = "localhost:9092";
+
+        String[] cgcArgs = new String[]{"--bootstrap-server", bootstrapServer, "--list"};
+        Admin adminClient = mock(KafkaAdminClient.class);
+        ListGroupsResult result = mock(ListGroupsResult.class);
+        when(result.all()).thenReturn(KafkaFuture.completedFuture(Arrays.asList(
+                new GroupListing(firstGroup, Optional.of(GroupType.STREAMS), "streams", Optional.of(GroupState.STABLE)),
+                new GroupListing(secondGroup, Optional.of(GroupType.STREAMS), "streams", Optional.of(GroupState.EMPTY))
+        )));
+        when(adminClient.listGroups(any(ListGroupsOptions.class))).thenReturn(result);
+        StreamsGroupCommand.StreamsGroupService service = getStreamsGroupService(cgcArgs, adminClient);
+        Set<String> expectedGroups = new HashSet<>(Arrays.asList(firstGroup, secondGroup));
+
+        final Set[] foundGroups = new Set[]{Collections.emptySet()};
+        TestUtils.waitForCondition(() -> {
+            foundGroups[0] = new HashSet<>(service.listStreamsGroups());
+            return Objects.equals(expectedGroups, foundGroups[0]);
+        }, "Expected --list to show groups " + expectedGroups + ", but found " + foundGroups[0] + ".");
+        service.close();
+    }
+
+    @Test
+    public void testListWithUnrecognizedOption() {
+        String bootstrapServer = "localhost:9092";
+        String[] cgcArgs = new String[]{"--frivolous-nonsense", "--bootstrap-server", bootstrapServer, "--list"};
+        assertThrows(OptionException.class, () -> getStreamsGroupService(cgcArgs, new MockAdminClient()));
+    }
+
+    @Test
+    public void testListStreamsGroupsWithStates() throws Exception {
+        String firstGroup = "first-group";
+        String secondGroup = "second-group";
+        String bootstrapServer = "localhost:9092";
+
+        String[] cgcArgs = new String[]{"--bootstrap-server", bootstrapServer, "--list", "--state"};
+        Admin adminClient = mock(KafkaAdminClient.class);
+        ListGroupsResult resultWithAllStates = mock(ListGroupsResult.class);
+        when(resultWithAllStates.all()).thenReturn(KafkaFuture.completedFuture(Arrays.asList(
+                new GroupListing(firstGroup, Optional.of(GroupType.STREAMS), "streams", Optional.of(GroupState.STABLE)),
+                new GroupListing(secondGroup, Optional.of(GroupType.STREAMS), "streams", Optional.of(GroupState.EMPTY))
+        )));
+        when(adminClient.listGroups(any(ListGroupsOptions.class))).thenReturn(resultWithAllStates);
+        StreamsGroupCommand.StreamsGroupService service = getStreamsGroupService(cgcArgs, adminClient);
+        Set<GroupListing> expectedListing = new HashSet<>(Arrays.asList(
+                new GroupListing(firstGroup, Optional.of(GroupType.STREAMS), "streams", Optional.of(GroupState.STABLE)),
+                new GroupListing(secondGroup, Optional.of(GroupType.STREAMS), "streams", Optional.of(GroupState.EMPTY))));
+
+        final Set[] foundListing = new Set[]{Collections.emptySet()};
+        TestUtils.waitForCondition(() -> {
+            foundListing[0] = new HashSet<>(service.listStreamsGroupsInStates(new HashSet<>(Arrays.asList(GroupState.values()))));
+            return Objects.equals(expectedListing, foundListing[0]);
+        }, "Expected to show groups " + expectedListing + ", but found " + foundListing[0]);
+
+        ListGroupsResult resultWithStableState = mock(ListGroupsResult.class);
+        when(resultWithStableState.all()).thenReturn(KafkaFuture.completedFuture(Collections.singletonList(
+                new GroupListing(firstGroup, Optional.of(GroupType.STREAMS), "streams", Optional.of(GroupState.STABLE))
+        )));
+        when(adminClient.listGroups(any(ListGroupsOptions.class))).thenReturn(resultWithStableState);
+        Set<GroupListing> expectedListingStable = Collections.singleton(
+                new GroupListing(firstGroup, Optional.of(GroupType.STREAMS), "streams", Optional.of(GroupState.STABLE)));
+
+        foundListing[0] = Collections.emptySet();
+
+        TestUtils.waitForCondition(() -> {
+            foundListing[0] = new HashSet<>(service.listStreamsGroupsInStates(Collections.singleton(GroupState.STABLE)));
+            return Objects.equals(expectedListingStable, foundListing[0]);
+        }, "Expected to show groups " + expectedListingStable + ", but found " + foundListing[0]);
+        service.close();
+    }
+
+    @Test
+    public void testGroupStatesFromString() {
+        Set<GroupState> result = StreamsGroupCommand.groupStatesFromString("empty");
+        assertEquals(new HashSet<>(Collections.singletonList(GroupState.EMPTY)), result);
+        result = StreamsGroupCommand.groupStatesFromString("EMPTY");
+        assertEquals(new HashSet<>(Collections.singletonList(GroupState.EMPTY)), result);
+
+        result = StreamsGroupCommand.groupStatesFromString("notready");
+        assertEquals(new HashSet<>(Collections.singletonList(GroupState.NOT_READY)), result);
+        result = StreamsGroupCommand.groupStatesFromString("notReady");
+        assertEquals(new HashSet<>(Collections.singletonList(GroupState.NOT_READY)), result);
+
+        result = StreamsGroupCommand.groupStatesFromString("assigning");
+        assertEquals(new HashSet<>(Collections.singletonList(GroupState.ASSIGNING)), result);
+        result = StreamsGroupCommand.groupStatesFromString("ASSIGNING");
+        assertEquals(new HashSet<>(Collections.singletonList(GroupState.ASSIGNING)), result);
+
+        result = StreamsGroupCommand.groupStatesFromString("RECONCILING");
+        assertEquals(new HashSet<>(Collections.singletonList(GroupState.RECONCILING)), result);
+        result = StreamsGroupCommand.groupStatesFromString("reconCILING");
+        assertEquals(new HashSet<>(Collections.singletonList(GroupState.RECONCILING)), result);
+
+        result = StreamsGroupCommand.groupStatesFromString("STABLE");
+        assertEquals(new HashSet<>(Collections.singletonList(GroupState.STABLE)), result);
+        result = StreamsGroupCommand.groupStatesFromString("stable");
+        assertEquals(new HashSet<>(Collections.singletonList(GroupState.STABLE)), result);
+
+        result = StreamsGroupCommand.groupStatesFromString("DEAD");
+        assertEquals(new HashSet<>(Collections.singletonList(GroupState.DEAD)), result);
+        result = StreamsGroupCommand.groupStatesFromString("dead");
+        assertEquals(new HashSet<>(Collections.singletonList(GroupState.DEAD)), result);
+
+
+        result = StreamsGroupCommand.groupStatesFromString("PREPARINGREBALANCE");
+        assertEquals(new HashSet<>(Collections.singletonList(GroupState.PREPARING_REBALANCE)), result);

Review Comment:
   @aliehsaeedii as per discussion, can you please remove preparing_rebalance and completing rebalance from the possible states in `GroupState` for streams, and this test? Thanks! 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Implement kafka-streams-groups.sh --list [kafka]

Posted by "lucasbru (via GitHub)" <gi...@apache.org>.
lucasbru merged PR #18167:
URL: https://github.com/apache/kafka/pull/18167


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Implement kafka-streams-groups.sh --list [kafka]

Posted by "aliehsaeedii (via GitHub)" <gi...@apache.org>.
aliehsaeedii commented on code in PR #18167:
URL: https://github.com/apache/kafka/pull/18167#discussion_r1883902355


##########
tools/src/main/java/org/apache/kafka/tools/streams/StreamsGroupCommand.java:
##########
@@ -0,0 +1,170 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.tools.streams;
+
+import org.apache.kafka.clients.CommonClientConfigs;
+import org.apache.kafka.clients.admin.Admin;
+import org.apache.kafka.clients.admin.GroupListing;
+import org.apache.kafka.clients.admin.ListGroupsOptions;
+import org.apache.kafka.clients.admin.ListGroupsResult;
+import org.apache.kafka.common.GroupState;
+import org.apache.kafka.common.GroupType;
+import org.apache.kafka.common.utils.Utils;
+import org.apache.kafka.server.util.CommandLineUtils;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Properties;
+import java.util.Set;
+import java.util.concurrent.ExecutionException;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+import joptsimple.OptionException;
+
+public class StreamsGroupCommand {
+
+    public static void main(String[] args) {
+        StreamsGroupCommandOptions opts = new StreamsGroupCommandOptions(args);
+        try {
+            opts.checkArgs();
+
+            CommandLineUtils.maybePrintHelpOrVersion(opts, "This tool helps to list all streams groups.");

Review Comment:
   The message should change as new command are added.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Implement kafka-streams-groups.sh --list [kafka]

Posted by "lucasbru (via GitHub)" <gi...@apache.org>.
lucasbru commented on code in PR #18167:
URL: https://github.com/apache/kafka/pull/18167#discussion_r1886300914


##########
tools/src/test/java/org/apache/kafka/tools/streams/StreamsGroupCommandUnitTest.java:
##########
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.tools.streams;
+
+import org.apache.kafka.clients.admin.Admin;
+import org.apache.kafka.clients.admin.GroupListing;
+import org.apache.kafka.clients.admin.KafkaAdminClient;
+import org.apache.kafka.clients.admin.ListGroupsOptions;
+import org.apache.kafka.clients.admin.ListGroupsResult;
+import org.apache.kafka.clients.admin.MockAdminClient;
+import org.apache.kafka.common.GroupState;
+import org.apache.kafka.common.GroupType;
+import org.apache.kafka.common.KafkaFuture;
+import org.apache.kafka.test.TestUtils;
+
+import org.junit.jupiter.api.Test;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.Set;
+
+import joptsimple.OptionException;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+public class StreamsGroupCommandUnitTest {
+
+    @Test
+    public void testListStreamsGroups() throws Exception {
+        String firstGroup = "first-group";
+        String secondGroup = "second-group";
+        String bootstrapServer = "localhost:9092";
+
+        String[] cgcArgs = new String[]{"--bootstrap-server", bootstrapServer, "--list"};
+        Admin adminClient = mock(KafkaAdminClient.class);
+        ListGroupsResult result = mock(ListGroupsResult.class);
+        when(result.all()).thenReturn(KafkaFuture.completedFuture(Arrays.asList(
+                new GroupListing(firstGroup, Optional.of(GroupType.STREAMS), "streams", Optional.of(GroupState.STABLE)),
+                new GroupListing(secondGroup, Optional.of(GroupType.STREAMS), "streams", Optional.of(GroupState.EMPTY))
+        )));
+        when(adminClient.listGroups(any(ListGroupsOptions.class))).thenReturn(result);
+        StreamsGroupCommand.StreamsGroupService service = getStreamsGroupService(cgcArgs, adminClient);
+        Set<String> expectedGroups = new HashSet<>(Arrays.asList(firstGroup, secondGroup));
+
+        final Set[] foundGroups = new Set[]{Collections.emptySet()};
+        TestUtils.waitForCondition(() -> {
+            foundGroups[0] = new HashSet<>(service.listStreamsGroups());
+            return Objects.equals(expectedGroups, foundGroups[0]);
+        }, "Expected --list to show groups " + expectedGroups + ", but found " + foundGroups[0] + ".");
+        service.close();
+    }
+
+    @Test
+    public void testListWithUnrecognizedOption() {
+        String bootstrapServer = "localhost:9092";
+        String[] cgcArgs = new String[]{"--frivolous-nonsense", "--bootstrap-server", bootstrapServer, "--list"};
+        assertThrows(OptionException.class, () -> getStreamsGroupService(cgcArgs, new MockAdminClient()));
+    }
+
+    @Test
+    public void testListStreamsGroupsWithStates() throws Exception {
+        String firstGroup = "first-group";
+        String secondGroup = "second-group";
+        String bootstrapServer = "localhost:9092";
+
+        String[] cgcArgs = new String[]{"--bootstrap-server", bootstrapServer, "--list", "--state"};
+        Admin adminClient = mock(KafkaAdminClient.class);
+        ListGroupsResult resultWithAllStates = mock(ListGroupsResult.class);
+        when(resultWithAllStates.all()).thenReturn(KafkaFuture.completedFuture(Arrays.asList(
+                new GroupListing(firstGroup, Optional.of(GroupType.STREAMS), "streams", Optional.of(GroupState.STABLE)),
+                new GroupListing(secondGroup, Optional.of(GroupType.STREAMS), "streams", Optional.of(GroupState.EMPTY))
+        )));
+        when(adminClient.listGroups(any(ListGroupsOptions.class))).thenReturn(resultWithAllStates);
+        StreamsGroupCommand.StreamsGroupService service = getStreamsGroupService(cgcArgs, adminClient);
+        Set<GroupListing> expectedListing = new HashSet<>(Arrays.asList(
+                new GroupListing(firstGroup, Optional.of(GroupType.STREAMS), "streams", Optional.of(GroupState.STABLE)),
+                new GroupListing(secondGroup, Optional.of(GroupType.STREAMS), "streams", Optional.of(GroupState.EMPTY))));
+
+        final Set[] foundListing = new Set[]{Collections.emptySet()};
+        TestUtils.waitForCondition(() -> {
+            foundListing[0] = new HashSet<>(service.listStreamsGroupsInStates(new HashSet<>(Arrays.asList(GroupState.values()))));
+            return Objects.equals(expectedListing, foundListing[0]);
+        }, "Expected to show groups " + expectedListing + ", but found " + foundListing[0]);
+
+        ListGroupsResult resultWithStableState = mock(ListGroupsResult.class);
+        when(resultWithStableState.all()).thenReturn(KafkaFuture.completedFuture(Collections.singletonList(
+                new GroupListing(firstGroup, Optional.of(GroupType.STREAMS), "streams", Optional.of(GroupState.STABLE))
+        )));
+        when(adminClient.listGroups(any(ListGroupsOptions.class))).thenReturn(resultWithStableState);
+        Set<GroupListing> expectedListingStable = Collections.singleton(
+                new GroupListing(firstGroup, Optional.of(GroupType.STREAMS), "streams", Optional.of(GroupState.STABLE)));
+
+        foundListing[0] = Collections.emptySet();
+
+        TestUtils.waitForCondition(() -> {
+            foundListing[0] = new HashSet<>(service.listStreamsGroupsInStates(Collections.singleton(GroupState.STABLE)));
+            return Objects.equals(expectedListingStable, foundListing[0]);
+        }, "Expected to show groups " + expectedListingStable + ", but found " + foundListing[0]);
+        service.close();
+    }
+
+    @Test
+    public void testGroupStatesFromString() {
+        Set<GroupState> result = StreamsGroupCommand.groupStatesFromString("empty");
+        assertEquals(new HashSet<>(Collections.singletonList(GroupState.EMPTY)), result);
+        result = StreamsGroupCommand.groupStatesFromString("EMPTY");
+        assertEquals(new HashSet<>(Collections.singletonList(GroupState.EMPTY)), result);
+
+        result = StreamsGroupCommand.groupStatesFromString("notready");
+        assertEquals(new HashSet<>(Collections.singletonList(GroupState.NOT_READY)), result);
+        result = StreamsGroupCommand.groupStatesFromString("notReady");
+        assertEquals(new HashSet<>(Collections.singletonList(GroupState.NOT_READY)), result);
+
+        result = StreamsGroupCommand.groupStatesFromString("assigning");
+        assertEquals(new HashSet<>(Collections.singletonList(GroupState.ASSIGNING)), result);
+        result = StreamsGroupCommand.groupStatesFromString("ASSIGNING");
+        assertEquals(new HashSet<>(Collections.singletonList(GroupState.ASSIGNING)), result);
+
+        result = StreamsGroupCommand.groupStatesFromString("RECONCILING");
+        assertEquals(new HashSet<>(Collections.singletonList(GroupState.RECONCILING)), result);
+        result = StreamsGroupCommand.groupStatesFromString("reconCILING");
+        assertEquals(new HashSet<>(Collections.singletonList(GroupState.RECONCILING)), result);
+
+        result = StreamsGroupCommand.groupStatesFromString("STABLE");
+        assertEquals(new HashSet<>(Collections.singletonList(GroupState.STABLE)), result);
+        result = StreamsGroupCommand.groupStatesFromString("stable");
+        assertEquals(new HashSet<>(Collections.singletonList(GroupState.STABLE)), result);
+
+        result = StreamsGroupCommand.groupStatesFromString("DEAD");
+        assertEquals(new HashSet<>(Collections.singletonList(GroupState.DEAD)), result);
+        result = StreamsGroupCommand.groupStatesFromString("dead");
+        assertEquals(new HashSet<>(Collections.singletonList(GroupState.DEAD)), result);
+
+
+        result = StreamsGroupCommand.groupStatesFromString("PREPARINGREBALANCE");
+        assertEquals(new HashSet<>(Collections.singletonList(GroupState.PREPARING_REBALANCE)), result);

Review Comment:
   @aliehsaeedii as per discussion, can you please remove preparing_rebalance from the possible states?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org