[kafka] branch 3.0 updated: HOTFIX: Modify system test config to reduce time to stable task assignment. (#11090)

[ca...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

cadonna pushed a commit to branch 3.0
in repository https://gitbox.apache.org/repos/asf/kafka.git


The following commit(s) were added to refs/heads/3.0 by this push:
     new 00c51ea  HOTFIX: Modify system test config to reduce time to stable task assignment. (#11090)
00c51ea is described below

commit 00c51eae1c1e9ee2b080a7b0155fdb3c7c6b6d4c
Author: Bruno Cadonna <ca...@apache.org>
AuthorDate: Wed Jul 21 07:58:14 2021 +0200

    HOTFIX: Modify system test config to reduce time to stable task assignment. (#11090)
    
    Currently, we verify the startup of a Streams client by checking the transition
    from REBALANCING to RUNNING and if the client processed some records
    in the EOS system test. However, if the Streams client only
    has standby tasks assigned as it can happen if the client is catching
    up by using warm-up replicas, the client will never process
    records within the timeout of the startup verification. Hence, the test
    will fail although everything is fine. This commit fixes this by reducing
    the time to the next probing rebalance and by increasing the number of
    max warm-up replicas. In such a way, the catch up of the client and the
    following processing of records should still be within the startup verification
    timeout of the client.
    
    Reviewers: Anna Sophie Blee-Goldman <ab...@apache.org>
---
 streams/src/test/java/org/apache/kafka/streams/tests/EosTestClient.java | 2 ++
 tests/kafkatest/tests/streams/streams_eos_test.py                       | 2 +-
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/streams/src/test/java/org/apache/kafka/streams/tests/EosTestClient.java b/streams/src/test/java/org/apache/kafka/streams/tests/EosTestClient.java
index 18d9458..b9b24da 100644
--- a/streams/src/test/java/org/apache/kafka/streams/tests/EosTestClient.java
+++ b/streams/src/test/java/org/apache/kafka/streams/tests/EosTestClient.java
@@ -104,6 +104,8 @@ public class EosTestClient extends SmokeTestUtil {
         props.put(StreamsConfig.APPLICATION_ID_CONFIG, APP_ID);
         props.put(StreamsConfig.NUM_STREAM_THREADS_CONFIG, 1);
         props.put(StreamsConfig.NUM_STANDBY_REPLICAS_CONFIG, 2);
+        props.put(StreamsConfig.PROBING_REBALANCE_INTERVAL_MS_CONFIG, Duration.ofMinutes(1).toMillis());
+        props.put(StreamsConfig.MAX_WARMUP_REPLICAS_CONFIG, Integer.MAX_VALUE);
         props.put(StreamsConfig.REPLICATION_FACTOR_CONFIG, 3);
         props.put(StreamsConfig.CACHE_MAX_BYTES_BUFFERING_CONFIG, 0);
         props.put(StreamsConfig.COMMIT_INTERVAL_MS_CONFIG, 5000L); // increase commit interval to make sure a client is killed having an open transaction
diff --git a/tests/kafkatest/tests/streams/streams_eos_test.py b/tests/kafkatest/tests/streams/streams_eos_test.py
index bf07cb4..5e9091e 100644
--- a/tests/kafkatest/tests/streams/streams_eos_test.py
+++ b/tests/kafkatest/tests/streams/streams_eos_test.py
@@ -170,5 +170,5 @@ class StreamsEosTest(KafkaTest):
 
     def wait_for(self, monitor, processor, output):
         monitor.wait_until(output,
-                           timeout_sec=300,
+                           timeout_sec=480,
                            err_msg=("Never saw output '%s' on " % output) + str(processor.node.account))