[jira] [Created] (KAFKA-12521) Expand coverage of integration tests with more realistic applications

["A. Sophie Blee-Goldman (Jira)" <ji...@apache.org>]

A. Sophie Blee-Goldman created KAFKA-12521:
----------------------------------------------

             Summary: Expand coverage of integration tests with more realistic applications
                 Key: KAFKA-12521
                 URL: https://issues.apache.org/jira/browse/KAFKA-12521
             Project: Kafka
          Issue Type: Improvement
          Components: streams
            Reporter: A. Sophie Blee-Goldman


Many of the integration tests in Kafka Streams run a minimal topology, often just consuming a single input topic without doing any processing or producing to an output topic. Some of the complex concurrency bugs that we've found only showed up when we had some actual data to process and a stateful topology: [KAFKA-12503|https://issues.apache.org/jira/browse/KAFKA-12503] and [KAFKA-12500|https://issues.apache.org/jira/browse/KAFKA-12500]

We should try to expand the Streams applications in these test to really flex the tested feature or general stability of streams. These tests should

# Produce some data to the input topic so Streams will actually have something to consume
# Include one or more stateful operators in the topology, with caching enabled
# Insert a repartition, or force one with a key-changing operation, to get multiple subtopologies
# Write out some results to an output topic
# Verify the results in the output topic

We don't necessarily need to do all of these at the same time, any incremental improvement would help. Perhaps we could even write a standard framework for integration tests that includes all of the above, so we need only plug our test into the framework when designing new integration tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)