[jira] [Created] (KAFKA-8080) Remove streams_eos_test system test

["John Roesler (JIRA)" <ji...@apache.org>]

John Roesler created KAFKA-8080:
-----------------------------------

             Summary: Remove streams_eos_test system test
                 Key: KAFKA-8080
                 URL: https://issues.apache.org/jira/browse/KAFKA-8080
             Project: Kafka
          Issue Type: Improvement
          Components: streams
            Reporter: John Roesler


After KAFKA-7944 / [https://github.com/apache/kafka/pull/6382] , the system test streams_eos_test.py is mostly redundant.

Quoting my analysis from [https://github.com/apache/kafka/pull/6382#discussion_r263536548,] 
{quote}Ok, so the smoke test and the eos test are similar, but not identical.

The smoke test application has more features in it, though. So, we have more feature coverage under eos when we test with the smoke test.

The eos test evaluates two topologies one with no repartition, and one with a repartition. The smoke test topology contains repartitions, so it only tests _with_ repartition. I think that it should be sufficient to test only _with_ repartition.

The eos test verification specifically checks that "all transactions finished" ({{org.apache.kafka.streams.tests.EosTestDriver#verifyAllTransactionFinished}}). I'm not clear on exactly what we're looking for here. It looks like we create a transactional producer and send a record to each partition and then expect to get all those records back, without seeing any other records. But I'm not sure why we're doing this. If we want to drop the eos test, then we might need to add this to the smoke test verification.
{quote}
And [~guozhang]'s reply:
{quote}{{verifyAllTransactionFinished}} aimed to avoid a situation that some dangling txn is open forever, without committing or aborting. Because the consumer needs to guarantee offset ordering when returning data, with read-committed they will also be blocked on those open txn's data forever (this usually indicates a broker-side issue, not streams though, but still).

I think we should still retain this check if we want to merge in the EosTest to SmokeTest.
{quote}
 

As described above, the task is simply to add a similar check to the end of the verification logic in the SmokeTestDriver. Then, we can remove the EOS system test, as well as all the Java code that supports it.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)