[jira] [Assigned] (SAMZA-2307) Migrate tests in samza-test to use in-memory system instead of Kafka/Zookeeper

["Cameron Lee (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SAMZA-2307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Cameron Lee reassigned SAMZA-2307:
----------------------------------

    Assignee:     (was: Cameron Lee)

> Migrate tests in samza-test to use in-memory system instead of Kafka/Zookeeper
> ------------------------------------------------------------------------------
>
>                 Key: SAMZA-2307
>                 URL: https://issues.apache.org/jira/browse/SAMZA-2307
>             Project: Samza
>          Issue Type: Improvement
>            Reporter: Cameron Lee
>            Priority: Major
>
> Many tests in samza-test use the IntegrationTestHarness which sets up local instances of Kafka and Zookeeper. However, these systems take time to start up. Also, using Kafka as a system is slower since there are inter-process calls.
> If we can replace the Kafka usage with an in-memory system, then we don't have to start up Kafka/Zookeeper, and test runs should be significantly faster.
> Some tests that could possibly be updated (not a complete list). I have added some notes about the current state of some of the tests. For example, the tests which directly use Kafka would need changes to produce to the in-memory system instead.
> - FaultInjectionTest: directly produces to kafka
>  - WatermarkIntegrationTest: need special runner to inspect processors
>  - EndOfStreamIntegrationTest
>  - TestStreamProcessor: directly depends on kafka
>  - TestZkLocalApplicationRunner: tied to publishing kafka events
>  - TestRepartitionJoinWindowApp: tied to KafkaSystemFactory for checking delete messages
>  - StreamApplicationIntegrationTestHarness: need to create input topics with correct partition counts
>  - TestLocalTable*: already uses some system that is in-memory-like; uses kafka for default systems
>  - update IntegrationTestHarness.createTopic to use in-memory system instead of kafka?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)