[jira] [Created] (KAFKA-10848) Allow fine grained control over cross-partition processing order

["Matthias J. Sax (Jira)" <ji...@apache.org>]

Matthias J. Sax created KAFKA-10848:
---------------------------------------

             Summary: Allow fine grained control over cross-partition processing order
                 Key: KAFKA-10848
                 URL: https://issues.apache.org/jira/browse/KAFKA-10848
             Project: Kafka
          Issue Type: Improvement
          Components: streams
            Reporter: Matthias J. Sax


Currently, KafkaStreams implements a hard-coded timestamp based strategy to pick the next record to process for a task, given that a task has multiple partitions.

In general, this strategy works well for the DSL, but for PAPI users, there might be cases when the strategy should be customized. And even for the DSL, there is one corner case (for a stream-table join) for which the table-side record should be processed first if two records have the same timestamp (at least, this gap exists as long as we don't have multi-version KTables), while we cannot enforce this behavior because at runtime we don't know anything about KStream vs KTable or an existing downstream join.

Thus, we might want to allow users to plugin a custom strategy to pick the next record for processing.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)