[jira] [Commented] (KAFKA-10848) Allow fine grained control over cross-partition processing order

["Sagar Rao (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-10848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17253224#comment-17253224 ] 

Sagar Rao commented on KAFKA-10848:
-----------------------------------

[~mjsax], You have added a needs kip label to this. Can I take a stab at this one?

> Allow fine grained control over cross-partition processing order
> ----------------------------------------------------------------
>
>                 Key: KAFKA-10848
>                 URL: https://issues.apache.org/jira/browse/KAFKA-10848
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: Matthias J. Sax
>            Priority: Major
>              Labels: needs-kip
>
> Currently, KafkaStreams implements a hard-coded timestamp based strategy to pick the next record to process for a task, given that a task has multiple partitions.
> In general, this strategy works well for the DSL, but for PAPI users, there might be cases when the strategy should be customized. And even for the DSL, there is one corner case (for a stream-table join) for which the table-side record should be processed first if two records have the same timestamp (at least, this gap exists as long as we don't have multi-version KTables), while we cannot enforce this behavior because at runtime we don't know anything about KStream vs KTable or an existing downstream join.
> Thus, we might want to allow users to plugin a custom strategy to pick the next record for processing.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)