You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Victoria Xia (Jira)" <ji...@apache.org> on 2023/04/11 00:52:00 UTC
[jira] [Updated] (KAFKA-14834) Improved processor semantics for versioned stores
[ https://issues.apache.org/jira/browse/KAFKA-14834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Victoria Xia updated KAFKA-14834:
---------------------------------
Description:
With the introduction of versioned state stores in [KIP-889|https://cwiki.apache.org/confluence/display/KAFKA/KIP-889%3A+Versioned+State+Stores], we should leverage them to provide improved join semantics.
As described in [KIP-914|https://cwiki.apache.org/confluence/display/KAFKA/KIP-914%3A+DSL+Processor+Semantics+for+Versioned+Stores], we will make the following four improvements:
* stream-table joins will perform a timestamped lookup (using the stream-side record timestamp) if the table is versioned
* table-table joins, including foreign key joins, will not produce new join results on out-of-order records (by key) from versioned tables
* table filters will disable the existing optimization to not send duplicate tombstones when applied to a versioned table
* table aggregations will ignore out-of-order records when aggregating a versioned table
was:
With the introduction of versioned state stores in [KIP-889|https://cwiki.apache.org/confluence/display/KAFKA/KIP-889%3A+Versioned+State+Stores], we should leverage them to provide improved join semantics.
As described in [KIP-914|https://cwiki.apache.org/confluence/display/KAFKA/KIP-914%3A+Join+Processor+Semantics+for+Versioned+Stores], we will make the following two improvements:
* stream-table joins will perform a timestamped lookup (using the stream-side record timestamp) if the table is materialized with a versioned store
* table-table joins, including foreign key joins, will not produce new join results on out-of-order records (by key) from tables materialized with versioned stores
Summary: Improved processor semantics for versioned stores (was: Improved stream-table and table-table join semantics for versioned stores)
> Improved processor semantics for versioned stores
> -------------------------------------------------
>
> Key: KAFKA-14834
> URL: https://issues.apache.org/jira/browse/KAFKA-14834
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Victoria Xia
> Assignee: Victoria Xia
> Priority: Major
> Labels: kip, streams
>
> With the introduction of versioned state stores in [KIP-889|https://cwiki.apache.org/confluence/display/KAFKA/KIP-889%3A+Versioned+State+Stores], we should leverage them to provide improved join semantics.
> As described in [KIP-914|https://cwiki.apache.org/confluence/display/KAFKA/KIP-914%3A+DSL+Processor+Semantics+for+Versioned+Stores], we will make the following four improvements:
> * stream-table joins will perform a timestamped lookup (using the stream-side record timestamp) if the table is versioned
> * table-table joins, including foreign key joins, will not produce new join results on out-of-order records (by key) from versioned tables
> * table filters will disable the existing optimization to not send duplicate tombstones when applied to a versioned table
> * table aggregations will ignore out-of-order records when aggregating a versioned table
--
This message was sent by Atlassian Jira
(v8.20.10#820010)