[jira] [Created] (KAFKA-12453) Guidance on whether a topology is eligible for optimisation

["Patrick O'Keeffe (Jira)" <ji...@apache.org>]

Patrick O'Keeffe created KAFKA-12453:
----------------------------------------

             Summary: Guidance on whether a topology is eligible for optimisation
                 Key: KAFKA-12453
                 URL: https://issues.apache.org/jira/browse/KAFKA-12453
             Project: Kafka
          Issue Type: Improvement
          Components: streams
            Reporter: Patrick O'Keeffe


Since the introduction of KStream.toTable() in Kafka 2.6.x, the decision about whether a topology is eligible for optimisation is no longer a simple one, and is related to whether toTable() operations are preceded by key changing operators.

This decision requires expert level knowledge, and there are serious implications associated with getting it wrong in terms of fault tolerance

Some idea spring to mind:
 # Topology.describe() could indicate whether this topology is eligible for optimisation
 # Topologies could be automatically optimised - note this may have an impact at deployment time, in that an application reset may be required. The developer would need to made aware of this and adjust the deployment instructions accordingly

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)