[jira] [Updated] (KAFKA-2649) Add support for custom partitioner in sink nodes

["Guozhang Wang (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-2649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Guozhang Wang updated KAFKA-2649:
---------------------------------
    Resolution: Fixed
        Status: Resolved  (was: Patch Available)

Issue resolved by pull request 309
[https://github.com/apache/kafka/pull/309]

> Add support for custom partitioner in sink nodes
> ------------------------------------------------
>
>                 Key: KAFKA-2649
>                 URL: https://issues.apache.org/jira/browse/KAFKA-2649
>             Project: Kafka
>          Issue Type: Sub-task
>          Components: kafka streams
>            Reporter: Randall Hauch
>            Assignee: Randall Hauch
>             Fix For: 0.9.1.0
>
>
> The only way for Processor implementations to control partitioning of forwarded messages is to set the partitioner class as property {{ProducerConfig.PARTITIONER_CLASS_CONFIG}} in the StreamsConfig, which should be set to the name of a {{org.apache.kafka.clients.producer.Partitioner}} implementation. However, doing this requires the partitioner knowing how to properly partition *all* topics, not just the one or few topics used by the Processor.
> Instead, Kafka Streams should make it easy to optionally add a partitioning function for each sink used in a topology. Each sink represents a single output topic, and thus is far simpler to implement. Additionally, the sink is already typed with the key and value types (via serdes for the keys and values), so the partitioner can be also be typed with the key and value types. Finally, this also keeps the logic of choosing partitioning strategies where it belongs, as part of building the topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)