You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Sönke Liebau (JIRA)" <ji...@apache.org> on 2019/04/15 07:14:00 UTC
[jira] [Commented] (KAFKA-8226) New MirrorMaker option
partition.to.partition
[ https://issues.apache.org/jira/browse/KAFKA-8226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16817610#comment-16817610 ]
Sönke Liebau commented on KAFKA-8226:
-------------------------------------
Hi [~ernisv],
the functionality that you are implementing is definitely useful, however it can also easily be achieved by implementing a custom MessageHandler that preserves partitioning.
Personally I think we should try to avoid adding too many features for which extensions point were already designed in Kafka.
Additionally there is [MirrorMaker 2.0|[https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0#KIP-382:MirrorMaker2.0-RemoteTopics,Partitions]] coming up, which I believe has this functionality built in.
> New MirrorMaker option partition.to.partition
> ---------------------------------------------
>
> Key: KAFKA-8226
> URL: https://issues.apache.org/jira/browse/KAFKA-8226
> Project: Kafka
> Issue Type: Improvement
> Components: core
> Reporter: Ernestas Vaiciukevičius
> Priority: Major
>
> Currently when MirrorMaker moves data between topics with records with null keys - it shuffles records between destination topic's partitions. Sometimes it's desirable to try preserving the original partition.
> Related PR adds new command line option to do that:
> When partition.to.partition=true MirrorMaker retains the partition number when mirroring records even without the keys.
> When using this option - source and destination topics are assumed to have the same number of partitions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)