You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kafka.apache.org by ew...@apache.org on 2016/02/26 19:38:01 UTC
kafka git commit: KAFKA-3243: Fix Kafka basic ops documentation for
Mirror maker, blacklist is not supported for new consumers
Repository: kafka
Updated Branches:
refs/heads/trunk d51dd5fc7 -> 4542b027c
KAFKA-3243: Fix Kafka basic ops documentation for Mirror maker, blacklist is not supported for new consumers
…list is not supported for new consumers
Author: Ashish Singh <as...@cloudera.com>
Reviewers: Grant Henke <gr...@gmail.com>, Ewen Cheslack-Postava <ew...@confluent.io>
Closes #923 from SinghAsDev/KAFKA-3243
Project: http://git-wip-us.apache.org/repos/asf/kafka/repo
Commit: http://git-wip-us.apache.org/repos/asf/kafka/commit/4542b027
Tree: http://git-wip-us.apache.org/repos/asf/kafka/tree/4542b027
Diff: http://git-wip-us.apache.org/repos/asf/kafka/diff/4542b027
Branch: refs/heads/trunk
Commit: 4542b027c1ac361cc8770383b66abac2129a05e7
Parents: d51dd5f
Author: Ashish Singh <as...@cloudera.com>
Authored: Fri Feb 26 10:37:41 2016 -0800
Committer: Ewen Cheslack-Postava <me...@ewencp.org>
Committed: Fri Feb 26 10:37:41 2016 -0800
----------------------------------------------------------------------
.../main/scala/kafka/tools/MirrorMaker.scala | 21 +++++++++++++++-----
docs/ops.html | 2 +-
2 files changed, 17 insertions(+), 6 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/kafka/blob/4542b027/core/src/main/scala/kafka/tools/MirrorMaker.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/kafka/tools/MirrorMaker.scala b/core/src/main/scala/kafka/tools/MirrorMaker.scala
index 95b0aad..4bc38d9 100755
--- a/core/src/main/scala/kafka/tools/MirrorMaker.scala
+++ b/core/src/main/scala/kafka/tools/MirrorMaker.scala
@@ -173,9 +173,22 @@ object MirrorMaker extends Logging with KafkaMetricsGroup {
}
CommandLineUtils.checkRequiredArgs(parser, options, consumerConfigOpt, producerConfigOpt)
- if (List(whitelistOpt, blacklistOpt).count(options.has) != 1) {
- println("Exactly one of whitelist or blacklist is required.")
- System.exit(1)
+
+ val useNewConsumer = options.has(useNewConsumerOpt)
+ if (useNewConsumer) {
+ if (options.has(blacklistOpt)) {
+ error("blacklist can not be used when using new consumer in mirror maker. Use whitelist instead.")
+ System.exit(1)
+ }
+ if (!options.has(whitelistOpt)) {
+ error("whitelist must be specified when using new consumer in mirror maker.")
+ System.exit(1)
+ }
+ } else {
+ if (List(whitelistOpt, blacklistOpt).count(options.has) != 1) {
+ error("Exactly one of whitelist or blacklist is required.")
+ System.exit(1)
+ }
}
abortOnSendFailure = options.valueOf(abortOnSendFailureOpt).toBoolean
@@ -200,8 +213,6 @@ object MirrorMaker extends Logging with KafkaMetricsGroup {
producerProps.setProperty(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.ByteArraySerializer")
producer = new MirrorMakerProducer(producerProps)
- val useNewConsumer = options.has(useNewConsumerOpt)
-
// Create consumers
val mirrorMakerConsumers = if (!useNewConsumer) {
val customRebalanceListener = {
http://git-wip-us.apache.org/repos/asf/kafka/blob/4542b027/docs/ops.html
----------------------------------------------------------------------
diff --git a/docs/ops.html b/docs/ops.html
index 33080e2..3cec8d6 100644
--- a/docs/ops.html
+++ b/docs/ops.html
@@ -118,7 +118,7 @@ Here is an example showing how to mirror a single topic (named <i>my-topic</i>)
</pre>
Note that we specify the list of topics with the <code>--whitelist</code> option. This option allows any regular expression using <a href="http://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html">Java-style regular expressions</a>. So you could mirror two topics named <i>A</i> and <i>B</i> using <code>--whitelist 'A|B'</code>. Or you could mirror <i>all</i> topics using <code>--whitelist '*'</code>. Make sure to quote any regular expression to ensure the shell doesn't try to expand it as a file path. For convenience we allow the use of ',' instead of '|' to specify a list of topics.
<p>
-Sometime it is easier to say what it is that you <i>don't</i> want. Instead of using <code>--whitelist</code> to say what you want to mirror you can use <code>--blacklist</code> to say what to exclude. This also takes a regular expression argument.
+Sometimes it is easier to say what it is that you <i>don't</i> want. Instead of using <code>--whitelist</code> to say what you want to mirror you can use <code>--blacklist</code> to say what to exclude. This also takes a regular expression argument. However, <code>--blacklist</code> is not supported when using <code>--new.consumer</code>.
<p>
Combining mirroring with the configuration <code>auto.create.topics.enable=true</code> makes it possible to have a replica cluster that will automatically create and replicate all data in a source cluster even as new topics are added.