You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Calvin Lei <ck...@gmail.com> on 2013/07/14 16:43:04 UTC

Questions on mirror maker

Hi all,
    I have two questions regarding setting up mirror maker for our cross cluster replication (DC1 to DC2, for instance)
    1. In what use case you would want to specify multiplied consumer configs? 
    2. It seems like the consumer inside the mirror is a SimpleConsumer. Is it possible to switch it to HghLevelConsumer? Assuming our DC2 is down, I would shut down the mirror until DC2 is back up. Once it is up, I want the mirror to pick up when it is left off and start replicating messages that has not been delivered to DC2. By using a Highlevel consumer, that kind of behavior is built in because the offset is being saved in the DC1 zookeeper. am i correct?

thanks,
Cal

Re: Questions on mirror maker

Posted by Calvin Lei <ck...@gmail.com>.
Joel,
   For #1, I meant multiple consumer configs. Please excuse me for the typo.
   For #2, turns out i started the mirror before i brought up the kafka
cluster, hence all the messages failed to send to the remote cluster.

thanks,
Cal


On Tue, Jul 16, 2013 at 3:16 AM, Joel Koshy <jj...@gmail.com> wrote:

> Calvin,
>
> For (1) can you clarify what you mean by "multiplied" consumer configs?
> For (2) the mirror-maker actually uses the high level consumer.
>
> Thanks,
>
> Joel
>
> On Sun, Jul 14, 2013 at 7:43 AM, Calvin Lei <ck...@gmail.com> wrote:
> > Hi all,
> >     I have two questions regarding setting up mirror maker for our cross
> cluster replication (DC1 to DC2, for instance)
> >     1. In what use case you would want to specify multiplied consumer
> configs?
> >     2. It seems like the consumer inside the mirror is a SimpleConsumer.
> Is it possible to switch it to HghLevelConsumer? Assuming our DC2 is down,
> I would shut down the mirror until DC2 is back up. Once it is up, I want
> the mirror to pick up when it is left off and start replicating messages
> that has not been delivered to DC2. By using a Highlevel consumer, that
> kind of behavior is built in because the offset is being saved in the DC1
> zookeeper. am i correct?
> >
> > thanks,
> > Cal
>

Re: Questions on mirror maker

Posted by Joel Koshy <jj...@gmail.com>.
Calvin,

For (1) can you clarify what you mean by "multiplied" consumer configs?
For (2) the mirror-maker actually uses the high level consumer.

Thanks,

Joel

On Sun, Jul 14, 2013 at 7:43 AM, Calvin Lei <ck...@gmail.com> wrote:
> Hi all,
>     I have two questions regarding setting up mirror maker for our cross cluster replication (DC1 to DC2, for instance)
>     1. In what use case you would want to specify multiplied consumer configs?
>     2. It seems like the consumer inside the mirror is a SimpleConsumer. Is it possible to switch it to HghLevelConsumer? Assuming our DC2 is down, I would shut down the mirror until DC2 is back up. Once it is up, I want the mirror to pick up when it is left off and start replicating messages that has not been delivered to DC2. By using a Highlevel consumer, that kind of behavior is built in because the offset is being saved in the DC1 zookeeper. am i correct?
>
> thanks,
> Cal