Running mirrormaker in HA?

[Prakash Gowri Shankor <pr...@gmail.com>]

Hi,

I've looked at the documentation for mirrormaker. It doesnt appear that
there is a notion of HA ( high availability ) for the mirrormaker tool
itself. Ideally, I would like to have some topics flowing to another
datacenter with a 2 node mirrormaker in between, where one mirrormaker is
always active  and the other passive until it needs to become primary. The
transfer from passive to active also needs to happen with offset awareness
( ie previously written topic offset should be picked up by the new
primary) . Is anything like this planned in future releases ?

Thanks,
Prakash

Re: Running mirrormaker in HA?

[Prakash Gowri Shankor <pr...@gmail.com>]

 I see what you mean by the implicit HA.Thanks Guozhang.


On Wed, Jun 18, 2014 at 1:45 PM, Guozhang Wang <wa...@gmail.com> wrote:

> Hi Prakash,
>
> Actually the MM is HA such that it used a consumer group fetching data from
> the source DC and another producer pushing them to the destination DC. So
> suppose there are two instances of MM as in your case, each of the instance
> will get half of the traffic mirroring to the other DC since they have
> consumers in the same group. When one of them dies, the load balance will
> automatically moves all its traffic to the other instance.
>
> Guozhang
>
>
> On Wed, Jun 18, 2014 at 1:10 PM, Prakash Gowri Shankor <
> prakash.shankor@gmail.com> wrote:
>
> > Hi,
> >
> > I've looked at the documentation for mirrormaker. It doesnt appear that
> > there is a notion of HA ( high availability ) for the mirrormaker tool
> > itself. Ideally, I would like to have some topics flowing to another
> > datacenter with a 2 node mirrormaker in between, where one mirrormaker is
> > always active  and the other passive until it needs to become primary.
> The
> > transfer from passive to active also needs to happen with offset
> awareness
> > ( ie previously written topic offset should be picked up by the new
> > primary) . Is anything like this planned in future releases ?
> >
> > Thanks,
> > Prakash
> >
>
>
>
> --
> -- Guozhang
>

Re: Running mirrormaker in HA?

[Guozhang Wang <wa...@gmail.com>]

Hi Prakash,

Actually the MM is HA such that it used a consumer group fetching data from
the source DC and another producer pushing them to the destination DC. So
suppose there are two instances of MM as in your case, each of the instance
will get half of the traffic mirroring to the other DC since they have
consumers in the same group. When one of them dies, the load balance will
automatically moves all its traffic to the other instance.

Guozhang


On Wed, Jun 18, 2014 at 1:10 PM, Prakash Gowri Shankor <
prakash.shankor@gmail.com> wrote:

> Hi,
>
> I've looked at the documentation for mirrormaker. It doesnt appear that
> there is a notion of HA ( high availability ) for the mirrormaker tool
> itself. Ideally, I would like to have some topics flowing to another
> datacenter with a 2 node mirrormaker in between, where one mirrormaker is
> always active  and the other passive until it needs to become primary. The
> transfer from passive to active also needs to happen with offset awareness
> ( ie previously written topic offset should be picked up by the new
> primary) . Is anything like this planned in future releases ?
>
> Thanks,
> Prakash
>



-- 
-- Guozhang