PIP-95 Live migration of producer consumer from one cluster to another.

[Prashant Kumar <pr...@gmail.com>]

Dear Pulsar community.
I am happy to submit a proposal for *Seamless migration of live traffic
from one cluster to another cluster.*

Marked down version of the proposal
<https://github.com/pkumar-singh/pulsar/wiki/PIP-95:-Live-migration-of-producer-consumer-or-reader-from-one-Pulsar-cluster-to-another>
.
https://github.com/pkumar-singh/pulsar/wiki/PIP-95:-Live-migration-of-producer-consumer-or-reader-from-one-Pulsar-cluster-to-another

Requesting your support.
Thanks and regards
-Prashant

Re: PIP-95 Live migration of producer consumer from one cluster to another.

[Joe F <jo...@gmail.com>]

 Is this mostly addressing a topic cutover or a whole cluster cutover?

What is "pulsar-admin clusters set-generation --generation <number>"

That seems to dangle in the doc, without any explanation





On Thu, Sep 16, 2021 at 11:19 AM Prashant Kumar <
prashant.kumar.singh@gmail.com> wrote:

> Dear Pulsar community.
> I am happy to submit a proposal for *Seamless migration of live traffic
> from one cluster to another cluster.*
>
> Marked down version of the proposal
> <
> https://github.com/pkumar-singh/pulsar/wiki/PIP-95:-Live-migration-of-producer-consumer-or-reader-from-one-Pulsar-cluster-to-another
> >
> .
>
> https://github.com/pkumar-singh/pulsar/wiki/PIP-95:-Live-migration-of-producer-consumer-or-reader-from-one-Pulsar-cluster-to-another
>
> Requesting your support.
> Thanks and regards
> -Prashant
>

Re: PIP-95 Live migration of producer consumer from one cluster to another.

[Matteo Merli <ma...@gmail.com>]

Hi Prashant,

I think that this proposal could be worded in a slightly more general
way. Other than migrating to different clusters, it could be very well
useful for migrating to a different topic name, or to move the topic
to a different namespace.
For that the redirection could also include a "topic" field.

I couldn't find in the doc the reason why we need to add a
`generation_id` field to MessageIdData. In general we should try to
avoid that in order to avoid introducing incompatibilities, or in
general to grow even more the message id structures in most cases.

Matteo


--
Matteo Merli
<ma...@gmail.com>

On Thu, Sep 16, 2021 at 11:19 AM Prashant Kumar
<pr...@gmail.com> wrote:
>
> Dear Pulsar community.
> I am happy to submit a proposal for *Seamless migration of live traffic
> from one cluster to another cluster.*
>
> Marked down version of the proposal
> <https://github.com/pkumar-singh/pulsar/wiki/PIP-95:-Live-migration-of-producer-consumer-or-reader-from-one-Pulsar-cluster-to-another>
> .
> https://github.com/pkumar-singh/pulsar/wiki/PIP-95:-Live-migration-of-producer-consumer-or-reader-from-one-Pulsar-cluster-to-another
>
> Requesting your support.
> Thanks and regards
> -Prashant