You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by Rajan Dhabalia <rd...@apache.org> on 2022/07/12 17:18:53 UTC
Re: [VOTE] PIP-136: Sync Pulsar policies across multiple clouds
Hi,
After multiple discussions during Community meetings: I have updated PIP
with the changes and approach we agreed on. This PIP allows any
metadata-store to generate change-events using plugable
MetadataSynchronizer to publish and handle change events. PIP also uses
change-event topic as a WAL topic to persist change events and applies it
async in local and remote regions.
Can we please review the PIP: https://github.com/apache/pulsar/issues/16424
and PR: https://github.com/apache/pulsar/pull/16425
Thanks,
Rajan
On Tue, Mar 22, 2022 at 6:07 PM PengHui Li <pe...@apache.org> wrote:
> > This feature allows users to sync two separate clusters which are having
> independent global-zookeeper (metadata-store) instances. So, this feature
> will not be limited to sync a few fields of policies but think in terms of
> auto creation and syncing namespaces/tenants to entirely different
> metadata-stores where they can not talk to each other.
>
> Yes, we are on the same page. If this proposal is for syncing
> tenant/namespace/topic
> between multiple metadata stores, it looks good to me. If the proposal also
> wants to
> sync the namespace policy between multiple metadata stores, I think we
> should leverage
> the existing __change_events topic under each namespace.
>
> > Again, #12136 talks about applying policies to topics but this PIP
> addresses a separate problem where it makes it possible to integrate
> independent/isolated metadata-stores with each other.
>
> Yes, it's not the same problem, but related. The namespace can disabled the
> geo-replication but enabled in topic level, in this case if we have
> multiple metadata store,
> and if the topic is a partitioned topic, we also need a way to sync the
> partitioned metadata
> to the remote cluster's metadata store?
>
> Thanks,
> Penghui
>
> On Wed, Mar 23, 2022 at 3:39 AM Rajan Dhabalia <rd...@apache.org>
> wrote:
>
> > >> Do we need to provide the ability for users to decide to replicate the
> > ACLs and replication cluster or not?
> >
> > This feature allows users to sync two separate clusters which are having
> > independent global-zookeeper (metadata-store) instances. So, this feature
> > will not be limited to sync a few fields of policies but think in terms
> of
> > auto creation and syncing namespaces/tenants to entirely different
> > metadata-stores where they can not talk to each other.
> >
> > >> BTW, we already supported topic level replication cluster
> > configuration[5], looks like in this case, [5]
> > https://github.com/apache/pulsar/pull/12136
> >
> > Again, #12136 talks about applying policies to topics but this PIP
> > addresses a separate problem where it makes it possible to integrate
> > independent/isolated metadata-stores with each other.
> >
> > Thanks,
> > Rajan
> >
> > On Mon, Mar 21, 2022 at 6:29 PM PengHui Li <pe...@apache.org> wrote:
> >
> > > Thanks for the explanation.
> > >
> > > > yes, local policies doesn't need to be replicate to other clusters
> and
> > it
> > > will only replicate global policies which is shared across multiple
> > > clusters such tenant/namespace's identity-creation, ACLs, replication
> > > clusters, etc.
> > >
> > > As described in this blog [1], section "Aggregation Replication".
> > > Do we need to provide the ability for users to decide to replicate
> > > the ACLs and replication cluster or not?
> > >
> > > Currently, if users want to achieve "Aggregation Replication", it needs
> > > multiple configuration stores. So they need to maintain the namespace,
> > > partitioned topics in each cluster. A new namespace created in one
> > cluster,
> > > it need to create the namespace in other clusters if they want to
> > replicate
> > > data to those clusters.
> > >
> > > After this proposal, they don't need to create a namespace for other
> > > clusters,
> > > Pulsar will help to replicate the configuration store changes to the
> > > replicated cluster,
> > > if the new created namespace with replication cluster A, B, and C in
> > > cluster A, the
> > > namespace will be replicated to B and C.
> > >
> > > But for the ACLs and replication clusters, it should be controlled by
> > > users?
> > > e.g. only replicate the namespace to B and C, but not the replication
> > > clusters and ACLs.
> > > So that we can achieve "Aggregation Replication" with this proposal.
> > >
> > > > Topic that will be used to share policies across clusters is
> > configurable
> > > and it can be named anything. However, we should keep it a separate
> topic
> > > as it requires unique schema and special handling to synchronize
> policies
> > > across the clusters.
> > >
> > > Yes, looks like currently we already have a mechanism to replicate
> > > policies.
> > > We have a system topic under the namespace "__change_events", which
> only
> > > has
> > > topic policy changes for now. It can replicate anything under a
> > namespace.
> > > We have defined "EventType"[2] in PulsarEvent(structure used in
> > > "__change_events").
> > > And we already have a implementation for selective PulsarEvent
> > > replication[3], and schema
> > > replication[4].
> > >
> > > So it looks like we can use the "__change_events" to replicate
> namespace
> > > policies, and use a
> > > new topic which belongs to a system namespace to replicate
> > > tenant/namespace's identity-creation,
> > > partitioned topic creation?
> > >
> > > BTW, we already supported topic level replication cluster
> > configuration[5],
> > > looks like in this case,
> > > the partitioned topic is created first in one cluster without
> replication
> > > clusters first, after the replication
> > > clusters changed, pulsar will replicate the partitioned topic to remote
> > > cluster. The same mechanism is
> > > required for non-partitioned topics(users might disabled the topic
> > > auto-creation).
> > >
> > > [1]
> > >
> > >
> >
> https://www.splunk.com/en_us/blog/devops/geo-replication-in-apache-pulsar-part-2-patterns-and-practices.html
> > > [2]
> > >
> > >
> >
> https://github.com/apache/pulsar/blob/4dcb166e0bfcce7fc85fd8d59a25b881f6f9c6fa/pulsar-common/src/main/java/org/apache/pulsar/common/events/PulsarEvent.java#L36
> > > [3]
> > >
> > >
> >
> https://github.com/apache/pulsar/wiki/PIP-92%3A-Topic-policy-across-multiple-clusters
> > > [4]
> > >
> > >
> >
> https://github.com/apache/pulsar/wiki/PIP-88%3A-Replicate-schemas-across-multiple
> > > [5] https://github.com/apache/pulsar/pull/12136
> > >
> > > Regards,
> > > Penghui
> > >
> > > On Tue, Mar 22, 2022 at 6:37 AM Rajan Dhabalia <rd...@apache.org>
> > > wrote:
> > >
> > > > >> If it contains namespace policy replication, There are some
> policies
> > > no
> > > > need to replicate to another cluster
> > > > yes, local policies doesn't need to be replicate to other clusters
> and
> > it
> > > > will only replicate global policies which is shared across multiple
> > > > clusters such tenant/namespace's identity-creation, ACLs, replication
> > > > clusters, etc.
> > > >
> > > > >> The new partitioned topic also needs to be replicated to the
> remote
> > > > cluster?
> > > > Yes.
> > > >
> > > > Topic that will be used to share policies across clusters is
> > configurable
> > > > and it can be named anything. However, we should keep it a separate
> > topic
> > > > as it requires unique schema and special handling to synchronize
> > policies
> > > > across the clusters.
> > > >
> > > > Thanks,
> > > > Rajan
> > > >
> > > > On Fri, Mar 18, 2022 at 9:12 PM PengHui Li <pe...@apache.org>
> wrote:
> > > >
> > > > > Hi Rajan,
> > > > >
> > > > > Thanks for the great proposal.
> > > > >
> > > > > Will all the namespace policies be replicated to the remote
> cluster?
> > > > > I noticed the PIP title mentioned policies, but looks like from the
> > > > > `MetadataChangeEvent`,
> > > > > no namespaces policies defined. If it contains namespace policy
> > > > > replication,
> > > > > There are some policies no need to replicate to another cluster,
> > > > > for example, the rate limiter, max producers/consumers limiter.
> > > > > In
> > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/pulsar/wiki/PIP-92%3A-Topic-policy-across-multiple-clusters
> > > > > ,
> > > > > it introduced a --global option to provide ability to apply the
> > policy
> > > in
> > > > > global or local.
> > > > >
> > > > > The new partitioned topic also needs to be replicated to the remote
> > > > > cluster?
> > > > >
> > > > > Currently, we already have a PulsarEvent struct to define the
> pulsar
> > > > system
> > > > > events,
> > > > > Looks like we can use a unified event definition by PulsarEvent.
> > > > >
> > > > > Others look good to me.
> > > > >
> > > > > Regards,
> > > > > Penghui
> > > > >
> > > > >
> > > > >
> > > > > On Sat, Mar 19, 2022 at 1:32 AM Joe F <jo...@gmail.com>
> wrote:
> > > > >
> > > > > > +1
> > > > > >
> > > > > > On Thu, Mar 17, 2022 at 12:07 PM Rajan Dhabalia <
> > > rdhabalia@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I would like to start VOTE on PIP-136:
> > > > > > > https://github.com/apache/pulsar/issues/13728
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Rajan
> > > > > > >
> > > > > > > On Tue, Feb 8, 2022 at 4:58 PM Rajan Dhabalia <
> > > dhabalia.me@gmail.com
> > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > >> How do we designate the host broker? Is it manual? How
> does
> > it
> > > > > work
> > > > > > > > when the host broker is removed from the cluster?
> > > > > > > > No, it will not be manual but as I explained earlier a broker
> > > which
> > > > > > has a
> > > > > > > > failover consumer to consume remote events will be the
> > publisher
> > > > for
> > > > > > > > metadata update. If that broker is removed then a new
> failover
> > > > > > > > consumer/broker will be selected for the same.
> > > > > > > >
> > > > > > > > >> I look forward to seeing more about this design for
> conflict
> > > > > > > resolution.
> > > > > > > > Sure, I have updated PIP to handle such race condition:
> > > > > > > https://github.com/apache/pulsar/issues/13728
> > > > > > > >
> > > > > > > >
> > > > > > > > >> (1) scenarios where the Pulsar cluster operators and
> tenant
> > > > admins
> > > > > > > are
> > > > > > > > different entities and tenants can be malicious, or more
> > > probably,
> > > > > > write
> > > > > > > > bad code that will produce malicious outcomes.
> > > > > > > > I agree, Pulsar should have provision to prevent such
> scenarios
> > > > where
> > > > > > > > changes from one tenant in a cluster can impact other
> clusters.
> > > > This
> > > > > > PIP
> > > > > > > > considers the tenant/admin will be the same at both the ends
> > but
> > > > that
> > > > > > can
> > > > > > > > not be true in all cases. We can add an enhancement later or
> we
> > > can
> > > > > > > create
> > > > > > > > a separate PIP to start discussion with the possible
> solutions.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Rajan
> > > > > > > >
> > > > > > > > On Thu, Feb 3, 2022 at 9:59 AM Joe F <jo...@gmail.com>
> > > > wrote:
> > > > > > > >
> > > > > > > >> >On my first reading, it wasn't clear if there was only one
> > > topic
> > > > > > > >> required for this feature. I now see that the topic is not
> > tied
> > > > to a
> > > > > > > >> specific tenant or namespace. As such, we can avoid
> > complicated
> > > > > > > >> authorization questions by putting the required event
> topic(s)
> > > > into
> > > > > a
> > > > > > > >> "system" tenant and namespace
> > > > > > > >>
> > > > > > > >> We should consider complicated questions. We can say why we
> > > chose
> > > > > not
> > > > > > to
> > > > > > > >> address it, or why it does not apply. for a particular
> > situation
> > > > > > > >>
> > > > > > > >> Many namespace policies are administered by tenants. As
> such
> > > any
> > > > > > tenant
> > > > > > > >> can load this topic. Is it possible for one abusive tenant
> to
> > > > make
> > > > > > your
> > > > > > > >> system topic dysfunctional?
> > > > > > > >>
> > > > > > > >> Pulsar committers should think about
> > > > > > > >> (1) scenarios where the Pulsar cluster operators and tenant
> > > admins
> > > > > > are
> > > > > > > >> different entities and tenants can be malicious, or more
> > > probably,
> > > > > > write
> > > > > > > >> bad code that will produce malicious outcomes.
> > > > > > > >> (2) whether the changes introduce additional SPOFs into the
> > > > > cluster.
> > > > > > > >>
> > > > > > > >> I don't think this PIP has those issues, but as a matter of
> > > > > > practice, I
> > > > > > > >> would like to see backend/system PIPs consider these
> questions
> > > > and
> > > > > > > >> explicitly state the conclusions with rationale
> > > > > > > >>
> > > > > > > >> Joe
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> On Wed, Feb 2, 2022 at 9:27 PM Michael Marshall <
> > > > > mmarshall@apache.org
> > > > > > >
> > > > > > > >> wrote:
> > > > > > > >>
> > > > > > > >> > Thanks for your responses.
> > > > > > > >> >
> > > > > > > >> > > I don't see a need of protobuf for this particular
> usecase
> > > > > > > >> >
> > > > > > > >> > If no one else feels strongly on this point, I am good
> with
> > > > using
> > > > > a
> > > > > > > >> POJO.
> > > > > > > >> >
> > > > > > > >> > > It doesn't matter if it's system-topic or not because
> it's
> > > > > > > >> > > configurable and the admin of the system can decide and
> > > > > configure
> > > > > > it
> > > > > > > >> > > according to the required persistent policy.
> > > > > > > >> >
> > > > > > > >> > On my first reading, it wasn't clear if there was only one
> > > topic
> > > > > > > >> > required for this feature. I now see that the topic is not
> > > tied
> > > > > to a
> > > > > > > >> > specific tenant or namespace. As such, we can avoid
> > > complicated
> > > > > > > >> > authorization questions by putting the required event
> > topic(s)
> > > > > into
> > > > > > a
> > > > > > > >> > "system" tenant and namespace, by default. The
> > `pulsar/system`
> > > > > > tenant
> > > > > > > >> > and namespace seem appropriate to me.
> > > > > > > >> >
> > > > > > > >> > > I would keep the system topic
> > > > > > > >> > > separate because this topic serves a specific purpose
> with
> > > > > > specific
> > > > > > > >> > schema,
> > > > > > > >> > > replication policy and retention policy.
> > > > > > > >> >
> > > > > > > >> > I think we need a more formal definition for system
> topics.
> > > This
> > > > > > topic
> > > > > > > >> > is exactly the kind of topic I would call a system topic:
> > its
> > > > > > intended
> > > > > > > >> > producers and consumers are Pulsar components. However,
> > > because
> > > > > > > >> > this feature can live on a topic in a system namespace, we
> > can
> > > > > avoid
> > > > > > > >> > the classification discussion for this PIP.
> > > > > > > >> >
> > > > > > > >> > > Source region will have a broker which will create a
> > > failover
> > > > > > > >> consumer on
> > > > > > > >> > > that topic and a broker with an active consumer will
> watch
> > > the
> > > > > > > >> metadata
> > > > > > > >> > > changes and publish the changes to the event topic.
> > > > > > > >> >
> > > > > > > >> > How do we designate the host broker? Is it manual? How
> does
> > it
> > > > > work
> > > > > > > >> > when the host broker is removed from the cluster?
> > > > > > > >> >
> > > > > > > >> > If we collocate the active consumer with the broker
> hosting
> > > the
> > > > > > event
> > > > > > > >> > topic, can we skip creating the failover consumer?
> > > > > > > >> >
> > > > > > > >> > > PIP briefly talks about it but I will update the PIP
> with
> > > more
> > > > > > > >> > > explanation.
> > > > > > > >> >
> > > > > > > >> > I look forward to seeing more about this design for
> conflict
> > > > > > > resolution.
> > > > > > > >> >
> > > > > > > >> > Thanks,
> > > > > > > >> > Michael
> > > > > > > >> >
> > > > > > > >> >
> > > > > > > >> >
> > > > > > > >> > On Tue, Feb 1, 2022 at 3:01 AM Rajan Dhabalia <
> > > > > > dhabalia.me@gmail.com>
> > > > > > > >> > wrote:
> > > > > > > >> > >
> > > > > > > >> > > Please find my response inline.
> > > > > > > >> > >
> > > > > > > >> > > On Mon, Jan 31, 2022 at 9:17 PM Michael Marshall <
> > > > > > > >> mmarshall@apache.org>
> > > > > > > >> > > wrote:
> > > > > > > >> > >
> > > > > > > >> > > > I think this is a very appropriate direction to take
> > > > Pulsar's
> > > > > > > >> > > > geo-replication. Your proposal is essentially to make
> > the
> > > > > > > >> > > > inter-cluster configuration event driven. This
> increases
> > > > fault
> > > > > > > >> > > > tolerance and better decouples clusters.
> > > > > > > >> > > >
> > > > > > > >> > > > Thank you for your detailed proposal. After reading
> > > through
> > > > > it,
> > > > > > I
> > > > > > > >> have
> > > > > > > >> > > > some questions :)
> > > > > > > >> > > >
> > > > > > > >> > > > 1. What do you think about using protobuf to define
> the
> > > > event
> > > > > > > >> > > > protocol? I know we already have a topic policy event
> > > stream
> > > > > > > >> > > > defined with Java POJOs, but since this feature is
> > > > > specifically
> > > > > > > >> > > > designed for egressing cloud providers, ensuring
> compact
> > > > data
> > > > > > > >> transfer
> > > > > > > >> > > > would keep egress costs down. Additionally, protobuf
> can
> > > > help
> > > > > > make
> > > > > > > >> it
> > > > > > > >> > > > clear that the schema is strict, should evolve
> > > thoughtfully,
> > > > > and
> > > > > > > >> > > > should be designed to work between clusters of
> different
> > > > > > versions.
> > > > > > > >> > > >
> > > > > > > >> > >
> > > > > > > >> > > >>> I don't see a need of protobuf for this particular
> > > > usecase
> > > > > > > >> because
> > > > > > > >> > of
> > > > > > > >> > > two reasons:
> > > > > > > >> > > >> a. policy changes don't generate huge traffic which
> > > could
> > > > > be
> > > > > > 1
> > > > > > > >> rps
> > > > > > > >> > b.
> > > > > > > >> > > and it doesn't need performance optimization.
> > > > > > > >> > > >> It should be similar as storing policy in text
> > instead
> > > > > > protobuf
> > > > > > > >> > which
> > > > > > > >> > > doesn't impact footprint size or performance due to
> > limited
> > > > > number
> > > > > > > of
> > > > > > > >> > > >> update operations and relatively less complexity. I
> > > agree
> > > > > that
> > > > > > > >> > protobuf
> > > > > > > >> > > could be another option but in this case it's not
> needed.
> > > > Also,
> > > > > > POJO
> > > > > > > >> > > >> can also support schema and versioning.
> > > > > > > >> > >
> > > > > > > >> > >
> > > > > > > >> > >
> > > > > > > >> > > >
> > > > > > > >> > > > 2. In your view, which tenant/namespace will host
> > > > > > > >> > > > `metadataSyncEventTopic`? Will there be several of
> these
> > > > > topics
> > > > > > or
> > > > > > > >> is
> > > > > > > >> > > > it just hosted in a system tenant/namespace? This
> > question
> > > > > gets
> > > > > > > back
> > > > > > > >> > > > to my questions about system topics on this mailing
> list
> > > > last
> > > > > > week
> > > > > > > >> > [0]. I
> > > > > > > >> > > > view this topic as a system topic, so we'd need to
> make
> > > sure
> > > > > > that
> > > > > > > it
> > > > > > > >> > > > has the right authorization rules and that it won't be
> > > > > affected
> > > > > > by
> > > > > > > >> > calls
> > > > > > > >> > > > like "clearNamespaceBacklog".
> > > > > > > >> > >
> > > > > > > >> > >
> > > > > > > >> > > >> It doesn't matter if it's system-topic or not
> because
> > > > it's
> > > > > > > >> > > configurable and the admin of the system can decide and
> > > > > configure
> > > > > > it
> > > > > > > >> > > according to the required persistent policy. I would
> keep
> > > the
> > > > > > system
> > > > > > > >> > topic
> > > > > > > >> > > separate because this topic serves a specific purpose
> with
> > > > > > specific
> > > > > > > >> > schema,
> > > > > > > >> > > replication policy and retention policy.
> > > > > > > >> > >
> > > > > > > >> > >
> > > > > > > >> > >
> > > > > > > >> > > >
> > > > > > > >> > > > 3. Which broker will host the metadata update
> > publisher? I
> > > > > > assume
> > > > > > > we
> > > > > > > >> > > > want the producer to be collocated with the bundle
> that
> > > > hosts
> > > > > > the
> > > > > > > >> > > > event topic. How will this be coordinated?
> > > > > > > >> > > >
> > > > > > > >> > > >> It's already explained into PIP in section: "Event
> > > > publisher
> > > > > > and
> > > > > > > >> > handler"
> > > > > > > >> > > >> Every isolated cluster deployed on a separate cloud
> > > > platform
> > > > > > will
> > > > > > > >> > have a
> > > > > > > >> > > source region and part of replicated clusters for the
> > event
> > > > > topic.
> > > > > > > The
> > > > > > > >> > > Source region will have a broker which will create a
> > > failover
> > > > > > > >> consumer on
> > > > > > > >> > > that topic and a broker with an active consumer will
> watch
> > > the
> > > > > > > >> metadata
> > > > > > > >> > > changes and publish the changes to the event topic.
> > > > > > > >> > >
> > > > > > > >> > >
> > > > > > > >> > >
> > > > > > > >> > > >
> > > > > > > >> > > > 4. Why isn't a topic a `ResourceType`? Is this because
> > the
> > > > > topic
> > > > > > > >> level
> > > > > > > >> > > > policies already have this feature? If so, is there a
> > way
> > > to
> > > > > > > >> integrate
> > > > > > > >> > > > this feature with the existing topic policy feature?
> > > > > > > >> > > >
> > > > > > > >> > > >> Yes, ResourceType can be extensible to a topic as
> well.
> > > > > > > >> > >
> > > > > > > >> > >
> > > > > > > >> > >
> > > > > > > >> > > >
> > > > > > > >> > > > 5. By decentralizing the metadata store, it looks like
> > > there
> > > > > is
> > > > > > a
> > > > > > > >> > > > chance for conflicts due to concurrent updates. How do
> > we
> > > > > handle
> > > > > > > >> those
> > > > > > > >> > > > conflicts?
> > > > > > > >> > > >
> > > > > > > >> > > >> PIP briefly talks about it but I will update the PIP
> > > with
> > > > > more
> > > > > > > >> > > explanation. MetadataChangeEvent contains source-cluster
> > and
> > > > > > updated
> > > > > > > >> > time.
> > > > > > > >> > > Also, resources Tenant/Namespace will also contain
> > > > > lastUpdatedTime
> > > > > > > >> which
> > > > > > > >> > > will help to destination clusters to handle
> > stale/duplicate
> > > > > events
> > > > > > > and
> > > > > > > >> > race
> > > > > > > >> > > conditions. Also, snapshot-sync an additional task helps
> > all
> > > > > > > clusters
> > > > > > > >> to
> > > > > > > >> > be
> > > > > > > >> > > synced with each other eventually.
> > > > > > > >> > >
> > > > > > > >> > >
> > > > > > > >> > >
> > > > > > > >> > > > I'll also note that I previously proposed a system
> event
> > > > topic
> > > > > > > here
> > > > > > > >> > > > [1] and it was proposed again here [2]. Those features
> > > were
> > > > > for
> > > > > > > >> > > > different use cases, but ultimately looked very
> similar.
> > > In
> > > > my
> > > > > > > >> view, a
> > > > > > > >> > > > stream of system events is a very natural feature to
> > > expect
> > > > > in a
> > > > > > > >> > > > streaming technology. I wonder if there is a way to
> > > > generalize
> > > > > > > this
> > > > > > > >> > > > feature to fulfill local cluster consumers and
> > > > geo-replication
> > > > > > > >> > > > consumers. Even if this PIP only implements the
> > > > > geo-replication
> > > > > > > >> > > > portion of the feature, it'd be good to design it in
> an
> > > > > > extensible
> > > > > > > >> > fashion.
> > > > > > > >> > > >
> > > > > > > >> > > >> I think answer (2) addresses this concern as well.
> > > > > > > >> > >
> > > > > > > >> > >
> > > > > > > >> > >
> > > > > > > >> > > > Thanks,
> > > > > > > >> > > > Michael
> > > > > > > >> > > >
> > > > > > > >> > > > [0]
> > > > > > > >>
> > > https://lists.apache.org/thread/pj4n4wzm3do8nkc52l7g7obh0sktzm17
> > > > > > > >> > > > [1]
> > > > > > > >>
> > > https://lists.apache.org/thread/h4cbvwjdomktsq2jo66x5qpvhdrqk871
> > > > > > > >> > > > [2]
> > > > > > > >>
> > > https://lists.apache.org/thread/0xkg0gpsobp0dbgb6tp9xq097lpm65bx
> > > > > > > >> > > >
> > > > > > > >> > > >
> > > > > > > >> > > >
> > > > > > > >> > > > On Sun, Jan 30, 2022 at 10:33 PM Rajan Dhabalia <
> > > > > > > >> rdhabalia@apache.org>
> > > > > > > >> > > > wrote:
> > > > > > > >> > > > >
> > > > > > > >> > > > > Hi,
> > > > > > > >> > > > >
> > > > > > > >> > > > > I would like to start a discussion about PIP-136:
> Sync
> > > > > Pulsar
> > > > > > > >> > policies
> > > > > > > >> > > > > across multiple clouds.
> > > > > > > >> > > > >
> > > > > > > >> > > > > PIP documentation:
> > > > > > > https://github.com/apache/pulsar/issues/13728
> > > > > > > >> > > > >
> > > > > > > >> > > > > *Motivation*
> > > > > > > >> > > > > Apache Pulsar is a cloud-native, distributed
> messaging
> > > > > > framework
> > > > > > > >> > which
> > > > > > > >> > > > > natively provides geo-replication. Many
> organizations
> > > > deploy
> > > > > > > >> pulsar
> > > > > > > >> > > > > instances on-prem and on multiple different cloud
> > > > providers
> > > > > > and
> > > > > > > at
> > > > > > > >> > the
> > > > > > > >> > > > same
> > > > > > > >> > > > > time they would like to enable replication between
> > > > multiple
> > > > > > > >> clusters
> > > > > > > >> > > > > deployed in different cloud providers. Pulsar
> already
> > > > > provides
> > > > > > > >> > various
> > > > > > > >> > > > > proxy options (Pulsar proxy/ enterprise proxy
> > solutions
> > > on
> > > > > > SNI)
> > > > > > > to
> > > > > > > >> > > > fulfill
> > > > > > > >> > > > > security requirements when brokers are deployed on
> > > > different
> > > > > > > >> security
> > > > > > > >> > > > zones
> > > > > > > >> > > > > connected with each other. However, sometimes it's
> not
> > > > > > possible
> > > > > > > to
> > > > > > > >> > share
> > > > > > > >> > > > > metadata-store (global zookeeper) between pulsar
> > > clusters
> > > > > > > >> deployed on
> > > > > > > >> > > > > separate cloud provider platforms, and synchronizing
> > > > > > > configuration
> > > > > > > >> > > > metadata
> > > > > > > >> > > > > (policies) can be a critical path to share
> > > > > > > tenant/namespace/topic
> > > > > > > >> > > > policies
> > > > > > > >> > > > > between clusters and administrate pulsar policies
> > > > uniformly
> > > > > > > across
> > > > > > > >> > all
> > > > > > > >> > > > > clusters. Therefore, we need a mechanism to sync
> > > > > configuration
> > > > > > > >> > metadata
> > > > > > > >> > > > > between clusters deployed on the different cloud
> > > > platforms.
> > > > > > > >> > > > >
> > > > > > > >> > > > > *Sync Pulsar policies across multiple clouds*
> > > > > > > >> > > > > https://github.com/apache/pulsar/issues/13728
> > > > > > > >> > > > > Prototype git-hub-link
> > > > > > > >> > > > > <
> > > > > > > >> > > >
> > > > > > > >> >
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/rdhabalia/pulsar/commit/e59803b942918076ce6376b50b35ca827a49bcf6
> > > > > > > >> > > > >
> > > > > > > >> > > > > Thanks,
> > > > > > > >> > > > > Rajan
> > > > > > > >> > > >
> > > > > > > >> >
> > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>