You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by Yubiao Feng <yu...@streamnative.io.INVALID> on 2022/11/10 06:39:41 UTC
[DISCUSS] Make the behavior of producers that use the mode WaitForExclusive clear
Hi all,
Now if an Exclusive-Mode producer has been registered, we can register
another WaitForExclusive-Mode producer. When the old producer goes
offline, the new producer will take over as the new exclusive producer.
< Highlight >But if some Shared-Mode producers have been registered, we can
register another WaitForExclusive-Mode producer too, but even if all
Shared-Mode goes offline, the new producer will not take over as the new
exclusive producer. Then, if we register a new WaitForExclusive-Mode
producer, the earlier WaitForExclusive-Mode producer will take over as the
new exclusive producer.
< Highlight >I want to make the behavior of producers that use the mode
WaitForExclusive clear: When no other mode of producer exists for a topic,
the first producer in the WaitForExclusive queue can auto succeeds as the
exclusive producer.
Thanks
Yubiao
Re: [DISCUSS] Make the behavior of producers that use the mode WaitForExclusive clear
Posted by Yubiao Feng <yu...@streamnative.io.INVALID>.
Hi Haiting
> I wonder what's the actual use case of mixing Exclusive-Mode and
Share-Mode of the same topic?
I think there might be a scenario like this:
Users used multiple producers in shared mode before, but need to change to
exclusive producers later.
> From what I see, we should make clear the priority of Exclusive-Mode and
Share-Mode, and Exclusive > Share makes more sense to me. Currently,
depending on the time of which producer registered first will make the
usage very confusing and hard to debug for users.
Yes.The simpler, the easier to use
Thanks
Yubiao
On Thu, Nov 10, 2022 at 4:10 PM Haiting Jiang <ji...@gmail.com>
wrote:
> Hi Yubiao
>
> I wonder what's the actual use case of mixing Exclusive-Mode and
> Share-Mode of the same topic?
>
> From what I see, we should make clear the priority of Exclusive-Mode
> and Share-Mode, and Exclusive > Share makes more sense to me.
> Currently, depending on the time of which producer registered first
> will make the usage very confusing and hard to debug for users.
>
> Thanks,
> Haiting
>
> On Thu, Nov 10, 2022 at 2:40 PM Yubiao Feng
> <yu...@streamnative.io.invalid> wrote:
> >
> > Hi all,
> >
> > Now if an Exclusive-Mode producer has been registered, we can register
> > another WaitForExclusive-Mode producer. When the old producer goes
> > offline, the new producer will take over as the new exclusive producer.
> >
> > < Highlight >But if some Shared-Mode producers have been registered, we
> can
> > register another WaitForExclusive-Mode producer too, but even if all
> > Shared-Mode goes offline, the new producer will not take over as the new
> > exclusive producer. Then, if we register a new WaitForExclusive-Mode
> > producer, the earlier WaitForExclusive-Mode producer will take over as
> the
> > new exclusive producer.
> >
> > < Highlight >I want to make the behavior of producers that use the mode
> > WaitForExclusive clear: When no other mode of producer exists for a
> topic,
> > the first producer in the WaitForExclusive queue can auto succeeds as the
> > exclusive producer.
> >
> > Thanks
> > Yubiao
>
Re: [DISCUSS] Make the behavior of producers that use the mode WaitForExclusive clear
Posted by Haiting Jiang <ji...@gmail.com>.
Hi Yubiao
I wonder what's the actual use case of mixing Exclusive-Mode and
Share-Mode of the same topic?
From what I see, we should make clear the priority of Exclusive-Mode
and Share-Mode, and Exclusive > Share makes more sense to me.
Currently, depending on the time of which producer registered first
will make the usage very confusing and hard to debug for users.
Thanks,
Haiting
On Thu, Nov 10, 2022 at 2:40 PM Yubiao Feng
<yu...@streamnative.io.invalid> wrote:
>
> Hi all,
>
> Now if an Exclusive-Mode producer has been registered, we can register
> another WaitForExclusive-Mode producer. When the old producer goes
> offline, the new producer will take over as the new exclusive producer.
>
> < Highlight >But if some Shared-Mode producers have been registered, we can
> register another WaitForExclusive-Mode producer too, but even if all
> Shared-Mode goes offline, the new producer will not take over as the new
> exclusive producer. Then, if we register a new WaitForExclusive-Mode
> producer, the earlier WaitForExclusive-Mode producer will take over as the
> new exclusive producer.
>
> < Highlight >I want to make the behavior of producers that use the mode
> WaitForExclusive clear: When no other mode of producer exists for a topic,
> the first producer in the WaitForExclusive queue can auto succeeds as the
> exclusive producer.
>
> Thanks
> Yubiao
回复：Re: [DISCUSS] Make the behavior of producers that use the mode WaitForExclusive clear
Posted by Baodi Shi <ba...@icloud.com.INVALID>.
Hi Yubiao:1. kick out all the connected Shared producers. This same as the mode ExclusiveWithFencing, so the mode WaitForExclusive is no longer needed.I agree this, but that doesn't mean we don't need `WaitForExclusive` . If there was a producer of Exclusive before, WaitForExclusive still has to wait.Baodi Shi
Thanks2022年11月10日下午4:39，Yubiao Feng <yu...@streamnative.io.INVALID> 写道：Hi EnricoI think that we should kick out all the connected Shared producers.The competition is among Exclusive producers and Shared don't play the game.Yes, Mode "ExclusiveWithFencing" does this. I think that WaitForExclusiveis a soft form of occupation. Just like the read and writer lock, thewriter takes the write lock after all readers released the read lock.This looks like a bug to me.I agree. If some shared producers already exist, then try to registera WaitForExclusive-Modeproducer. we have 3 ways:1. kick out all the connected Shared producers. This same as the modeExclusiveWithFencing, so the mode WaitForExclusive is no longer needed.2. just append the new producer to the waiting queue, and take over as thenew exclusive producer after all shared producers are offline.3. Reject the registration request of the new WaitForExclusive-Mode producer.there has been a pr attempt to do so(#18380).I prefer the second approach because it provides new usage and is closer tothe original design.On Thu, Nov 10, 2022 at 3:13 PM Enrico Olivelli <eo...@gmail.com> wrote:Yubiao,Il Gio 10 Nov 2022, 07:40 Yubiao Feng <yu...@streamnative.io.invalid>ha scritto:> Hi all,>> Now if an Exclusive-Mode producer has been registered, we can register> another WaitForExclusive-Mode producer. When the old producer goes> offline, the new producer will take over as the new exclusive producer.>> < Highlight >But if some Shared-Mode producers have been registered, wecan> register another WaitForExclusive-Mode producer too, but even if all> Shared-Mode goes offline, the new producer will not take over as the new> exclusive producer.This looks like a bug to me.When you start a xxxx-Exclusive producer you want to take control over thetopic.I think that we should kick out all the connected Shared producers.The competition is among Exclusive producers and Shared don't play thegame.The fix should be easyEnricoThen, if we register a new WaitForExclusive-Mode> producer, the earlier WaitForExclusive-Mode producer will take over asthe> new exclusive producer.>> < Highlight >I want to make the behavior of producers that use the mode> WaitForExclusive clear: When no other mode of producer exists for atopic,> the first producer in the WaitForExclusive queue can auto succeeds as the> exclusive producer.>> Thanks> Yubiao>
Re: [DISCUSS] Make the behavior of producers that use the mode WaitForExclusive clear
Posted by Yubiao Feng <yu...@streamnative.io.INVALID>.
Hi Enrico
> I think that we should kick out all the connected Shared producers.
The competition is among Exclusive producers and Shared don't play the game.
Yes, Mode "ExclusiveWithFencing" does this. I think that WaitForExclusive
is a soft form of occupation. Just like the read and writer lock, the
writer takes the write lock after all readers released the read lock.
> This looks like a bug to me.
I agree. If some shared producers already exist, then try to register
a WaitForExclusive-Mode
producer. we have 3 ways:
1. kick out all the connected Shared producers. This same as the mode
ExclusiveWithFencing, so the mode WaitForExclusive is no longer needed.
2. just append the new producer to the waiting queue, and take over as the
new exclusive producer after all shared producers are offline.
3. Reject the registration request of the new WaitForExclusive-Mode producer.
there has been a pr attempt to do so(#18380).
I prefer the second approach because it provides new usage and is closer to
the original design.
On Thu, Nov 10, 2022 at 3:13 PM Enrico Olivelli <eo...@gmail.com> wrote:
> Yubiao,
>
> Il Gio 10 Nov 2022, 07:40 Yubiao Feng <yubiao.feng@streamnative.io
> .invalid>
> ha scritto:
>
> > Hi all,
> >
> > Now if an Exclusive-Mode producer has been registered, we can register
> > another WaitForExclusive-Mode producer. When the old producer goes
> > offline, the new producer will take over as the new exclusive producer.
> >
> > < Highlight >But if some Shared-Mode producers have been registered, we
> can
> > register another WaitForExclusive-Mode producer too, but even if all
> > Shared-Mode goes offline, the new producer will not take over as the new
> > exclusive producer.
>
>
> This looks like a bug to me.
> When you start a xxxx-Exclusive producer you want to take control over the
> topic.
> I think that we should kick out all the connected Shared producers.
> The competition is among Exclusive producers and Shared don't play the
> game.
> The fix should be easy
>
> Enrico
>
>
> Then, if we register a new WaitForExclusive-Mode
> > producer, the earlier WaitForExclusive-Mode producer will take over as
> the
> > new exclusive producer.
> >
> > < Highlight >I want to make the behavior of producers that use the mode
> > WaitForExclusive clear: When no other mode of producer exists for a
> topic,
> > the first producer in the WaitForExclusive queue can auto succeeds as the
> > exclusive producer.
> >
> > Thanks
> > Yubiao
> >
>
Re: [DISCUSS] Make the behavior of producers that use the mode WaitForExclusive clear
Posted by Enrico Olivelli <eo...@gmail.com>.
Yubiao,
Il Gio 10 Nov 2022, 07:40 Yubiao Feng <yu...@streamnative.io.invalid>
ha scritto:
> Hi all,
>
> Now if an Exclusive-Mode producer has been registered, we can register
> another WaitForExclusive-Mode producer. When the old producer goes
> offline, the new producer will take over as the new exclusive producer.
>
> < Highlight >But if some Shared-Mode producers have been registered, we can
> register another WaitForExclusive-Mode producer too, but even if all
> Shared-Mode goes offline, the new producer will not take over as the new
> exclusive producer.
This looks like a bug to me.
When you start a xxxx-Exclusive producer you want to take control over the
topic.
I think that we should kick out all the connected Shared producers.
The competition is among Exclusive producers and Shared don't play the game.
The fix should be easy
Enrico
Then, if we register a new WaitForExclusive-Mode
> producer, the earlier WaitForExclusive-Mode producer will take over as the
> new exclusive producer.
>
> < Highlight >I want to make the behavior of producers that use the mode
> WaitForExclusive clear: When no other mode of producer exists for a topic,
> the first producer in the WaitForExclusive queue can auto succeeds as the
> exclusive producer.
>
> Thanks
> Yubiao
>